Main Content

Ruby - yield; parallel routines

Archive - Originally posted on "The Horse's Mouth" - 2010-10-01 21:34:49 - Graham Ellis

When you call a function / method in your program, you expect it to run to completion before returning control to you, passing back a result. For example, you might call a method to read a series of objects and store them into an array ... and then you could take that array of objects and, iterating through it, analyse each object in turn.

Such an approach works well if you've not got very many objects, but it's not clever if you have a lot. You're going to spend a lot of time - and fill up a lot of memory - storing the objects; perhaps it will lead to out-of-memory conditions, or "thrashing" of swap space as your application pages data on and off the disc. When you think about it, reading the data in is like feeding water into a reservoir, and if you keep doing it the dam holding the water overflows or bursts.

A better approach is to read in the information object by object, then manipulate them one at a time. Such code can be easily enough written, but the code's all going to be in a main program loop (or at least it will contain a series of calls at the application level to atomic routines that process one record at a time - a poor structure with things that should be internal to the data handlers being visible at the application level.

The solution offered in Ruby - and it's a neat one - it to use an iterator function / method - a method which yields a result as it runs, making a temporary return and carrying on when resumed. This is one aspect of what's sometimes referred to as parallelism.

I have added three examples to our web site in the last couple of days. A "shows you how" mechanism is [here]. There's an example that drip feeds data as if from a tap [here] (and that comparative analogy is a good one), and there's a final example [here] in which two routines are paralleled - the first feeding the data from the tap drip by drip, and the second drinking it one drop at a time.


The concept of iterators is also applied in Lua where they're known as coroutines, and in Python where they're generator functions.

Illustration - Public domain / Dam at Kaw Lake. [source]