Main Content

Co-routines in Lua - co-operative processing

Archive - Originally posted on "The Horse's Mouth" - 2008-06-29 17:21:09 - Graham Ellis

CoRoutines in Lua allow you to have a number of pieces of co-operative code which (whilst they’re not running in parallel in a true multithreaded way) allow you to progress through multiple strands of your code at the sametime.

Let’s take an example. I have a rather large text file that lists out all the railway stations in the British Isles, together with the number of passengers joining the railway and leaving the railway there, and various other miscellaneous bits of data, and I want to iterate through the data processing it.

The data format is rather ugly - sometimes the numbers are quoted and sometimes not, and there are commas in the numbers over 999 too. Spaces and tabs mean different things, and zero is actually represented by "-". I would love to write code that’s a simple loop that says "get station" as a function call, which would read from the file and process and return the next line ... but there are all sorts of issues with static variables that would need to be considered.

Lua’s "coroutine"s can be created, then resumed (until they yield) time and again, so that in effect using them is like walking up to a coffee machine and pressing the button for the next cup ... which will keep coming while there’s coffee in the machine.

Here’s a snippet of code to call a coroutine ...

instream = coroutine.create(recordfeeder)
while coroutine.status(instream) ~= "dead" do
  coroutine.resume(instream)
  if fields then
    -- process those fields
    end
  end

Yes, it does look very similar to Java’s iterators, or Python’s generators.

So why don’t I loop through all the records in my incoming file and produce a table of station objects, then write another loop to handle them? Because I don’t really want or need to have 2520 station objects or tables from Waterloo (88,219,856 passengers per year) to Tyndrum Lower (17 passengers in a year) in memory after reading through my data - I want to filter on the fly. Rather than filling a reservoir with water (and having to build it big enough) then draining all the water back out, I prefer to have a pipe with a tap which I can turn on when I need the next record.

The Complete source code for the example I'm talking about here is available here on our web site