Jump to content




BB's Guide to Coroutines

help lua

11 replies to this topic

#1 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,027 posts
  • LocationTasmania (AU)

Posted 15 January 2016 - 02:50 PM

BB's First Law of Coroutines: You probably don't need to be using coroutines, whatever it is you think you need them for.

BB's Second Law of Coroutines: You probably don't understand coroutines as well as you think you do - the Dunning-Kruger effect applies here more often than not.

BB's Third Law of Coroutines: In those cases where you do need to use coroutines, the parallel API can probably handle them for you.

I didn't really want to write this, because it's a rather complex topic and I can get rather verbose even on simple ones. However, people keep trying to produce code based around these things and I suppose it's either this or I keep on typing out the same hints over and over (most of which seem to consist of "you don't need to be using coroutines for whatever it is you're trying to do", but somehow that one never seems to get through). While this tutorial covers much of the same material (and I suggest that you read it), I figure another one couldn't hurt.

First off: You probably don't need to know any of this. Most scripts do not require an understanding of coroutines to write. Fewer need to manage them. Those that do, can probably have their needs met by the pre-written coroutine manager available in the parallel API. That leaves a very, very small area of "scripts that need to manually manage coroutines themselves - like writing a multi-tasking shell. And I find shells boring. Please stop writing them.

Even though you're still reading and have thus far clearly ignored what I've told you, I'd at least like to imagine that, prior to going any further, you've covered the first nine topics from this directory: LuaTypesTutorial through to the ScopeTutorial. I'll be re-covering the relevant bits anyway, as even if you have read through them (and ideally, put the knowledge they gave you to the test), it's important that the basic concepts that lead up to this one are fully understood.

And despite the massive walls of text that come below (compared to which the wall of text that is this intro pales; I said I was verbose!), bear in mind that I am not a professional coder. I'm merely a hobbyist. I'm also leaving some things out, intentionally; long story short, there's always something left to learn, and that'll still be true even if you absorb everything here. For starters, learning how coroutines work in relation to Lua isn't going to teach you how threading works in other languages. It may help lead you into it. That's true of a lot of things in Lua, I suppose.

With that all said, if you can wrap your head around all this stuff you'll hopefully be better off as a ComputerCraft coder (whether you ever put it to practical use or not) - you'll have a deeper understanding as to how all scripts run, whether they attempt to take control of coroutines or not. I've tried to keep things simple, so even beginners should be able to follow and learn something (assuming they read the aforementioned tutorials - table knowledge is a must!). So, let's have at it...

Chapter One: Functions

Chapter Two: Basic Coroutine Usage

Chapter Three: Yielding

Chapter Four: ComputerCraft & Events

Chapter Five: Multitasking


#2 Creator

    Mad Dash Victor

  • Members
  • 2,168 posts
  • LocationYou will never find me, muhahahahahaha

Posted 15 January 2016 - 04:07 PM

This guide is amazing! Thanks Bomb Bloke.

Stuff like this is very, very gut!

#3 KingofGamesYami

  • Members
  • 2,978 posts
  • LocationUnited States of America

Posted 15 January 2016 - 08:33 PM

I'll upvote for the comprehensive tutorial.

I'm bookmarking this for people who think an OS is a good first project.

#4 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,027 posts
  • LocationTasmania (AU)

Posted 16 January 2016 - 11:24 AM

Thanks guys. :)

View PostKingofGamesYami, on 15 January 2016 - 08:33 PM, said:

I'm bookmarking this for people who think an OS is a good first project.

Heh, if that doesn't talk them out of it, nothing will. ;)

And I was tempted to make it longer...

#5 Zambonie

  • Members
  • 477 posts
  • LocationSpring Hill, Fl

Posted 17 January 2016 - 01:22 AM

Super useful, will be sure to look at this again. +1

#6 wilcomega

  • Members
  • 466 posts
  • LocationHolland

Posted 19 January 2016 - 01:55 PM

co routines are good for running multiple servers on a single machine or for being the party above the program to run, so you can pass it the events you want it to have and you can detect diffrent types if yields,
if you understand them along with Environments then you can do some really cool stuff, and i know it ALL!! wooo!

#7 cyanisaac

  • Members
  • 369 posts
  • LocationSan Diego, CA

Posted 19 January 2016 - 06:09 PM

Quote

And I find shells boring. Please stop writing them.

Nah.

But on a serious note, good tutorial. It's very informative, and I think it will really help out people like me who didn't know how to use them.

EDIT:

View PostBomb Bloke, on 16 January 2016 - 11:24 AM, said:

Thanks guys. :)

View PostKingofGamesYami, on 15 January 2016 - 08:33 PM, said:

I'm bookmarking this for people who think an OS is a good first project.

Heh, if that doesn't talk them out of it, nothing will. ;)

And I was tempted to make it longer...

Haha, OpenTerminalOS was my first project. I ended up using the parallel API to do multitasking. It mostly worked, as your third law dictates.

Edited by cyanisaac, 19 January 2016 - 06:17 PM.


#8 SquidDev

  • Members
  • 1,323 posts
  • LocationDoes anyone put something serious here?

Posted 21 January 2016 - 04:33 PM

This is one of the best descriptions I've read on coroutines. A couple of other things I'd mention though:

Generators
Many other languages have the concept of "generators": a function that produces something that can be iterated through. You can achieve this with Lua using coroutine.yield and coroutine.wrap

local function chars(str)
  return coroutine.wrap(function()
	for i = 1, #str do
	  coroutine.yield(str:sub(i, i))
	end
  end)
end

for char in chars("HELLO") do
	print(char)
end

This is incredibly useful when it comes to recursive functions, as you can yield from inside them too.

Avoiding coroutines
A common pattern for downloading several files would be to handle each download in a separate coroutine. However one must note that each coroutine is a separate Java thread. There is a limit to how many threads the JVM can handle before giving up the ghost (This seems to vary between 2000 and 30000). Whilst you are unlikely to hit this limit, it is always something worth considering avoid the parallel API and using os.pullEvent then delegating to another method.

Edited by SquidDev, 21 January 2016 - 04:34 PM.


#9 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,027 posts
  • LocationTasmania (AU)

Posted 22 January 2016 - 02:56 AM

View PostSquidDev, on 21 January 2016 - 04:33 PM, said:

Many other languages have the concept of "generators": a function that produces something that can be iterated through. You can achieve this with Lua using coroutine.yield and coroutine.wrap

Yes, coroutine.wrap() is to coroutine.resume() much as peripheral.wrap() is to peripheral.call().

With your particular example, and at least within ComputerCraft, you get significantly better performance without the coroutine. Eg:

local function chars(str)
  local i = 0  
  return function()
        if i == #str then return nil end
        i = i + 1
        return str:sub(i, i)
  end
end

for char in chars("HELLO") do
        print(char)
end

Building the iterator takes about the same amount of time, but its execution speeds up no end. The technique is still useful if you would otherwise need to move a ton of variables outside of their ideal scope, though (that is to say - it allows you to be lazy!).

View PostSquidDev, on 21 January 2016 - 04:33 PM, said:

There is a limit to how many threads the JVM can handle before giving up the ghost (This seems to vary between 2000 and 30000).

Mind you, there's another limitation you'll run into long before the thread cap, and that's the size of the event queue.

Each system has an associated array which ComputerCraft populates with the events it'll use to resume it with. New events go on the end, and old events are removed from the front. This is why functions like sleep() cause you to "miss" other events - they keep pulling events from the front of the queue (the only place they can be taken from) until they get the one they want.

The event queue holds a maximum of 256 events at a time. If you allow more than that to queue up without constantly pulling events out to make room, your script won't automatically crash, but you will start losing events.

So let's say you start a thousand coroutines, and each makes an http.get() call. That function works by yielding until it gets an event (eg "http_success"). You'll find those events flood into your queue far faster than you can distribute them to their coroutines, and hey presto... your script stalls, forever yielding waiting for events that have occurred but were never able to fit into the queue.

You might make a coroutine that doesn't care what events it gets back, and yields merely to avoid the 10 second "yield protection" system. It can be tempting to simply have it call os.queueEvent() with some random data, and then yield without a filter (simply so that you know your code'll be resumed as soon as possible).

But that isn't safe, either - if something else on the system is generating events at about the same rate, then you run the risk of eventually flooding the queue with your "garbage" events (as you'll only be pulling events at half the rate they're entering the queue)! Therefore, when I need to crunch a lot of data without otherwise having any use for events, I yield like so:

local function snooze()
	local myEvent = tostring({})
	os.queueEvent(myEvent)
	os.pullEvent(myEvent)
end

This still resumes nearly instantly while preventing queue flood. Using a static value for "myEvent" is technically faster again, but will lead to problems if you try to run multiple instances of the same script via eg multishell.

Why not just sleep(0), you may ask? Well, sleep() works by creating a timer and waiting for the associated event... and timer events are always generated on the server tick after after os.startTimer() was called, at the earliest. Given that server ticks take a twentieth of a second, this is far slower than simply throwing an event into the queue straight away and pulling that.

#10 Lyqyd

    Lua Liquidator

  • Moderators
  • 8,436 posts

Posted 22 January 2016 - 05:02 AM

I also use the tostring-a-table method for generating unique parameters for yielding like that, though I also use a human-readable event type name with it. It's not quite as fast, but it is much easier to figure out where it's coming from if a human is examining the event stream for whatever reason.

#11 Windows10User

  • Members
  • 53 posts
  • LocationC:\Minecraft\saves\stuff\computer

Posted 26 May 2018 - 02:02 PM

Great tutorial but I don't understand, what do coroutines even do in OSes?!

#12 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,027 posts
  • LocationTasmania (AU)

Posted 26 May 2018 - 02:07 PM

Generally, the only reason you'd want to use them within an "OS" is so that your users can run multiple scripts side-by-side. For example, multishell uses one coroutine for every tab you open.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users