Jump to content




Compile And Run Lua

lua

  • You cannot reply to this topic
51 replies to this topic

#21 MudkipTheEpic

  • Members
  • 639 posts
  • LocationWhere you'd least expect it.

Posted 19 May 2013 - 09:03 PM

View Posttheoriginalbit, on 19 May 2013 - 08:46 PM, said:

View Postpingoleon60, on 19 May 2013 - 07:50 PM, said:

View Posttheoriginalbit, on 19 May 2013 - 07:50 AM, said:

os.queueEvent('bogus')
coroutine.yield()
its much faster than sleep(0)
It's also much faster than

View Postjesusthekiller, on 19 May 2013 - 05:24 PM, said:

or
os.queueEvent("derp")
os.pullEvent("derp")

Wouldn't queuing an empty string, "", be faster by any terms? (Computational, maybe?) Because in Java, a string is an array of chars, which in LuaJ (i think) get reconstructed into a lua string. So would the length matter?

#22 theoriginalbit

    Semi-Professional ComputerCrafter

  • Moderators
  • 7,332 posts
  • LocationAustralia

Posted 19 May 2013 - 11:07 PM

View PostMudkipTheEpic, on 19 May 2013 - 09:03 PM, said:

Wouldn't queuing an empty string, "", be faster by any terms? (Computational, maybe?) Because in Java, a string is an array of chars, which in LuaJ (i think) get reconstructed into a lua string. So would the length matter?
Its not actually the string that I'm saying is faster. Yes there may be slightly less overhead by using empty strings, but using coroutine.yield instead of sleep or os.pullEvent is where the real speed comes into effect. If you look at sleep, it calls os.pullEvent and keeps taking an event off the event queue waiting until it finds its timer. Then if you look at os.pullEvent it calls os.pullEventRaw into a table and then checks if it was a termination event and if it wasn't then returns the event table unpacked. Now if you look at os.pullEventRaw it quite simply returns the result of coroutine.yield, so in this case, why not just skip all the middle men and just call coroutine.yield, esp since we are only using it to prevent the yielding error.

#23 Pharap

  • Members
  • 816 posts
  • LocationEngland

Posted 20 May 2013 - 08:28 AM

View PostSammich Lord, on 19 May 2013 - 11:40 AM, said:

This is pretty cool. I have seen Xuma, IIRC, back on CCNet create one of these. But you can just easily decompile it with a tool such as this http://sourceforge.n...rojects/unluac/
True, but compiling achieves more than just obfuscating the code, it makes the code faster to load and generally smaller.

View Postjesusthekiller, on 19 May 2013 - 04:10 PM, said:

"Supporter of Closed Source"? *pukes*
I'm not here to argue over open source and closed source, that's not the point of compiling.
If you don't like it, you can bog off to another thread.

View PostNina, on 19 May 2013 - 04:52 PM, said:

There are quite some problems with this...
  • Closed-source/proprietary software (well except ComputerCraft (why is it closed-source anyway?)) really has no place here. Isn't this a community after all?
  • The sleep(0.1) trick is bad. Take any large program (I tried with LuaIDE) and attempt to compile and run it. Running LuaIDE like this would take 5954.9 seconds. That's around 1 hour and 40 minutes!
  • It is impossible to pass any arguments to programs executed with the supplied "Run" program.
  • Having a "Run" program in the first place is stupid -- you make programs compiled with this dependent on your "Run" program, while they might as well have the code that does this by themselves.
Did you actually bother to read any of the posts?
  • Firstly, people should have the right to have their programs compiled. Aside from allowing the ability to have closed source it introduces all the other benefits that were mentioned such as reducing code size and improving program loading times.
  • Secondly, yes, we've covered that. I did say I only threw these together in a few minutes and I would fix all issues later. Same goes for accepting extra arguments. Both of these things you could fix yourself by the way, it would take less time than complaining about it.
  • Lastly, the reason it needs run is because of the way the CraftOS shell runs programs, which isn't my fault. I believe we also mentioned how it's possible to swap out some of the shell code in exchange for the loadfile function provided by run as it will run non-compiled programs as well as compiled ones.


#24 jesusthekiller

  • Banned
  • 562 posts
  • LocationWrocław, Poland

Posted 20 May 2013 - 01:14 PM

View Posttheoriginalbit, on 19 May 2013 - 11:07 PM, said:

Its not actually the string that I'm saying is faster. Yes there may be slightly less overhead by using empty strings, but using coroutine.yield instead of sleep or os.pullEvent is where the real speed comes into effect. If you look at sleep, it calls os.pullEvent and keeps taking an event off the event queue waiting until it finds its timer. Then if you look at os.pullEvent it calls os.pullEventRaw into a table and then checks if it was a termination event and if it wasn't then returns the event table unpacked. Now if you look at os.pullEventRaw it quite simply returns the result of coroutine.yield, so in this case, why not just skip all the middle men and just call coroutine.yield, esp since we are only using it to prevent the yielding error.

Well, still got to learn about corountines.

View PostPharap, on 20 May 2013 - 08:28 AM, said:

View Postjesusthekiller, on 19 May 2013 - 04:10 PM, said:

"Supporter of Closed Source"? *pukes*
I'm not here to argue over open source and closed source, that's not the point of compiling.
If you don't like it, you can bog off to another thread.

I think you missed second half of my post...

#25 Lyqyd

    Lua Liquidator

  • Moderators
  • 8,465 posts

Posted 20 May 2013 - 01:39 PM

This is an interesting tool, but all programs for ComputerCraft posted to the forums need to be in plain Lua due to the rules regarding malicious code.

#26 nutcase84

  • Members
  • 711 posts
  • LocationIn My Lonely Little Computer Corner

Posted 20 May 2013 - 02:32 PM

View PostLyqyd, on 20 May 2013 - 01:39 PM, said:

This is an interesting tool, but all programs for ComputerCraft posted to the forums need to be in plain Lua due to the rules regarding malicious code.

Why not compiled?

#27 Sammich Lord

    IRC Addict

  • Members
  • 1,212 posts
  • LocationThe Sammich Kingdom

Posted 20 May 2013 - 02:52 PM

View Postnutcase84, on 20 May 2013 - 02:32 PM, said:

View PostLyqyd, on 20 May 2013 - 01:39 PM, said:

This is an interesting tool, but all programs for ComputerCraft posted to the forums need to be in plain Lua due to the rules regarding malicious code.

Why not compiled?
Because you can't tell if it is malicious or not. You give me a piece of code and I can pretty much tell what it's supposed to do. If you have compiled code you have no idea what it does until you run it.

#28 nutcase84

  • Members
  • 711 posts
  • LocationIn My Lonely Little Computer Corner

Posted 20 May 2013 - 02:53 PM

View PostSammich Lord, on 20 May 2013 - 02:52 PM, said:

View Postnutcase84, on 20 May 2013 - 02:32 PM, said:

View PostLyqyd, on 20 May 2013 - 01:39 PM, said:

This is an interesting tool, but all programs for ComputerCraft posted to the forums need to be in plain Lua due to the rules regarding malicious code.

Why not compiled?
Because you can't tell if it is malicious or not. You give me a piece of code and I can pretty much tell what it's supposed to do. If you have compiled code you have no idea what it does until you run it.

*DERP* Lol.

#29 M4sh3dP0t4t03

  • Members
  • 255 posts
  • LocationGermany

Posted 20 May 2013 - 03:01 PM

View PostSammich Lord, on 20 May 2013 - 02:52 PM, said:

View Postnutcase84, on 20 May 2013 - 02:32 PM, said:

View PostLyqyd, on 20 May 2013 - 01:39 PM, said:

This is an interesting tool, but all programs for ComputerCraft posted to the forums need to be in plain Lua due to the rules regarding malicious code.

Why not compiled?
Because you can't tell if it is malicious or not. You give me a piece of code and I can pretty much tell what it's supposed to do. If you have compiled code you have no idea what it does until you run it.
Except if you decompile it.

#30 Sammich Lord

    IRC Addict

  • Members
  • 1,212 posts
  • LocationThe Sammich Kingdom

Posted 20 May 2013 - 03:13 PM

View PostKingOfNoobs, on 20 May 2013 - 03:01 PM, said:

View PostSammich Lord, on 20 May 2013 - 02:52 PM, said:

View Postnutcase84, on 20 May 2013 - 02:32 PM, said:

View PostLyqyd, on 20 May 2013 - 01:39 PM, said:

This is an interesting tool, but all programs for ComputerCraft posted to the forums need to be in plain Lua due to the rules regarding malicious code.

Why not compiled?
Because you can't tell if it is malicious or not. You give me a piece of code and I can pretty much tell what it's supposed to do. If you have compiled code you have no idea what it does until you run it.
Except if you decompile it.
Which of course doesn't always work. Then let's say you have 50 different files and the main script pieces all 50 scripts together to form the script. That would make it very hard to check if it's malicious or not. Meaning, more people will get infected then if a moderator looks over the code and sees that it's malicious.

#31 Pharap

  • Members
  • 816 posts
  • LocationEngland

Posted 20 May 2013 - 09:23 PM

View Postjesusthekiller, on 20 May 2013 - 01:14 PM, said:

I think you missed second half of my post...

Complementing me doesn't exactly withdraw the former complaint.


View PostLyqyd, on 20 May 2013 - 01:39 PM, said:

This is an interesting tool, but all programs for ComputerCraft posted to the forums need to be in plain Lua due to the rules regarding malicious code.

You wouldn't be able to put it on the forum anyway, the text formatting would screw things up.
Unless you upload it as a file attached to a thread or something, which people never do with programs anyway.

Personally I'm still hunting for an alternative to dropbox to host anything in binary format.
I would use github, but I don't like github, don't like the clienty programs and you have to pay to have things private.

There's only two lua-based things that I wouldn't want open sourced anyway, which means that if the forum's not going to accept them, I'll find a way to discuss it with interested parties off the forums.

#32 ElvishJerricco

  • Members
  • 803 posts

Posted 20 May 2013 - 11:36 PM

Just FYI, when you loadstring() in Run.lua, you need to pass a second argument of fs.getName(inFile) so that error messages know the file name. Also, you can replace _G.loadfile with this version of loadfile and it works perfectly with both compiled code and uncompiled code. (yes I tested). The best part of that is that anything anywhere that loads code via os.run, shell.run, or anything else based on loadfile will run the compiled code just fine, so it even works from the shell without modifying the shell.

And finally, your os.queueEvent("") coroutine.yield() lines negate all the performance you're trying to save by running compiled code. Those two lines make the loading process very, very slow (again, tested). I had a three line program that when compiled and run with this, took like a half second to load (and that's not counting the compile time). I took those lines out and it was instant. You're probably not going to run into files so large that they reach that 5 second limit, so I'd say take the lines out.

#33 Grim Reaper

  • Members
  • 503 posts
  • LocationSeattle, WA

Posted 21 May 2013 - 01:48 AM

View PostNina, on 19 May 2013 - 04:52 PM, said:

  • It is impossible to pass any arguments to programs executed with the supplied "Run" program.

I'm pretty sure his code is already passing any additional arguments received from the shell off to the loaded function which is returned by loadfile().
loadfile(args[1])(select(2,unpack(args)))

However, you can simply tweak the run program yourself to allow for passing arguments to a compiled program. Considering that loadstring() is simply taking your string data which is a dump of actual Lua code, you can pass augments to the function returned.

Like the following:

local compiledProgram = loadfile (args[1])
table.remove (args, 1) -- Remove file path from arguments.

-- Run with arguments.
compiledProgram (unpack (args))


#34 Grim Reaper

  • Members
  • 503 posts
  • LocationSeattle, WA

Posted 21 May 2013 - 01:49 AM

View PostElvishJerricco, on 20 May 2013 - 11:36 PM, said:

You're probably not going to run into files so large that they reach that 5 second limit, so I'd say take the lines out.

He's right there, but if you still wanted to implement this safety feature, you might have a flag which is set only if the file size is large enough to exceed the five second limit.

#35 MudkipTheEpic

  • Members
  • 639 posts
  • LocationWhere you'd least expect it.

Posted 21 May 2013 - 09:00 AM

Maybe if it is still slow, only yield when i%50==0 in the loop.

Edit: Also for getting better file-size, use this function written by Espen.

Spoiler

Edited by MudkipTheEpic, 21 May 2013 - 09:07 AM.


#36 Pharap

  • Members
  • 816 posts
  • LocationEngland

Posted 21 May 2013 - 02:25 PM

View PostElvishJerricco, on 20 May 2013 - 11:36 PM, said:

Just FYI, when you loadstring() in Run.lua, you need to pass a second argument of fs.getName(inFile) so that error messages know the file name. Also, you can replace _G.loadfile with this version of loadfile and it works perfectly with both compiled code and uncompiled code. (yes I tested). The best part of that is that anything anywhere that loads code via os.run, shell.run, or anything else based on loadfile will run the compiled code just fine, so it even works from the shell without modifying the shell.

And finally, your os.queueEvent("") coroutine.yield() lines negate all the performance you're trying to save by running compiled code. Those two lines make the loading process very, very slow (again, tested). I had a three line program that when compiled and run with this, took like a half second to load (and that's not counting the compile time). I took those lines out and it was instant. You're probably not going to run into files so large that they reach that 5 second limit, so I'd say take the lines out.

Run.lua will only run one file at a time, if someone doesn't know what file they're trying to run when they've just typed it into the shell, I don't hold up much hope for their skills as a programmer. As for the function, yep, absolutely no error checking, but pcall can be used to handle that for now. It's early days yet, so a working function is all I can promise for now.

I would certainly expect swapping the loadfile functions to work. (The fact I named the function loadfile is no mere coincidence.) Up until now though I hadn't got round to testing, but as there is now confirmation, I can amend the documentation.

I could develop an option for size and all sorts of other enhancements, but frankly this code is barely 2 or 3 days old, so I'm not worried about optimising it yet. If you want to optimise it, feel free to, but otherwise I will optimise and improve it when I have the time.
(as it happens I do have a solution that will work without the need for any special command line options or testing the size of the file, but I will sort that when I get time)


View PostGrim Reaper, on 21 May 2013 - 01:49 AM, said:

View PostElvishJerricco, on 20 May 2013 - 11:36 PM, said:

You're probably not going to run into files so large that they reach that 5 second limit, so I'd say take the lines out.

He's right there, but if you still wanted to implement this safety feature, you might have a flag which is set only if the file size is large enough to exceed the five second limit.

It's not quite as simple as testing file size because some things take longer for the lexical parser to handle than others, plus a file could be absolutely crammed full of whitespace, which shouldn't really effect the compile time massively.

View PostGrim Reaper, on 21 May 2013 - 01:48 AM, said:

View PostNina, on 19 May 2013 - 04:52 PM, said:

  • It is impossible to pass any arguments to programs executed with the supplied "Run" program.

I'm pretty sure his code is already passing any additional arguments received from the shell off to the loaded function which is returned by loadfile().
loadfile(args[1])(select(2,unpack(args)))

However, you can simply tweak the run program yourself to allow for passing arguments to a compiled program. Considering that loadstring() is simply taking your string data which is a dump of actual Lua code, you can pass augments to the function returned.

Like the following:

local compiledProgram = loadfile (args[1])
table.remove (args, 1) -- Remove file path from arguments.

-- Run with arguments.
compiledProgram (unpack (args))

I won't deny here, Nina's comment was made before I made that change.
I considered the table remove thing, but I thought the solution I used looked neater and fit on fewer lines.
Many people forget about the select function, I thought it deserved some attention.

#37 ElvishJerricco

  • Members
  • 803 posts

Posted 21 May 2013 - 02:43 PM

View PostPharap, on 21 May 2013 - 02:25 PM, said:


...snip...


I've actually been working on a major overhaul of CraftOS lately, and I added your function (with credit given in the source, credits, and helpfiles) to _G and it works flawlessly. From my custom shell, or from the default shell, you can run a compiled program with no extra program. As for optimization, compiling can sometimes take too long, and so I do recommend using the coroutine.yield stuff for the compiler. But running is almost instant. I've compiled and run some pretty big files and they ran pretty much instantly.

#38 Pharap

  • Members
  • 816 posts
  • LocationEngland

Posted 21 May 2013 - 09:12 PM

View PostElvishJerricco, on 21 May 2013 - 02:43 PM, said:

I've actually been working on a major overhaul of CraftOS lately, and I added your function (with credit given in the source, credits, and helpfiles) to _G and it works flawlessly. From my custom shell, or from the default shell, you can run a compiled program with no extra program. As for optimization, compiling can sometimes take too long, and so I do recommend using the coroutine.yield stuff for the compiler. But running is almost instant. I've compiled and run some pretty big files and they ran pretty much instantly.

I'm currently working on my own OS as well, but it's much closer to Windows than CraftOS.
Not sure any more if I'm going to make it available to the forums for many reasons.

You'll be happy to know I just submitted a new version that might be faster, assuming an if statement and os.clock are faster than queue event and coroutine yielding.

#39 theoriginalbit

    Semi-Professional ComputerCrafter

  • Moderators
  • 7,332 posts
  • LocationAustralia

Posted 21 May 2013 - 11:37 PM

View PostPharap, on 21 May 2013 - 09:12 PM, said:

You'll be happy to know I just submitted a new version that might be faster, assuming an if statement and os.clock are faster than queue event and coroutine yielding.
Bug in the new version. The if statements are missing the () on os.clock.

#40 Grim Reaper

  • Members
  • 503 posts
  • LocationSeattle, WA

Posted 22 May 2013 - 01:30 AM

On your newest version, although I may be mistaken, you never check to see if the path provided is a directory. Nothing wrong with your compiling code itself, but it could crash the program if someone messed up or was deliberately trying to break your code.

Just a suggestion, though.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users