42 replies to this topic

[ElvishJerricco]

A long long time ago, in a forum post far away, a user made a cool program, that could run Java from Lua. He called it JVML. I was immediately interested. As soon as he put his work on GitHub, I cloned the code and got to work. It was amazing that this was possible, but there was lots to do. Classes with native methods had to be built from the ground up in Lua, exceptions didn't work, interfaces didn't work. I could tell how difficult this was going to be.

Then one day someone called Yevano took interest. He wanted to boost the speed of JVML by turning it into a JIT, which compiles the Java bytecode into Lua bytecode for better performance. ds84182 had been basically absent from JVML since his initial work, so we decided to work on this exclusively in Yevano's repo. Since then, it has undergone some amazing work that has turned it into a very capable tool. I'm proud to announce JVML-JIT.

JVML-JIT

Install: pastebin run NzNwtBQy
Or: grin-get install Team-CC-Corp/JVML-JIT

Github: Team-CC-Corp/JVML-JIT

Usage

If you didn't use grin-get for installation, make sure you add jvml/bin/ to your shell's path at startup.

jvml MyMainClass
jvml -jar MyExecutableJarFile

JVML-JIT uses the notion of the classpath. By default, the cc_rt.jar file is included, and the current directory the shell is at is included. The -cp argument can be used to add to this.

Building for JVML-JIT

Although in many cases, it will work out to just compile your java code normally and run those class files, it runs some risks. Our runtime library is incomplete and the compiler might use methods and classes we don't have implemented. To be safe about this and avoid that problem, use the bootclasspath compiler option.

javac -bootclasspath jvml/CCLib/jar/cc_rt.jar MySource.java

How it Works

JVML-JIT works by loading class files, and compiling their methods into Lua. In an attempt to be more efficient, no method is compiled until the first time it's called. These compiled methods are now just ordinary Lua functions.

FAQ

• Can I run Minecraft in Minecraft?
  Sorry, but probably not. Minecraft makes use of extensive APIs that would take years to implement. That's just not going to happen.
  
• Can I run ______ in Minecraft?
  Sure, if you can get all the classes and natives it depends on ported over. Any additions to the runtime library would be appreciated in a pull request! =)
  
• Why is it so slow? I thought the JIT was supposed to speed it up!
  JVML-JIT will appear slow for small projects or projects that do many different things one time each. This is because compiling the methods to Lua is a bit of a slow process. Compiling roughly 60k lines of Lua assembly (which is a lot) takes the JIT about 15 seconds. Once a method is compiled, it's very, very quick. But if your program isn't using the same methods multiple times, you'll notice that you spend a lot of time compiling the variety of methods you use. Point is, JVML-JIT is better suited for larger projects that are meant to do long running, repetitive tasks. Perfect for a server!
  
• How can I contribute to help advance the runtime library?
  If there's any classes or methods you'd like to see implemented in the runtime, anyone is free to fork the repo, add the features in, and send us a pull request. As long as your change works, we'll probably take all the help we can get.
  
• Where do I report bugs and issues?
  Please direct any and all bugs and issues to JVML-JIT's issues page.

Special thanks to ds84182! Took all three of us to get this where it is today.

Edited by ElvishJerricco, 19 September 2015 - 05:13 AM.

[ElvishJerricco]

I've had a couple of people ask me how to let other people use the Java programs they've made, since you can't exactly pastebin a binary file. Really the only decent option out there right now is Grin or Grin-Get. But you should be using Git for your projects anyway

[Yevano]

The hope is that one day we'll be able to run javac inside JVML, so you can edit and compile your Java classes all inside ComputerCraft.

[JustPingo]

Yevano, on 10 October 2014 - 08:22 PM, said:

The hope is that one day we'll be able to run javac inside JVML, so you can edit and compile your Java classes all inside ComputerCraft.

Isn't what that actually do ?
Never mind.

That's awesome though, but I wonder if that's pratically usable...

Edited by pingoleon60, 10 October 2014 - 08:39 PM.

[ElvishJerricco]

pingoleon60, on 10 October 2014 - 08:38 PM, said:

Yevano, on 10 October 2014 - 08:22 PM, said:

The hope is that one day we'll be able to run javac inside JVML, so you can edit and compile your Java classes all inside ComputerCraft.

Isn't what that actually do ?
Never mind.

That's awesome though, but I wonder if that's pratically usable...

Getting javac is definitely a long term goal. It's a big thing. But it's definitely practical.

[JustPingo]

ElvishJerricco, on 10 October 2014 - 08:59 PM, said:

Getting javac is definitely a long term goal. It's a big thing. But it's definitely practical.

I meant :

JVML-JIT is awesome though, but I wonder if JVML-JIT is pratically usable...

Edited by pingoleon60, 10 October 2014 - 09:12 PM.

[ElvishJerricco]

pingoleon60, on 10 October 2014 - 09:11 PM, said:

JVML-JIT is awesome though, but I wonder if JVML-JIT is pratically usable...

Ah. Gotcha. It's still got a ways to go to be where we want it, but for now it's definitely usable. It's got peripheral, terminal, redstone, and event support among many other things. It can do a lot.

[JustPingo]

I'll test it and give you feedback, but if that works as good as you say it does, I will surely spotlight it in my french ComputerCraft tutorial serie, because that is completly awesome.

Edited by pingoleon60, 10 October 2014 - 09:33 PM.

[ds84182]

I disappear way too much... It isn't good for my health... Anyways, this is really cool, I didn't know a JIT would be possible !

[ElvishJerricco]

ds84182, on 11 October 2014 - 12:34 AM, said:

I disappear way too much... It isn't good for my health... Anyways, this is really cool, I didn't know a JIT would be possible !

Thanks =) Yea it's come a long way.

[Engineer]

Just omg!

Why didnt I find this sooner? I knew there was something like this out there written by ds, but it didnt seem practical at that point. You really got me motivated writing more programs, since I fell in love with Java for its OOP and classes implementation thing.. Just wow guys, keep it up!

You got yourself a bookmark to your repo, so I can check things while Im writing it

Edited by Engineer, 19 October 2014 - 08:33 PM.

[Engineer]

After creeping around for a while in the repo, it seems that one can only use the classes defined in JVML-JIT/CCLib/src/java. I get that everything should be added in some way in lua, but I don't get how.
If I can get some explanation on that subject, that would be very appreciated.
In return for that I will contribute by adding classes which I'm going to need for myself (im actually planning to use this for programs).

I get that you use native a lot, and you add your method to the natives table to make it function. You also have to pass arguments there as java bytecode string (if you follow me) and do some fancy stuff with toJString and all.
I really like this project, and I wish I had found it sooner than this. Thank you guys for creating it

[ElvishJerricco]

Engineer, on 20 October 2014 - 12:30 AM, said:

it seems that one can only use the classes defined in JVML-JIT/CCLib/src/java.

What do you mean by this? I have no troubles using classes in the cc package or from external classpaths.

[Engineer]

ElvishJerricco, on 20 October 2014 - 04:59 AM, said:

What do you mean by this? I have no troubles using classes in the cc package or from external classpaths.

I was speculating, and I meant all default java packages

Edit: after setting up a proper environment in my IDE, my speculation is correct. To explain myself more transparant with my previous post, let's say I want to use the FileSystem. Obviously, we want that to be through a java.io.File Object, however that class does not exist yet in JVML-JIT, we'd have to add it.

Obviously, I dont want to be a pain in the *** and ask you guys to implement (in this case) the File class. If I could do it myself, it will be easier for all of us, more classes get added and I can work through Java on cc projects!

Edited by Engineer, 20 October 2014 - 11:45 AM.

[Yevano]

Engineer, on 20 October 2014 - 12:30 AM, said:

After creeping around for a while in the repo, it seems that one can only use the classes defined in JVML-JIT/CCLib/src/java. I get that everything should be added in some way in lua, but I don't get how.
If I can get some explanation on that subject, that would be very appreciated.
In return for that I will contribute by adding classes which I'm going to need for myself (im actually planning to use this for programs).

I get that you use native a lot, and you add your method to the natives table to make it function. You also have to pass arguments there as java bytecode string (if you follow me) and do some fancy stuff with toJString and all.
I really like this project, and I wish I had found it sooner than this. Thank you guys for creating it

Right now natives loading is a bit wonky, but I'll explain as best I can.

First, you'll want to create your class with native methods. Let's call it NativeTest and place it in the package mypackage. We'll make a native method in it called repeat, which will repeat a string a certain number of times.

package mypackage.NativeTest;

public class NativeTest {
	public static native String repeat(String str, int rep);
}

Now, here's the wonky bit. Right now all runtime natives are loaded automatically from the java/lang/native folder. We just need to create a file called NativeTest.lua in there. This is just a convention though. Technically you can name the file anything you want, or even place your methods in other files.

--# This just creates the natives table for this class if it hasn't been created yet.
--# Note that the native Lua files share the environment with the whole JVML runtime.
natives["mypackage.NativeTest"] = natives["mypackage.NativeTest"] or { }

--# Here we add our method definition. If you've not done too much with JNI before, the string here is a method signature.
--# The method signature describes the name, return type, and argument types of the method.
--# In this case, it translates to String repeat(String, int)
natives["mypackage.NativeTest"]["repeat(Ljava/lang/String;I)Ljava/lang/String;"] = function(str, rep)
	--# Java String requires conversion to a Lua string.
	local luaStr = toLString(str)

	--# ints are already represented as normal Lua numbers. (Note that longs are represented as bigints.)
	luaStr = luaStr:rep(rep)

	--# Now convert back to the Java String object and return.
	return toJString(luaStr)
end

Now we can call that native like this.

import mypackage.NativeTest;

public class Tester {
	public static void main(String[] args) {
		System.out.println(NativeTest.repeat("lol", 20));
	}
}

Phew... At least it's easier than normal JNI.

A few tips:

• Make sure to run ant even after just editing the Lua files. They don't need to be compiled, but doing this will make ant add them to the runtime jar.
  
• The conversions between Lua and Java types are as follows
  
  • boolean, byte, short, int, and double are all represented as Lua numbers. boolean is either 0 or 1, not false or true.
    
  • long is represented as a bigint from the bigint.lua library. You shouldn't try converting them to normal Lua numbers unless you know what you're doing.
    
  • All Java objects (including strings) are represented as tables. These tables are numerically indexed to prevent doing hashtable lookups when possible. You can manipulate these objects in the following way:
    
    • getObjectField(obj, name) returns the stored value in the named field. *If speed is important, you can use the fieldIndexByName table field within the object's class to get the field index ahead of time, and then get the field using obj[2][fieldIndex].
      
    • setObjectField(obj, name, value) sets the named field. The above description of performance speedup applies here, too.
      
    • newInstance(class) returns a new instance of a Java class without calling any constructor. You need to remember to call the constructor on the object explicitly.
      
    • findMethod(class, name) searched and returns a method reference from the given class. I emphasize reference, because it is a table which contains the Lua function for calling the method, not the Lua functions itself. To call the method, do local ret = method[1](...).
  • JVML doesn't try to do conversion on tables passed into the runtime from natives or anything like that. Passing a raw table to a method will most likely crash the VM.

* This is done like this:

local fieldTable = obj[2]
local fieldIndex = cl.fieldIndexByName["myField"]
--# intensive loop
for i = 1, 1000 do
	fieldTable[fieldIndex] = someMethodA[1]()
	someMethodB[1](fieldTable[fieldIndex])
end

Hope this helped.

Edited by Yevano, 20 October 2014 - 03:03 PM.

[ElvishJerricco]

Yevano, on 20 October 2014 - 02:58 PM, said:

Now, here's the wonky bit. Right now all runtime natives are loaded automatically from the java/lang/native folder.

Actually, you can load natives from anywhere on the classpath. The ones in java/lang/native are loaded automatically, but there's some classes in the cc package that show that you can load natives at any place.

package mypackage;
public class NativeTest {
    public static native String rep(String s);

    static {
        System.load("path/in/classpath/to/my/natives.lua");
    }
}

[ElvishJerricco]

Updated. Changes:

• System.in
  
• Turtle API

[ElvishJerricco]

Updated: Bugfixes for Turtle and arithmetic

[ElvishJerricco]

Updated: Change in ownership to the new team Yevano and I have started.

Upcoming: New JIT backend. Should be significantly faster. Also that new backend will be put into its own framework for other people to use as JIT backends.

[Geforce Fan]

From Java to Lua to Java again...
...is what this is doing