Jump to content




Grin - Github Release INstaller


  • You cannot reply to this topic
37 replies to this topic

#21 MarioBG

  • Members
  • 43 posts
  • LocationSpain (or Hisperia)

Posted 17 December 2014 - 04:56 PM

View PostElvishJerricco, on 17 December 2014 - 04:17 PM, said:

View PostMarioBG, on 17 December 2014 - 03:01 PM, said:

-snip-

That tool you're using prefixes the output with
data:;base64,
Make sure you're only copying what comes after that comma. I believe that is what's causing your error.


I know that, and have only copied the string of chars from the comma onwards. I have successfully downloaded the thing in ccemuredux, which uses some 1.6 ComputerCraft, but not in the old CC emulator, which uses 1.45, so there must be some kind of issue there. Again, thanks for your patience :D

Edited by MarioBG, 17 December 2014 - 04:57 PM.


#22 Geforce Fan

  • Members
  • 846 posts
  • LocationMissouri, United States, America, Earth, Solar System, Milky Way, Universe 42B, Life Street, Multiverse, 4th Dimension

Posted 16 February 2015 - 12:03 AM

This program makes me grin

#23 FUNCTION MAN!

  • Members
  • 292 posts

Posted 16 February 2015 - 10:56 AM

View PostGeforce Fan, on 16 February 2015 - 12:03 AM, said:

This program makes me grin

This post made me cringe.

#24 Geforce Fan

  • Members
  • 846 posts
  • LocationMissouri, United States, America, Earth, Solar System, Milky Way, Universe 42B, Life Street, Multiverse, 4th Dimension

Posted 18 February 2015 - 12:06 AM

View PostDr. Poof, on 16 February 2015 - 10:56 AM, said:

View PostGeforce Fan, on 16 February 2015 - 12:03 AM, said:

This program makes me grin

This post made me cringe.
Haters gonna hate.

#25 Jarle212

  • Members
  • 198 posts
  • LocationNorway

Posted 19 February 2015 - 04:11 PM

You could try to open files in luaJ in a binary format, by opening the files as txt files then convert every character to an integer value using string.byte(char), if this was the problem and there isn't a problem sending the binary files trough http.

Edited by Jarle212, 19 February 2015 - 04:46 PM.


#26 KingofGamesYami

  • Members
  • 3,002 posts
  • LocationUnited States of America

Posted 19 February 2015 - 05:16 PM

View PostJarle212, on 19 February 2015 - 04:11 PM, said:

You could try to open files in luaJ in a binary format, by opening the files as txt files then convert every character to an integer value using string.byte(char), if this was the problem and there isn't a problem sending the binary files trough http.

Why not just open the file in rb (read binary) mode?

#27 Jarle212

  • Members
  • 198 posts
  • LocationNorway

Posted 19 February 2015 - 06:22 PM

View PostKingofGamesYami, on 19 February 2015 - 05:16 PM, said:

View PostJarle212, on 19 February 2015 - 04:11 PM, said:

You could try to open files in luaJ in a binary format, by opening the files as txt files then convert every character to an integer value using string.byte(char), if this was the problem and there isn't a problem sending the binary files trough http.

Why not just open the file in rb (read binary) mode?
Because with binary mode you can't use readAll, but should still work. Not certain what hindrance the luaJ bug poses tho.

#28 ElvishJerricco

  • Members
  • 803 posts

Posted 19 February 2015 - 07:23 PM

View PostJarle212, on 19 February 2015 - 04:11 PM, said:

You could try to open files in luaJ in a binary format, by opening the files as txt files then convert every character to an integer value using string.byte(char), if this was the problem and there isn't a problem sending the binary files trough http.

Binary files ought to be dealt with with "rb" mode. That said, there is a bug with LuaJ that breaks any binary data being converted from java Strings to Lua strings. So an HTTP request has to go through that conversion, breaking any binary downloads.

#29 Jarle212

  • Members
  • 198 posts
  • LocationNorway

Posted 19 February 2015 - 08:50 PM

View PostElvishJerricco, on 19 February 2015 - 07:23 PM, said:

View PostJarle212, on 19 February 2015 - 04:11 PM, said:

You could try to open files in luaJ in a binary format, by opening the files as txt files then convert every character to an integer value using string.byte(char), if this was the problem and there isn't a problem sending the binary files trough http.

Binary files ought to be dealt with with "rb" mode. That said, there is a bug with LuaJ that breaks any binary data being converted from java Strings to Lua strings. So an HTTP request has to go through that conversion, breaking any binary downloads.
Only breaks binary data where the character value is greater than 127(0x7F), at the place where the ascii table ends.
Edit: There seems to be a pattern, so it should be possible to fix the broken string.

Edited by Jarle212, 19 February 2015 - 10:18 PM.


#30 ElvishJerricco

  • Members
  • 803 posts

Posted 19 February 2015 - 10:33 PM

View PostJarle212, on 19 February 2015 - 08:50 PM, said:

View PostElvishJerricco, on 19 February 2015 - 07:23 PM, said:

View PostJarle212, on 19 February 2015 - 04:11 PM, said:

You could try to open files in luaJ in a binary format, by opening the files as txt files then convert every character to an integer value using string.byte(char), if this was the problem and there isn't a problem sending the binary files trough http.

Binary files ought to be dealt with with "rb" mode. That said, there is a bug with LuaJ that breaks any binary data being converted from java Strings to Lua strings. So an HTTP request has to go through that conversion, breaking any binary downloads.
Only breaks binary data where the character value is greater than 127(0x7F), at the place where the ascii table ends.
Edit: There seems to be a pattern, so it should be possible to fix the broken string.

Right and binary data quite often uses bytes over 127. And whatever pattern there may be can't be fixed because any byte that might be corrupted, also might not. There isn't any way to tell.

#31 Jarle212

  • Members
  • 198 posts
  • LocationNorway

Posted 19 February 2015 - 10:52 PM

View PostElvishJerricco, on 19 February 2015 - 10:33 PM, said:

View PostJarle212, on 19 February 2015 - 08:50 PM, said:

View PostElvishJerricco, on 19 February 2015 - 07:23 PM, said:

View PostJarle212, on 19 February 2015 - 04:11 PM, said:

You could try to open files in luaJ in a binary format, by opening the files as txt files then convert every character to an integer value using string.byte(char), if this was the problem and there isn't a problem sending the binary files trough http.

Binary files ought to be dealt with with "rb" mode. That said, there is a bug with LuaJ that breaks any binary data being converted from java Strings to Lua strings. So an HTTP request has to go through that conversion, breaking any binary downloads.
Only breaks binary data where the character value is greater than 127(0x7F), at the place where the ascii table ends.
Edit: There seems to be a pattern, so it should be possible to fix the broken string.

Right and binary data quite often uses bytes over 127. And whatever pattern there may be can't be fixed because any byte that might be corrupted, also might not. There isn't any way to tell.
There is :P
if the char has a value of 194 or 195 the correct byte value will be between 0xA0 and 0xFF, by looking at the next character after that you can figure out what the original value was.
So let take: 0xF2 witch produces the two values 195 and 178
195 in binary: 1100 0011
178 in binary: 1011 [0010] <-- this will be the lower 4 bits of the original so we got xxxx 0010
to get the 4 higher bits of the original byte we take the lower two bits of 195 like this -> 1100 00[11] put them in the 5th and 6th position so we got xx11 0010
to get the last two bits take the 5th and 6th bits of 178 like this -> 10[11] 0010 and put them in the 7th and 8th position and then we get 1111 0010 witch is the value of 0xF2.
This method works for bytes where the value is between 0xA0 and 0xFF. I haven't figured out the pattern for values between 0x80 and 0x9F yet.

Here is the code I got so far:
function fixString(brokenString)
local fixedString = "";
local bVal;
for i=1, brokenString:len() do
  bVal = string.byte(brokenString:sub(i,i))
  if bit.band(0x80,bVal) == 0 then
   --Nothing to fix
   fixedString = fixedString .. brokenString:sub(i,i);
  else
   --Fix broken string
   if (bVal == 195) or (bVal == 194) then
	--Value between 0xA0 and 0xFF
	i = i + 1;
	local bVal2 = string.byte(brokenString:sub(i,i));
	local hi = bit.bor(bit.blshift(bit.band(bVal,3),6),bit.band(bVal2,48));
	local lo = bit.band(bVal2,15);
	local fixed = bit.bor(hi,lo);
	fixedString = fixedString .. string.char(fixed);
   else
	--Value is 0x8X or 0x9X
   end
  end
end
return fixedString;
end

Edited by Jarle212, 19 February 2015 - 10:55 PM.


#32 ElvishJerricco

  • Members
  • 803 posts

Posted 20 February 2015 - 01:24 AM

View PostJarle212, on 19 February 2015 - 10:52 PM, said:

- snip -

Oh wow I never really looked all that deeply into this. It would appear that this is a Unicode issue. I don't know much about Unicode, but it seems that when making a String from a byte[] in Java, it tries to see the values >127 as unicode values, and messes with the string entirely, leaving behind a trailing byte or two from which you can extrapolate the original data.

So while now it appears we have a way to fix this on Lua side, I would guess that HTTP requests in CC itself can be changed from reading the response as a String, to reading the response as a byte[]. Then it can construct its own char[], one char for each byte, and produce a correct String from that char[].

Of course without CC being open source, we can't know if CC already does this or not, and we can't submit a pull request fixing the bug...

EDIT: Then again, the issue exists when calling Java code from Lua, passing a string. I would have to look into this, but I'd guess a Lua string is stored in a byte[], and LuaJ is using the direct byte[] → String method of converting a Lua string to a Java String. Of course this is basically speculation, but maybe I'll look into this...

Edited by ElvishJerricco, 20 February 2015 - 01:27 AM.


#33 Jarle212

  • Members
  • 198 posts
  • LocationNorway

Posted 20 February 2015 - 01:29 AM

View PostElvishJerricco, on 20 February 2015 - 01:24 AM, said:

View PostJarle212, on 19 February 2015 - 10:52 PM, said:

- snip -

Oh wow I never really looked all that deeply into this. It would appear that this is a Unicode issue. I don't know much about Unicode, but it seems that when making a String from a byte[] in Java, it tries to see the values >127 as unicode values, and messes with the string entirely, leaving behind a trailing byte or two from which you can extrapolate the original data.

So while now it appears we have a way to fix this on Lua side, I would guess that HTTP requests in CC itself can be changed from reading the response as a String, to reading the response as a byte[]. Then it can construct its own char[], one char for each byte, and produce a correct String from that char[].

Of course without CC being open source, we can't know if CC already does this or not, and we can't submit a pull request fixing the bug...

EDIT: Then again, the issue exists when calling Java code from Lua, passing a string. I would have to look into this, but I'd guess a Lua string is stored in a byte[], and LuaJ is using the direct byte[] → String method of converting a Lua string to a Java String. Of course this is basically speculation, but maybe I'll look into this...
Cool :>

Edited by Jarle212, 20 February 2015 - 01:29 AM.


#34 ElvishJerricco

  • Members
  • 803 posts

Posted 20 February 2015 - 01:36 AM

Ok so looking at LuaString.java, we find these methods.

	public static LuaString valueOf(String string) {
		LuaString s = index_get( index_java, string );
		if ( s != null ) return s;
		char[] c = string.toCharArray();
		byte[] b = new byte[lengthAsUtf8(c)];
		encodeToUtf8(c, b, 0);
		s = valueOf(b, 0, b.length);
		index_set( index_java, string, s );
		return s;
	}

	public String tojstring() {
		return decodeAsUtf8(m_bytes, m_offset, m_length);
	}

	public static String decodeAsUtf8(byte[] bytes, int offset, int length) {
		int i,j,n,b;
		for ( i=offset,j=offset+length,n=0; i<j; ++n ) {
			switch ( 0xE0 & bytes[i++] ) {
			case 0xE0: ++i;
			case 0xC0: ++i;
			}
		}
		char[] chars=new char[n];
		for ( i=offset,j=offset+length,n=0; i< j; ) {
			chars[n++] = (char) (
				((b=bytes[i++])>=0||i>=j)? b:
				(b<-32||i+1>=j)? (((b&0x3f) << 6) | (bytes[i++]&0x3f)):
					(((b&0xf) << 12) | ((bytes[i++]&0x3f)<<6) | (bytes[i++]&0x3f)));
		}
		return new String(chars);
	}
	public static void encodeToUtf8(char[] chars, byte[] bytes, int off) {
		final int n = chars.length;
		char c;
		for ( int i=0, j=off; i< n; i++ ) {
			if ( (c = chars[i]) < 0x80 ) {
				bytes[j++] = (byte) c;
			} else if ( c < 0x800 ) {
				bytes[j++] = (byte) (0xC0 | ((c>>6)  & 0x1f));
				bytes[j++] = (byte) (0x80 | ( c      & 0x3f));				
			} else {
				bytes[j++] = (byte) (0xE0 | ((c>>12) & 0x0f));
				bytes[j++] = (byte) (0x80 | ((c>>6)  & 0x3f));
				bytes[j++] = (byte) (0x80 | ( c      & 0x3f));				
			}
		}
	}

It would appear I'm correct. But there's also this method

	public static LuaString valueOf(byte[] bytes) {
		return valueOf(bytes, 0, bytes.length);
	}

So CC can encode and decode to and from byte arrays just fine. Might make for some tedious code but it'd fix the bug.

EDIT: Also, Jarle212 you can probably use the knowledge of how these methods work to your advantage in figure out how to fix it Lua side.

Edited by ElvishJerricco, 20 February 2015 - 01:38 AM.


#35 Jarle212

  • Members
  • 198 posts
  • LocationNorway

Posted 20 February 2015 - 01:43 AM

View PostElvishJerricco, on 20 February 2015 - 01:36 AM, said:

-snip-
Yeah, I'm looking into it. Might post this as an "api" too, because it is the same problem when you are using rednet :)
Edit: need some sleep first tho.
Edit2: Lookin at the decodeAsUtf8 method you could probably just convert that to lua script and it should work. And the offset part seems to be correct too. Characters between 0xA0 and 0xFF starts with 195 or 194 and both of them can be written as: 0xCX while most of the characters between 0x80 and 0x9F got 0xEX at the beginning of the first character.
Edit3: Nvm, ^ didn't work like that :/

Edit4: Hit a problem, 0x90 and 0x81 is conflicting, so think this has to be solved in Java

Edited by Jarle212, 21 February 2015 - 07:37 PM.


#36 Creator

    Mad Dash Victor

  • Members
  • 2,168 posts
  • LocationYou will never find me, muhahahahahaha

Posted 27 February 2015 - 07:14 PM

Hi,

when I type grin-get install myusername/repo, it returns grin-install.lua: 30: no releases found.

Can you help me out because I don't understand how it works.

~Creator

#37 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 28 February 2015 - 09:24 AM

View PostCreator, on 27 February 2015 - 07:14 PM, said:

Hi,

when I type grin-get install myusername/repo, it returns grin-install.lua: 30: no releases found.

Can you help me out because I don't understand how it works.

~Creator

Files need to be provided at GitHub releases. So If you look at ElvishJerricco/ClamShell you can see there is a clamshell.zip.base64 release. This is a zip of all the files base 64 encoded. You can automate this with a programming language though in the case of ClamShell it uses an Ant buildscript.
  • Zip your code
  • Base 64 encode it. You can run base64 --wrap=0 build.zip > build.zip.base64 on Linux.
  • On GitHub, go to releases, add a new one and add your build.zip.base64 file.


#38 ElvishJerricco

  • Members
  • 803 posts

Posted 01 March 2015 - 02:40 PM

View PostSquidDev, on 28 February 2015 - 09:24 AM, said:

- snip -

Thanks for explaining this for everyone =) It should be noted that the if_windows task in that ant script doesn't work as I expected it to, and should probably just be removed.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users