Jump to content




Reading bytes


42 replies to this topic

#1 emanuel12324

  • Members
  • 28 posts

Posted 19 July 2018 - 09:32 PM

Hey guys,
I'm working on a project where I'm trying to read some data out of a png, and I'm having some issues. I'm adapting a lot of code from others and kind of mixing it together, and one of the issues I have is actually reading data from the png file. Since it's binary, it presents some difficulties. Here is the code I'm trying, but it will go through and read bytes up until it errors saying "attempt to perform arithmatic __mult on nil and number.

local function readByte(stream)
	return stream.read()
end
local function readInt(stream, bps)
	local bytes = {}
	bps = bps or 4
	for i=1,bps do
		bytes[i] = readByte(stream)
	end
	return bytesToNum(bytes)
end
local function readChar(stream, num)
	num = num or 1
	local bytes = 0
for i = 0, num - 1, 1 do
  bytes = bytes + (readByte(stream) * (256 ^ i))
end
end

So it seems that readByte is giving readChar a nil for one of the bytes. Obviously I'm doing something wrong. BTW this code is adapted from SquidDev's bitmap library. The other difficulty I am having is downloading binary files. This one is a creation of my own, and it works well for downloading binary, but when it downloads a png and I try to open it with Windows Photo Viewer, it says that it's missing or corrupt. Thoughts?

function downloadBinaryFile(url, saveFile)
local httpHandle = http.get(url)
if not httpHandle then
  return false
end
local data = httpHandle.readAll()
httpHandle.close()
print(#data .. " bytes")
local fhandle = fs.open(saveFile, "wb")
for i = 1, #data do
  fhandle.write(string.byte(data, i))
end
fhandle.close()
return true
end

downloadBinaryFile(<image url>,"skin.png")

Edited by emanuel12324, 19 July 2018 - 09:32 PM.


#2 KingofGamesYami

  • Members
  • 3,002 posts
  • LocationUnited States of America

Posted 19 July 2018 - 09:56 PM

HTTP binary support was a long standing bug that was patched in 1.80pr1. If you are using an older version it will not work.

#3 emanuel12324

  • Members
  • 28 posts

Posted 20 July 2018 - 12:37 AM

Well putting aside the http portion of my question, what do you think about the byte reading part?

#4 KingofGamesYami

  • Members
  • 3,002 posts
  • LocationUnited States of America

Posted 20 July 2018 - 12:52 AM

I have very little experience with reading binary in CC, but my first thought is that num is larger than the number of bytes available for some reason.

#5 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,099 posts
  • LocationTasmania (AU)

Posted 20 July 2018 - 02:58 AM

I can tell you that fileHandle.read() returns nil once you reach EOF (and never before), but otherwise, there's not enough information here to debug your first block of code. Provide the rest of it, along with the file you're using for your "stream".

Regarding the http API, even with later builds of ComputerCraft, http.get uses text mode for web handles by default, mangling anything that isn't strictly ASCII. For binary mode, you need to pass in true as a third argument (eg http.get("someAddress.com", nil, true). Under older builds, there's nothing for it but to use binary-to-text encoding methods such as base64 - BBPack can help you with this.

Another perk of the more recent versions is that binary mode file handles also accept strings, allowing you to dump out all of your data to disk in a single write operation. If you really want to deal with numeric byte representations, though, it can be a bit faster to cut down on your function calls by doing the conversion all in one go:

data = {data:byte(1, #data)}
for i = 1, #data do
  fhandle.write(data[i])
end

Edited by Bomb Bloke, 20 July 2018 - 03:11 AM.


#6 emanuel12324

  • Members
  • 28 posts

Posted 20 July 2018 - 03:54 AM

https://pastebin.com/mQ6CVG8V is the entirety of the code. It includes a bunch of stuff for implementing require because the libraries I'm using need it. Also uses deflateLua. I commented out line 359 for testing, but when all is working correctly it should pass that if condition. The image I'm using is here.

Edited by emanuel12324, 20 July 2018 - 03:56 AM.


#7 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,099 posts
  • LocationTasmania (AU)

Posted 20 July 2018 - 04:53 AM

Line 342: you're using io.open(). File handles created in this manner require the use of colon notation for their function calls, like so (meaning you'd want stream:read() on line 151). If you wish to use dot notation instead, use fs.open().

#8 emanuel12324

  • Members
  • 28 posts

Posted 20 July 2018 - 05:03 AM

Fixing that (and also uncommenting line 359) yields an error "Not a png". This leads me to believe that it's still not reading bytes correctly. I manually read through the each byte at the very beginning of the png, and it matches correctly with the if condition on line 358.

#9 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,099 posts
  • LocationTasmania (AU)

Posted 20 July 2018 - 08:11 AM

The readChar() function used in that comparison statement doesn't return anything. I'm guessing you meant to do:

local function readChar(stream, num)
    num = num or 1
    local bytes = {}
    for i = 1, num do
        bytes[i] = string.char(readByte(stream))
    end
    return table.concat(bytes)
end


#10 emanuel12324

  • Members
  • 28 posts

Posted 20 July 2018 - 08:35 PM

I still seem to be getting the "Not a png" error. This is what has been troubling me for the past few days :wacko:

#11 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,099 posts
  • LocationTasmania (AU)

Posted 21 July 2018 - 08:16 AM

If you're not reading in the header you expect, then output what you are reading. If you can't figure things out from there, then provide that output here.

if readHeader ~= expectedHeader then
  print(readHeader:byte(1, #readHeader))
  error("Not a PNG")
end


#12 emanuel12324

  • Members
  • 28 posts

Posted 23 July 2018 - 11:36 PM

I'm now getting an error that reads "window:94 Arguments must be the same length". I haven't made any changes to the code myself other than what you have said here.

Edited by emanuel12324, 23 July 2018 - 11:37 PM.


#13 KingofGamesYami

  • Members
  • 3,002 posts
  • LocationUnited States of America

Posted 23 July 2018 - 11:57 PM

This is why you're getting that error.

#14 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,099 posts
  • LocationTasmania (AU)

Posted 24 July 2018 - 01:55 AM

And this should help you locate the exact line within your own script that's causing the failure within the window API.

#15 emanuel12324

  • Members
  • 28 posts

Posted 24 July 2018 - 06:54 PM

I've actually used trace before. Using it here actually prints nothing, however strange that may be. And I checked out BB's code for note posted in the other thread and when attempting to use it, it says attempt to call nil in reference to a function called unpack().

#16 KingofGamesYami

  • Members
  • 3,002 posts
  • LocationUnited States of America

Posted 24 July 2018 - 09:34 PM

unpack has been renamed to table.unpack in later versions.

#17 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,099 posts
  • LocationTasmania (AU)

Posted 24 July 2018 - 11:46 PM

That's not exactly true - unpack() is only disabled if disable_lua51_features is set to true within ComputerCraft.cfg. By default it isn't, and since it's only good for performing partial forwards compatibility testing, there typically isn't any good reason to have it set.

#18 emanuel12324

  • Members
  • 28 posts

Posted 25 July 2018 - 11:46 PM

Well with all this, it seems to work fine (to a point) with any PNG except the one I need. It seems that what I have may not be in the correct format for a PNG. I guess I'll have to sort that out later. The "to a point" I mentioned before refers to an error on line 382. It says attempt to add nil and number.

#19 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,099 posts
  • LocationTasmania (AU)

Posted 26 July 2018 - 01:48 AM

That line falls within StringStream.read(), which 1) needs to be called with colon notation and 2) needs a "num" parameter specifying the amount of bytes to read. It's called from line 151 (in readByte()) with neither of these, try changing that to:

return stream:read(1)

You could alternatively ditch the 1 parameter by adding a new line to the top of StringStream.read(), picking a default value for num:

num = num or 1

Note that either way this will force you to use io.open() on line 342, and that I only have access to an old version of your code.

#20 emanuel12324

  • Members
  • 28 posts

Posted 26 July 2018 - 03:59 AM

Right, well here is a new version of my code so you can make an updated diagnosis. I do see what you are saying about the num variable and I tried to feed it to readByte but I was unsuccessful.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users