11 replies to this topic

[Symmetryc]

I'm trying to write an API, but I've encountered a problem that I can't seem to figure out how to get around. tostring() is (or atleast appears to be) rounding its output. Here is an example:

print(tostring(123344.123455634))
--> 123344.125

If anyone happens to have a solution I'd be very grateful .

Edit: It seems that I need to bypass tonumber() rounding as well now, is there anyway to do that?

[theoriginalbit]

You can use string.format to bypass this.

print(string.format("%f", 123344.123455634))

%f means it is expecting a floating point number.

[Symmetryc]

theoriginalbit, on 01 June 2013 - 09:56 PM, said:

You can use string.format to bypass this.

print(string.format("%f", 123344.123455634))

%f means it is expecting a floating point number.

OK, thanks, is there anywhere I can get more documentation on string.format()? I attempted to use it with %d (from regex lol) and it actually didn't error, but it didn't round. Then I looked it up on the lua-users wiki and got this, but it just said to use it as printf() from C. I don't know any C-Based Languages, so I looked up printf(), but I didn't really find anything I understood.

[theoriginalbit]

Symmetryc, on 01 June 2013 - 10:05 PM, said:

OK, thanks, is there anywhere I can get more documentation on string.format()? I attempted to use it with %d (from regex lol) and it actually didn't error, but it didn't round. Then I looked it up on the lua-users wiki and got this, but it just said to use it as printf() from C. I don't know any C-Based Languages, so I looked up printf(), but I didn't really find anything I understood.

The link you just gave has more info on it. It states that it works just like printf.

Quote

• c, d, E, e, f, g, G, i, o, u, X, and x all expect a number as argument.
  
• q and s expect a string.

> = string.format("%s %q", "Hello", "Lua user!") -- string and quoted string
Hello "Lua user!"
> = string.format("%c%c%c", 76,117,97) -- char
Lua
> = string.format("%e, %E", math.pi,math.pi) -- exponent
3.141593e+000, 3.141593E+000
> = string.format("%f, %g", math.pi,math.pi) -- float and compact float
3.141593, 3.14159
> = string.format("%d, %i, %u", -100,-100,-100) -- signed, signed, unsigned integer
-100, -100, 4294967196
> = string.format("%o, %x, %X", -100,-100,-100) -- octal, hex, hex
37777777634, ffffff9c, FFFFFF9C

[Symmetryc]

theoriginalbit, on 01 June 2013 - 10:13 PM, said:

Symmetryc, on 01 June 2013 - 10:05 PM, said:

OK, thanks, is there anywhere I can get more documentation on string.format()? I attempted to use it with %d (from regex lol) and it actually didn't error, but it didn't round. Then I looked it up on the lua-users wiki and got this, but it just said to use it as printf() from C. I don't know any C-Based Languages, so I looked up printf(), but I didn't really find anything I understood.

The link you just gave has more info on it. It states that it works just like printf.

Quote

• c, d, E, e, f, g, G, i, o, u, X, and x all expect a number as argument.
  
• q and s expect a string.

> = string.format("%s %q", "Hello", "Lua user!") -- string and quoted string
Hello "Lua user!"
> = string.format("%c%c%c", 76,117,97) -- char
Lua
> = string.format("%e, %E", math.pi,math.pi) -- exponent
3.141593e+000, 3.141593E+000
> = string.format("%f, %g", math.pi,math.pi) -- float and compact float
3.141593, 3.14159
> = string.format("%d, %i, %u", -100,-100,-100) -- signed, signed, unsigned integer
-100, -100, 4294967196
> = string.format("%o, %x, %X", -100,-100,-100) -- octal, hex, hex
37777777634, ffffff9c, FFFFFF9C

I know, but what I was saying was that I don't know how printf works XD. Unless what they show in the examples is all that printf does.

[theoriginalbit]

Symmetryc, on 01 June 2013 - 10:25 PM, said:

I know, but what I was saying was that I don't know how printf works XD. Unless what they show in the examples is all that printf does.

That is pretty much all you need to worry about. printf does do more, but those features aren't available in Lua.

Just remember that you can do this, its not just about having the options in the string (i.e. "%c%i%s")

print(string.format("Hello %s this is a number %i, and a quoted string %q", "World!", 124, "haha"))

[Symmetryc]

theoriginalbit, on 01 June 2013 - 10:38 PM, said:

That is pretty much all you need to worry about. printf does do more, but those features aren't available in Lua.

Ah, OK, when I looked it up there were a lot of stuff I didn't really get, but didn't see it in the Lua examples. Well, thanks again!

[Symmetryc]

OK, now I need a way to bypass tonumber()'s rounding . Is there a way to do that? I've tried doing "32400834201.32149081"+0 and loadstring("return 2390108443.314081408")(), but they both didn't work.

[GopherAtl]

there are limits to floating-point precision, which is what you're running into. As a string, it's an array of characters, so can be arbitrarily long, but as a number, it's stored as a fixed-size set of bits. Almost all computer numbers have this issue. What on earth do you need that level of precision for, anyway? There are workarounds, but selecting an appropriate one requires knowing what you're doing with all this extreme precision.

[Symmetryc]

GopherAtl, on 02 June 2013 - 11:09 AM, said:

there are limits to floating-point precision, which is what you're running into. As a string, it's an array of characters, so can be arbitrarily long, but as a number, it's stored as a fixed-size set of bits. Almost all computer numbers have this issue. What on earth do you need that level of precision for, anyway? There are workarounds, but selecting an appropriate one requires knowing what you're doing with all this extreme precision.

I don't need the level of precision that I showed in my example (above your post), I only need about 8-10 characters for what I'm actually doing. I was just bored and trying to improve my base converter to support over base 64, but I was encountering problems when converting numbers where most of those characters were behind the radix.

[GopherAtl]

ah. Well, you've hit and passed the limits of what your cc computer's number format can handle as far as precision goes. If you really want to continue this experiment, you'll have to build your own number format, which, while possible, is not the sort of thing lua is actually very good at. Keeping them as strings might be your best bet, if you don't need to be able to do a lot of math on them.

[Symmetryc]

GopherAtl, on 02 June 2013 - 11:29 AM, said:

ah. Well, you've hit and passed the limits of what your cc computer's number format can handle as far as precision goes. If you really want to continue this experiment, you'll have to build your own number format, which, while possible, is not the sort of thing lua is actually very good at. Keeping them as strings might be your best bet, if you don't need to be able to do a lot of math on them.

Yeah I actually attempted to do that for just binary, by editing the math metamethods, but I found that doing traditional math within these metamethods was inevitable, ah well I guess it was a good try. Thanks for your help .