10 replies to this topic

[jag]

So I am trying to make a simple program.
It's just with some selection and stuff.

But while I was testing I noticed a weird "bug":
(">" means simply output from the computer while in the lua prompt)

sel = 3.2
>

sel - math.floor(sel)
> 0.2

sel - math.floor(sel) == .2
> false

sel - math.floor(sel) == 0.2
> false

sel % 1
> 0.2

sel % 1 == .2
> false

sel - math.floor(sel) == sel - math.floor(sel)
> true

sel % 1 == sel % 1
> true

sel - math.floor(sel) == sel % 1
> true

sel - math.floor(sel) == .2
> false

So in some way either the "==" operator is disfunctional or the math of my computer is collapsing...

EDIT: There might be some bug with the server that I am playing on...
EDIT 2: Nope we restarted the server but it stills does this weird thingy... This might still be a bug with the server, but I'm not sure.

[theoriginalbit]

jag_e_nummer_ett, on 20 January 2013 - 11:09 PM, said:

- snip -

math.floor ALWAYS rounds the number down making it 3.0 ... just like math.ceil always rounds up making it 4.0 ... i think the one you are after is math.modf

http://lua-users.org...LibraryTutorial

Edited by TheOriginalBIT, 20 January 2013 - 11:18 PM.

[jag]

TheOriginalBIT, on 20 January 2013 - 11:16 PM, said:

-snip-

No because I do "sel [MINUS] floor'ed sel":

sel - math.floor(sel)
> 0.2

So its:
3.2 - 3.0 = 0.2
simple

[Orwell]

Indeed, he is certainly using the correct calculations. Though, it might be worth looking into math.modf. In c++ at least, the % operator is only usable for integers (probably not true for Lua), and fmod() is what you'd want for floating point numbers. But even if it works with math.modf, it doesn't explain why your statements don't.

[theoriginalbit]

Orwell, on 21 January 2013 - 12:35 AM, said:

Indeed, he is certainly using the correct calculations. Though, it might be worth looking into math.modf. In c++ at least, the % operator is only usable for integers (probably not true for Lua), and fmod() is what you'd want for floating point numbers. But even if it works with math.modf, it doesn't explain why your statements don't.

fmod and modf are different in Lua...

Quote

math.modf

Return the integral and fractional parts of the given number.
> = math.modf(5)
5 0
> = math.modf(5.3)
5 0.3
> = math.modf(-5.3)
-5 -0.3

[Eric]

Welcome to floating point numbers! In the same way that you can't represent 1/3 accurately in decimal form, computers cannot represent 1/5 accurately in binary form. This very simple test fails:

assert(1.2 - 1 == 0.2)

Because:

lua> print((1.2 - 1) - 0.2)
-5.5511151231258e-17

Take a look at What Every Computer Scientist Should Know About Floating-Point Arithmetic

[jag]

Oh god I am confused...

How do I get this to work?

[jag]

Eric, on 21 January 2013 - 12:56 AM, said:

Take a look at What Every Computer Scientist Should Know About Floating-Point Arithmetic

The only way you can understand everything on that site is if you wrote it got damn by yourself..

[Eric]

That wasn't the article I remembered it to be. This one any better?

[Orwell]

Basically you can sum it up as 'rounding errors'. It seems that I was mistaken about the functionality of 'math.modf', from what TheOriginalBit posted about it, it seems like it should do the job just fine.

Edit: tested it, still got the same problems. Try out Eric's article. It should do the job. Alternatively, if you want a less elegant (and less robust) solution, use tostring() to compare them

[jag]

Yay thank you!
That simple but inefficient way of converting and comparing it as a string works!
My program wooooorks! ;D