5 replies to this topic

[Doyle3694]

Hello everyone! As the math geek I am, I was playing around with the famous 1/998001 fraction. For those of you who don't know, it will return a endless list of all 3 digit numbers, and when it reach 999 it will start over at 000 again etc. So I wanted to make a program and see if I could make that program to atleast print out 1 series of this. though I noticed lua returns 1.002003E-6

So my question is, how does lua answer to infinite numbers and why is this?

Here's my program btw:

test = 1/998001
tostring(test)
string.sub(test, 1, 51)
print(test)

[Lyqyd]

There is a limit to the precision with which computers can store numerical values.

[kazagistar]

Straight from the manual.

You can google for more information about floating point numbers all over the internet as well. Digging into the details is good!

[Doyle3694]

yeah I was more thinking if it could be cutting off the decimals it cant use rather than giving that wierd string or wahtever you wanna call it. but if the problem is in lua and not the cc port then I understand you can do nothing /> Thanks for the replies!

[Jarle212]

I got 1,002003004005006007008009010011e-6

[Doyle3694]

I was doing it on CC 1.41, that might be why