any way to prevent it?
my program is converting a string into a number and if its too long it will start corrupting
high numbers turning into scientific notation
Started by PixelToast, Aug 03 2012 10:40 PM
7 replies to this topic
#1
Posted 03 August 2012 - 10:40 PM
#2
Posted 04 August 2012 - 01:57 PM
Is the number solid and is rounding it acceptable?
If so you can probably convert is to xX10^y number eg 1.58295X10^8 would be 158295000. To get around this in CC(since lua doesn't have a dictionary/hashtable to my knowledge), you could add a decimal point in the string then log the decimal places to a variable and multiply the converted string by 10 to the power of the variable later.
would that be of any use?
If so you can probably convert is to xX10^y number eg 1.58295X10^8 would be 158295000. To get around this in CC(since lua doesn't have a dictionary/hashtable to my knowledge), you could add a decimal point in the string then log the decimal places to a variable and multiply the converted string by 10 to the power of the variable later.
would that be of any use?
#3
Posted 04 August 2012 - 04:53 PM
If the problem is displaying it or converting it to a string, you can use string.format instead of tostring:
local s = string.format("%f", theNumber)
print(s)
Not tested, but I think it will work.
#4
Posted 13 July 2013 - 10:31 PM
I am using that same code and still having issues.
and it returns 1.2345678123456879E7
Any idea on how I can print a large number with out it going to scientific notation?
amount = 12345678.123456879
local s = string.format("%f", amount)
print(s)
and it returns 1.2345678123456879E7
Any idea on how I can print a large number with out it going to scientific notation?
#5
Posted 13 July 2013 - 10:41 PM
From the first Google search, I got this link: http://stackoverflow...ntific-notation
Here is the code they use:
A comment on the answer also seems useful:
Not tested, but it got good responses on stack overflow.
Here is the code they use:
print (string.format ("%18.0f", number))
A comment on the answer also seems useful:
Quote
You can omit the width; "%.0f" will use exactly as many digits as needed.
Not tested, but it got good responses on stack overflow.
#6
Posted 13 July 2013 - 10:48 PM
amount = bc.number(12345678.123456789)
this may work if CC supports it
#7
Posted 14 July 2013 - 10:05 AM
with that number I still get Scientific notation... I am still going to write a program and just hope I never get a math answer that big...
#8
Posted 14 July 2013 - 08:22 PM
my god, ancient post
my problem was solved,i used the bigNum API i remember now, i rewrote the crappy encryption algorithm not to output the entire string as a number, but if you are having problems use bignum
my problem was solved,
1 user(s) are reading this topic
0 members, 1 guests, 0 anonymous users











