Jump to content




high numbers turning into scientific notation


  • You cannot reply to this topic
7 replies to this topic

#1 PixelToast

  • Signature Abuser
  • 2,265 posts
  • Location3232235883

Posted 03 August 2012 - 10:40 PM

any way to prevent it?
my program is converting a string into a number and if its too long it will start corrupting

#2 Pharap

  • Members
  • 816 posts
  • LocationEngland

Posted 04 August 2012 - 01:57 PM

Is the number solid and is rounding it acceptable?

If so you can probably convert is to xX10^y number eg 1.58295X10^8 would be 158295000. To get around this in CC(since lua doesn't have a dictionary/hashtable to my knowledge), you could add a decimal point in the string then log the decimal places to a variable and multiply the converted string by 10 to the power of the variable later.

would that be of any use?

#3 MysticT

    Lua Wizard

  • Members
  • 1,597 posts

Posted 04 August 2012 - 04:53 PM

If the problem is displaying it or converting it to a string, you can use string.format instead of tostring:
local s = string.format("%f", theNumber)
print(s)
Not tested, but I think it will work.

#4 Zelman89

  • Members
  • 20 posts

Posted 13 July 2013 - 10:31 PM

I am using that same code and still having issues.

amount = 12345678.123456879
local s = string.format("%f", amount)
print(s)

and it returns 1.2345678123456879E7

Any idea on how I can print a large number with out it going to scientific notation?

#5 Grim Reaper

  • Members
  • 503 posts
  • LocationSeattle, WA

Posted 13 July 2013 - 10:41 PM

From the first Google search, I got this link: http://stackoverflow...ntific-notation

Here is the code they use:
print (string.format ("%18.0f", number))

A comment on the answer also seems useful:

Quote

You can omit the width; "%.0f" will use exactly as many digits as needed.

Not tested, but it got good responses on stack overflow.

#6 MR_nesquick

  • Members
  • 106 posts
  • LocationNorway

Posted 13 July 2013 - 10:48 PM

amount = bc.number(12345678.123456789)


this may work if CC supports it

#7 Zelman89

  • Members
  • 20 posts

Posted 14 July 2013 - 10:05 AM

with that number I still get Scientific notation... I am still going to write a program and just hope I never get a math answer that big... :(

#8 PixelToast

  • Signature Abuser
  • 2,265 posts
  • Location3232235883

Posted 14 July 2013 - 08:22 PM

my god, ancient post
my problem was solved, i used the bigNum API i remember now, i rewrote the crappy encryption algorithm not to output the entire string as a number, but if you are having problems use bignum





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users