Jump to content




having issues with division and floating point numbers

computer

3 replies to this topic

#1 ackley14

  • Members
  • 8 posts

Posted 18 February 2014 - 07:43 PM

Hey, I'm new to the forums here but have recently taken an interest in computercraft. i started this program yesterday built around the direwolf20 button api (you can get here http://pastebin.com/HRbMF1Eg) i am basically working on a touch screen calculator to familiarize myself with lua (which I really think I have by the way :D) however the problem occurs when the division operation is chosen (i.e. player presses 1,2,/,8 in that order, and then it would come back 1 where it should come back something like 1.5. the weirdest part is that (i have two terminals, one for reference and the other for the program) on the reference terminal, lua has no problem handeling weird arangements of numbers while dividing within the lua test bed so i am so confused as to why its happening in my program.

you can download my code from http://pastebin.com/iUedYV3S
before you do, there are a few things you must know.

1. this is simply a proof of concept for me, it dosn't look great, it dosn't work perfectly (its also not finished lol)
2.i have spelling issues, i use double letters where they shouldn't be like the word opperation for example. please disregard typos (unless relevant of course)
and lastly, this program will only run properly on a 3x4 (3 wide, four tall) grid of advanced monitors using an advanced computer, its default, set to have the monitor placed above the computer

any help is appreciated, thanks~

Edited by ackley14, 18 February 2014 - 08:11 PM.


#2 Himself12794

  • Members
  • 56 posts

Posted 18 February 2014 - 08:51 PM

On line 166, you have nst1/nst1 instead of nst1/nst2. That is why it always equals 1.

elseif opptype == "divide" then
  nst2 = table.concat(numset2)
  nst1 = table.concat(numset1)
  opt = nst1/nst1
  printOut(opt)

Should be:

elseif opptype == "divide" then
  nst2 = table.concat(numset2)
  nst1 = table.concat(numset1)
  opt = nst1/nst2
  printOut(opt)

I changed that and it seems to work.

Edited by Himself12794, 18 February 2014 - 08:57 PM.


#3 ackley14

  • Members
  • 8 posts

Posted 18 February 2014 - 09:02 PM

.......Thank you! wow i feel stupid! xD i have caught EVERY other typo of mine like that until now! awesome!

#4 Himself12794

  • Members
  • 56 posts

Posted 19 February 2014 - 11:53 AM

View Postackley14, on 18 February 2014 - 09:02 PM, said:

.......Thank you! wow i feel stupid! xD i have caught EVERY other typo of mine like that until now! awesome!
I do the same thing ALL the time, believe me.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users