Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / Problem with rounding a float to two decimal places

Author
Message
Orgull V2
20
Years of Service
User Offline
Joined: 22nd Jul 2004
Location: ...and the Trogdor comes in the niiight!
Posted: 6th Sep 2004 14:19
Hi there.

I'm trying to round my floats to two decimal places using the very simple "int(num#*100+0.5)/100" but it's not working at all.

This code, with a character.wisdom value of 8, returns 0.319999992847



This code returns a value of 0



Can anyone help me figure out what's going wrong here?

Thanks
Dave J
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Feb 2003
Location: Secret Military Pub, Down Under
Posted: 6th Sep 2004 17:59
There's a problem with floats built into the very computer system we use, unfortunately there's nothing we can do about it. I'll let you know that it's not just DBP, this problem exists with floats in everything.


"Computers are useless they can only give you answers."
BatVink
Moderator
22
Years of Service
User Offline
Joined: 4th Apr 2003
Location: Gods own County, UK
Posted: 6th Sep 2004 19:00
Floats store numbers as a series of fractions:

1/2 + 1/4 + 1/8 + 1/16 + 1/32....

so 8/25 =

(0 * 1/2) + (1 * 1/4) + (0 * 1/8) + (1 * 1/16) + (0 * 1/32) + (0 * 1/64) + (0 * 1/128) + (1 * 1/256)

So, to 8 floating numbers, you're still not accurate enough to represent 8/25 yet, even though 8/25 is actually 0.32 as we see it.

Computers, huh...who needs 'em!

BatVink
http://biglaugh.co.uk/catalog AMD 3000+ Barton, 512Mb Ram, 120 Gig Drive space, GeForce 5200 FX 128 Mb, Asus A7N8X Mobo.
Terms & Conditions apply
Van B
Moderator
22
Years of Service
User Offline
Joined: 8th Oct 2002
Location: Sunnyvale
Posted: 6th Sep 2004 20:05
You could multiply by 100, then take the hundred away again, like:

pi#=3.1415926535897932384626433832795

decpi=int(pi#*100.0)/100.0

Should leave decpi=3.14


Van-B


Muhahahahaha.
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 6th Sep 2004 20:29
Try this function:



It converts your number to a string, then massages it to produce the output you want - you could modify it to return the string instead of printing it, so that you can use it elsewhere.

*** Coming soon - Network Plug-in - Check my site for info ***
For free Plug-ins, source and the Interface library for Visual C++ 6, .NET and now for Dev-C++ http://www.matrix1.demon.co.uk
Orgull V2
20
Years of Service
User Offline
Joined: 22nd Jul 2004
Location: ...and the Trogdor comes in the niiight!
Posted: 7th Sep 2004 00:36
Uh, VanB.. if you read the second line of my post you'll see that's exactly what I'm trying to do. It doesn't work.

I've seen this work in Pascal and one of the basics, VB I think. I'm still not clear why it wouldn't work in DBPro.

Thanks for the tips though... Ian M I don't quite follow your function there, but I just got up here in Canada so... I'm still a little fuzzy

I think I'll just write my own rounding function from scratch, that way I'll understand it completely, like the input function I wrote from scratch the other day. I don't know what it's a sign of but I'm spending more time writing my own functions than I am engine code. I never used to do that, but then I never used to actually have code that worked before
CattleRustler
Retired Moderator
21
Years of Service
User Offline
Joined: 8th Aug 2003
Location: case modding at overclock.net
Posted: 7th Sep 2004 00:43
Orgull is this just for display purposes? If so check our plugin command TC_FORMATNUM()


DBP_NETLIB_v1.4.3 DarkTOPIA - September 2004
Orgull V2
20
Years of Service
User Offline
Joined: 22nd Jul 2004
Location: ...and the Trogdor comes in the niiight!
Posted: 7th Sep 2004 01:06
Argh... sadly no Cattlerustler, in this particular instance I want the character's BASE critical hit chance to be wisdom/25 to 2 decimal places becuase the BASE amount will be added to by skills and equipment which will also be numbers with 2 decimal places and I'm counting on expected values for the upper and lower limits to work.

Therefore I need 8/25 to be 0.32 NOT 0 and NOT 0.319999992847

If I just use the numbers and format them for display witout rounding, by the time the BASE number has been through say 75 levels, 4 skills and 3 pieces of equipment, the player is going to look at his stuff and say "What! Why doesn't my critical hit chance add up properly?"

Fun eh?
CattleRustler
Retired Moderator
21
Years of Service
User Offline
Joined: 8th Aug 2003
Location: case modding at overclock.net
Posted: 7th Sep 2004 01:39
got ya. sorry


DBP_NETLIB_v1.4.3 DarkTOPIA - September 2004
Orgull V2
20
Years of Service
User Offline
Joined: 22nd Jul 2004
Location: ...and the Trogdor comes in the niiight!
Posted: 7th Sep 2004 01:50
At this point CattleRustler, I feel like stabbing myself in the foot with my screwdriver and then drinking vodka for the rest of the long weekend. I've been working on this damn problem since ysterday and I'm ready to vomit. I just noticed that VAL will only return integers, not floats... yay!

You know what's funny? number = int(number*100+0.5)/100 works on a freaking COMMODORE GODFORSAKEN 64! How hard can it be to make this work on a PC that has more processing power than the original Columbia Space Shuttle flight control computers?

(Orgull reaches for his screwdriver...)
JeBuS
20
Years of Service
User Offline
Joined: 20th Jul 2004
Location: Undisclosed Location, Dominion of JeBuS
Posted: 7th Sep 2004 01:53
I'd say the float to string to float conversion would be the easiest


High quality models and graphics, low prices. Graphics for the rest of us.
CattleRustler
Retired Moderator
21
Years of Service
User Offline
Joined: 8th Aug 2003
Location: case modding at overclock.net
Posted: 7th Sep 2004 01:57
that might work because when you reset a float it will hold the correct precision until you math on it.


DBP_NETLIB_v1.4.3 DarkTOPIA - September 2004
Orgull V2
20
Years of Service
User Offline
Joined: 22nd Jul 2004
Location: ...and the Trogdor comes in the niiight!
Posted: 7th Sep 2004 01:57 Edited at: 7th Sep 2004 02:00
Uh, how exactly would you do that Jebus? Since VAL doesn't convert strings to floats, only integers. I can go float to string no problem, but string to float has proved impossible so far.

Uh CattleRustler I don't understand your statement. "Reset" a float?
CattleRustler
Retired Moderator
21
Years of Service
User Offline
Joined: 8th Aug 2003
Location: case modding at overclock.net
Posted: 7th Sep 2004 01:59
mystr=tc_formatnum(arg,arg,arg)

myfloat=val(mystr)

?


DBP_NETLIB_v1.4.3 DarkTOPIA - September 2004
Orgull V2
20
Years of Service
User Offline
Joined: 22nd Jul 2004
Location: ...and the Trogdor comes in the niiight!
Posted: 7th Sep 2004 02:02
Sorry Cattle, that doesn't work. VAL does NOT convert strings to floats. Period. No matter how much we may want it to. It works in DB classic but not in DBPro. I've tested it, and others have complained about it so I know it's not just me (many, many posts I found)
Van B
Moderator
22
Years of Service
User Offline
Joined: 8th Oct 2002
Location: Sunnyvale
Posted: 7th Sep 2004 02:12
Quote: "VAL does NOT convert strings to floats"


It must do, use it all the time - my script and file system would fall on it's face if it did'nt.

Try this code and see what it spits out:



It returns:
3.14
3.1400001049
3

String, VAL string, and integer respectively.


Van-B


Muhahahahaha.
JeBuS
20
Years of Service
User Offline
Joined: 20th Jul 2004
Location: Undisclosed Location, Dominion of JeBuS
Posted: 7th Sep 2004 02:26 Edited at: 7th Sep 2004 02:27
Van, I think that just proves his point, you can't get it to round to 2 decimal places. Cuz any time you operate on it as a string then throw it back together as a float, it adds extra digits.


High quality models and graphics, low prices. Graphics for the rest of us.
CattleRustler
Retired Moderator
21
Years of Service
User Offline
Joined: 8th Aug 2003
Location: case modding at overclock.net
Posted: 7th Sep 2004 02:28
/me does a test when done cleaning my house...


DBP_NETLIB_v1.4.3 DarkTOPIA - September 2004
Orgull V2
20
Years of Service
User Offline
Joined: 22nd Jul 2004
Location: ...and the Trogdor comes in the niiight!
Posted: 7th Sep 2004 03:11
Well Van I'll be fish-slapped. For some reason your code works, specifically the line "var#=val(var$)" yet when I try it inside my main program with my variables, the same line returns only integers. Maybe the problem has something to do with user-defined types. Or maybe I've messed up an AS somewhere. At this point "My Head A Splode!" so I'm just going to go work on something else now for a while.

I think I'll just be content with displaying a string to two digits, which is easy to do, and forget trying to do precision math because apparently it's impossible. If 3.14 actually equals 3.1400001049 because that's how floats work, then I'm just going to have to work around the inaccuracies.

The thing that bugs me is I know the computers at STELCO return float values to specific decimal places with pinpoint accuracy, if they didn't, the furnace fuel flow regulators would be adding 3.1400001049 cf/m of oxygen instead of 3.14 cf/m, the second amount being just right, but the first amount pushing the fuel into explosive ratio territory. And the computers are Pentiums, so I know those cagy engineers have got some sort of clever workaround going.

What the hell do people do who require precision math for their programs? There's got to be a way, I've seen it work before. Argh.
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 7th Sep 2004 04:48
They don't - if they are using floating-point then they are using exactly the same floating point system that you are. All that they do, is cut off the decimal places when they display the number.

They will be using an internal fixed-point system instead, or using implied decimal places within integers ... who knows?

*** Coming soon - Network Plug-in - Check my site for info ***
For free Plug-ins, source and the Interface library for Visual C++ 6, .NET and now for Dev-C++ http://www.matrix1.demon.co.uk
Orgull V2
20
Years of Service
User Offline
Joined: 22nd Jul 2004
Location: ...and the Trogdor comes in the niiight!
Posted: 7th Sep 2004 05:19
Hmm... implied decimal places within integers... Ian M you are the KING! What an elegant and staggeringly simple solution.

(Orgull Bows)

That's how I'll do any math that has to be precise to a set decimal point. It's so beautiful I could weep! Ian u R teh roxxorz n eVeRyOnE elz is teh loozoorz! (sorry I'm not very good at leet speak)

Problem Solved thanks to Ian M, THE MAN.
Mentor
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: United Kingdom
Posted: 7th Sep 2004 05:31
exactly, the older 8 bit machines hid the imprecision by lopping the last two places off each number, since they used floats for both math and compares this was permissable, you can get a better precision inside Pro by using double precision float, due to some weirdness in the way numbers are calculated, if you break the calculation down to individual steps and use double precision then you will get the correct answer displayed, but try to make the calculation into one lump and it goes pear shaped
this will display the correct value.

number# as double float
number#=3.141592654
number# = number#*100.0
number#=number#+0.5
number#=int(number#)
number#=number#/100.0
print number#
wait key

make what use of it you will, I myself have complained that it`s weird that we had easier to work with precision on a Spectrum than we have on a system that needs all of 83 Spectrum 48k`s memorys to display "hello world", but the math precision still plays us up with problems like simple rounding, maybe Lee could include a precision suppresion switch Float rounding on/ Float rounding off for example, to give us the old style clipped and rounded values that made life so much easier and intuitive

Mentor.

PC1: P4 3ghz, 1gig mem, 2x160gig hd`s, Radeon 9800pro w cooler (3rd gfx card), 6 way speakers.
PC2: AMD 2ghz, 512mb ram, FX5200 ultra, 16 bit SB.
Mini ATX cases suck.
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 7th Sep 2004 21:44
Quote: "Problem Solved thanks to Ian M, THE MAN"


Uh, Okay ...

It's actually something that I use all of the time at work - you can't have your pennies being dropped accidentally.

Just be careful of your maths and maintaining your decimal places.

*** Coming soon - Network Plug-in - Check my site for info ***
For free Plug-ins, source and the Interface library for Visual C++ 6, .NET and now for Dev-C++ http://www.matrix1.demon.co.uk

Login to post a reply

Server time is: 2025-06-05 20:17:16
Your offset time is: 2025-06-05 20:17:16