Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Newcomers DBPro Corner / Floating Point Precision

Author
Message
Elustrian
16
Years of Service
User Offline
Joined: 17th May 2008
Location:
Posted: 17th May 2008 22:24
Link
Pretty basic question but cannot locate answer in forums.

How do I round the value of a floating point variable?

Currently have

N# = 3
D# = 11
Ratio# = (N# / D#) * 100

Print Ratio#

Outputs:
27.2000007629

Would like to be able to round to just one digit of precision (i.e., 27.2) either by definition of variable or at time of print.

Thanks,
Back to top
Profile PM
jinzai
18
Years of Service
User Offline
Joined: 19th Aug 2006
Location: USA
Posted: 17th May 2008 23:06 Edited at: 17th May 2008 23:08
Link
You can replace the print statement with:
print str$(Ratio#, 1)

Note that this just limits the decimal places in the print statement and does not affect the variable itself.
Back to top
Profile PM Email Website
Elustrian
16
Years of Service
User Offline
Joined: 17th May 2008
Location:
Posted: 18th May 2008 02:20
Link
Print str$(Ratio#,1)

gives an error:

#100008: Parameter for "str$' do not match 'Value' at line 90.

Just to check

Print Ratio#

still outputs

27.2000007629

Thanks
Back to top
Profile PM
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 18th May 2008 02:47
Link
You need to install the update for DBPro - the extra parameter for the number of decimal places was an addition at around update 6.0

[b]Utility plug-ins collection (updated 01/05/08) and
http://www.matrix1.demon.co.uk
Back to top
Profile PM Email Website
Back to top

Login to post a reply

Server time is: 2024-11-25 23:35:42
Your offset time is: 2024-11-25 23:35:42