onlabs_logo.gif

Instrument Interfacing
Quality Management
Electronic Lab Notebook

Wednesday, 20 August 2008
Home arrow Support arrow Number rounding
Main Menu
Home
About Us
Products
Markets
Services
Support
Resources
Partners & vendors
Site Map
Number Rounding

Online Worksheet Notes
Number Rounding
Issue
In science, all numbers are based upon measurements (except for a very few that are defined).
Since all measurements are uncertain, we must only use those numbers that are meaningful. A
common ruler cannot measure something to be 22.4072643 cm long. Not all of the digits have
meaning (significance) and, therefore, should not be written down. In science, only the numbers
that have significance (derived from measurement) are written.
Online Worksheet offers a very advanced number formatting tool, allowing automated number
reporting. The system is able to round in three different modes:
1. Rounding by decimals
2. Rounding by significant figures
3. Rounding by number ranges, by decimals or significant figures.
Rounding
Now that "everyone" has a calculator that will give a result to six or eight (or more) figures, it is
important to understand how to round the answer off correctly. The typical rule taught is that you
round up with five or more and round down with four or less.
THIS RULE IS WRONG!
However, please do not rush off to your elementary school teacher…!
The problem lies in rounding "up" (increasing) the number that is followed by a 5. For example,
numbers like 2.65 or 2.75, where you are to round off to the nearest tenth.
When you round off, you change the value of the number, except if you round off a zero.
Following the old rules, you can round a number down in value four times (rounding with one,
two, three, four) compared to rounding it upwards five times (five, six, seven, eight, nine).
Suppose you had a very large sample of numbers to round off. On average you would be
changing values in the sample downwards 4/9ths of the time, compared to changing values in the
sample upward 5/9ths of the time.
This means the average of the values AFTER rounding off would be greater than the average of
the values BEFORE rounding.
Training Workshop
23-Jun-2003
Page 2
This is not acceptable.
We can correct for this problem by rounding "off" (keeping the number the same) fifty percent of
the rounding-even numbers followed by a 5. Then, on average, the rounding "off" will cancel out
the rounding "up."
The following rules dictate the manner in which numbers are to be rounded to the number of
figures indicated. The first two rules are more-or-less the old ones. Rule three is additional to the
old way.
When rounding, examine the figure following (i.e., to the right of) the figure that is to be the last
one. This figure you are examining is the first figure to be dropped.
1. If it is less than 5, drop it and all the figures to the right of it.
2. If it is more than 5, increase the number to be rounded by 1, that is, the preceding figure.
3. If it is 5, round the number so that it will be even.
The following table shows the “standard” rounding and the “even” rounding. Conventional
rounding leads to bias error.
For more info at http://dbhs.wvusd.k12.ca.us/SigFigs/SigFigs.html
Training Workshop
23-Jun-2003
Page 3
Worksheet Solution
When using the WSH formula to calculate results, a dialog pops up:
Rounding by decimals ( +nDec )
The rounding by decimals rounds to the nDec decimal place. In the format field enter a positive
value ( 0 .. 8 ) as the number of decimal places to round to.
Format
-nDec ‘2’ -> round to 2 decimals (second decimal place)
Example:
‘2’ will reformat 12.348 as 12.35
Rounding by significant figures (-nFig)
There are three rules for determining how many significant figures are in a number:
• Non-zero digits are always significant.
• Any zeros between two significant digits are significant.
• A final zero or trailing zeros in the decimal portion ONLY are significant.
Format
-nFig ‘-3’ -> round to 3 significant figures
Example:
‘-3’ will reformat 12.348 as 12.3
‘-2’ will reformat 1234 as 1200
Training Workshop
23-Jun-2003
Page 4
Rounding by range
This function is used to round numbers depending on the numeric range. Enter as many ranges as
needed separated by a comma. The number range format means:
Round to nDec (or –nFig) up to Range
from the previous range. The first format takes 0 (zero) as the previous range. The last range
should end with an ‘~’ sign (meaning “to infinity”).
Format:
nDec1#Range1, nDec2#Range2, … nDecn#~
Example:
3#0.01, 2#0.5, 1#10, -2#~
1. round to 3 decimal places from 0 up to 0.01
2. round to 2 decimal places from .01 up to 0.1
3. round to 1 decimal place from 0.5 up to 10
4. round to 2 significant figures from 10 to infinite (note here the minus sigh on –2#~)
Relative and absolute references (‘a’ or ‘A’)
To simplify the handling of formulas, Online Worksheet has the ability to handle relative and
absolute cell column references.
Relative cell references are related to the result (calculation) column and must be entered in
lowercase.
Example:
If we have an AAS formula “AAS * Volume / Weight” we can enter the formula as a*b/c,
where a=AAS, b=Volume and c=Weight (note the lowercase). When calculating and the
result data is in column “C” the system will convert the relative formula to D*E/F as
shown on the next figure.
Training Workshop
23-Jun-2003
Page 5
Absolute cell references are related to the actual grid column and must be entered in uppercase
(like in Excel). These references are not converted.
It is possible to mix absolute and relative references in one formula
=AVG( a..c )/A*B
Conclusion
You can conclude now how important it is to have a correct number formatting in a system. Like
with a 4-digit balance, we don’t would like to report the same number precision delivered by a
expensive microbalance.
a*b/c

 
Client Login





Lost Password?
No account yet? Register
Full Version Download PDF Request Info