Re: Question: Significant digits?

[Jim Purcell <jimpur -at- MICROSOFT -dot- COM>]

Jennifer Kraus asks:

> I'm editing =
> a manual with measurements expressed in both "American" and SI units.
> =
> Our product manager wants to go by "significant digits," because he
> says =
> that will help conversions to remain consistent. So, in his thinking,
> =
> this is how the following conversions would be expressed:
>
> 250 gallons=3D940 liters       as opposed to    946 Liters
> 500 gallons=3D1890 liters     as opposed to   1893 Liters
> 30-125 psi=3D2.1-8.62 bar   as opposed to    2.06-8.62 bar
>
> This looks weird to me, but if it's a standard way of doing things, I
> =
> can go with it. But here's where I get confused. A lot of our
> literature =
> expresses flow rates in gallons per minute...commonly without using a
> =
> decimal. So do I just round up the liters? That seems so sloppy.=20
>
I've never seen this "3D" notation before, and I think there's also a
bit of confusion here as to what constitutes a significant figure.
Significant figures refer to the precision of a number. If a reference
to 250 gallons means 245-255 gallons, that's two significant figures. If
it means 249.5-250.5 gallons, that's three significant figures. If it
means _exactly_ 250 gallons, that's infinite significant figures,
because you're really saying 250.0000000... gallons.

The general rule is that the result of a calculation is no more precise
than its least precise member. If you are converting _exactly_ 250
gallons to liters, you have as many significant figures as your
conversion factor has. To get four significant figures:

        250.00 gallons = (250.00 * 3.785) liters = 946.25 liters

The result has five digits, but only the first four are significant
because the least precise number in the calculation (3.785) has four
significant figures. You could therefore say that 250.00 gallons = 946.2
liters (or 946.3 if you round up).

> To complicate things further...I have two separate flow rates listed
> in =
> two different areas of a performance data sheet. One flow rate is 0.6
> =
> gpm, the other is 1 gpm. How do I apply significant digits in this =
> situation? Do I treat each individually, using decimals for the first
> =
> conversion to liters and no decimals for the second? Or do I go by the
> =
> 0.6 gpm and use decimals for both metric conversions?
>
If both of these numbers are accurate to one significant figure, you
have to say 0.6 gpm and 1 gpm. If you say 1.0 gpm, you are expressing
two significant figures, which is more precision that you really have.
BTW, the conventional way to indicate that all trailing zeros are
significant is to end the number with a decimal point:

        250. gal

If some trailing zeros are significant but not others, you can either
explicitly say how many digits are significant or you can use scientific
notation:

        1,350,000 kg to four significant figures
--or--
        1.350 * 10**6 kg (Well, it's easier if you can use
superscripts.)

> I may be making a mountain of a molehill here...but I'm trying to set
> =
> standards for our literature, and I want to make sure I understand the
> =
> principle on which this idea of significant digits is based. I looked
> in =
> Chicago, Brusaw's Handbook of Technical Editing, and Eisenberg's Guide
> =
> to Technical Editing, and couldn't find anything that helped me.=20
>
Significant figures are very important, especially when you are dealing
with computers that will generate floating point numbers with more
decimal places than anyone ought to know about.

> So...anyone used this system? Anyone used an alternative system?
> Anyone =
> know of a reference that could help me out here? I would be very =
> greatful (as always) for your help.=20
>
A good college physics textbook (Halliday and Resnick was current in my
day, which is probably dating me) should have a discussion of
significant figures.

Hope this helps,

Jim Purcell
jimpur -at- microsoft -dot- com
My opinions, not Microsoft's

 TECHWR-L (Technical Communication) List Information: To send a message
to 2500+ readers, e-mail to TECHWR-L -at- LISTSERV -dot- OKSTATE -dot- EDU -dot-  Send commands
   to LISTSERV -at- LISTSERV -dot- OKSTATE -dot- EDU (e.g. HELP or SIGNOFF TECHWR-L).
  Search the archives at http://www.documentation.com/  or search and
browse the archives at http://listserv.okstate.edu/archives/techwr-l.html