Errors & standards formulas

Sign convention used throughout these notes: error = measured minus true, so a positive error means the instrument reads high. The correction is what must be added to the reading to get the true value. Some texts define the error with the opposite sign; check the convention a question uses.

Taken relative to the true value unless the question says otherwise. When the error is small, dividing by $A_m$ instead changes the result only in the second order. An accuracy quoted as a percentage of full scale is a different quantity (see limiting errors).

Accuracy is the complement of the relative error: a reading 2% in error is 98% accurate.

Change in steady output $q_o$ per change in input $q_i$; for a linear instrument, the slope of its calibration line. The unit combines output and input units.

Voltmeter of resistance $R_V$ across a source of open-circuit (Thevenin) voltage $V_s$ and internal (Thevenin) resistance $R_s$. The error is always negative (the meter reads low) and shrinks as $R_V/R_s$ grows. For a PMMC voltmeter, $R_V$ = (ohms-per-volt sensitivity) $\times$ (range in use), where ohms-per-volt $=1/I_{\text{fs}}$, the reciprocal of the full-scale current.

Ammeter of resistance $R_A$ inserted in a branch; $I_t$ is the current before insertion and $R_{\text{th}}$ the Thevenin resistance of the circuit seen from the break. The ammeter always reads low; the error is small when $R_A\ll R_{\text{th}}$.

$A_s$ = nominal or specified value, $\delta A$ = guaranteed bound, $A_a$ = actual value, which can lie anywhere in the band.

For an accuracy quoted as a fraction ${\epsilon }_{\text{FS}}$ of the full-scale value $A_{\text{FS}}$. At half scale the relative limiting error is twice the quoted figure; at one-tenth scale, ten times. Hence the range is chosen so that the reading falls in the upper part of the scale. A specification as a percentage of span uses the span (upper minus lower range value) in place of $A_{\text{FS}}$; the two differ only for a range that does not start at zero.

Absolute limiting errors add, for a difference as well as a sum. For a difference, $X$ is small when $x_1\approx x_2$, so the relative error can be very large.

Relative limiting errors add, for a quotient as well as a product.

Each relative error is weighted by the magnitude of its exponent, whether positive, negative or fractional ($k$ is an exact constant). $P=V^2/R$ gives $2{\epsilon }_V+{\epsilon }_R$; $Z_0=\sqrt{L/C}$ gives $\frac{1}{2}{\epsilon }_L+\frac{1}{2}{\epsilon }_C$.

Linearised, so valid for small errors. Use it whenever a variable appears more than once, as in $R_1{R}_2/(R_1+R_2)$: applying the product and sum rules separately treats the two appearances as independent, which can only overstate the error, never understate it.

The deviations from the mean always sum to zero, a useful arithmetic check.

Magnitudes are needed because the signed deviations cancel. For a large normally distributed set, $\bar{D}\approx 0.80\sigma$.

$\sigma$ (divide by $n$) describes a whole population or a very large set. $s$ (divide by $n-1$) is the estimate from a finite sample, because one degree of freedom is used up in computing $\bar{x}$; the difference matters only when $n$ is small. Use the one the question asks for.

Half of a large set of normally distributed readings lie within $\bar{x}\pm r$. Use the same $\sigma$ or $s$ that the question specifies.

Scatter of the mean of $n$ independent readings, each of standard deviation $\sigma$. Quadrupling the number of readings halves it; it does nothing to systematic error.

$s$ is the sample standard deviation (with $n-1$) of the $n$ readings.

$\pm a$ are the limits. Rectangular when any value within the limits is equally likely (a tolerance with no further information); a display of resolution $\Delta$ gives $a=\Delta /2$, so $u=\Delta /(2\sqrt{3})$. A certificate quoting $U$ at coverage factor $k$ gives $u=U/k$.

For $y=f(x_1,x_2,\dots )$ with uncorrelated inputs. $c_i$ is the sensitivity coefficient of input $x_i$. The same root-sum-square rule propagates the standard deviations, or the probable errors, of independent quantities: ${\sigma }_X=\sqrt{{\sum }_i(\partial X/\partial x_i{)}^2{\sigma }_i^2}$ and $r_X=\sqrt{{\sum }_i(\partial X/\partial x_i{)}^2{r}_i^2}$.

The statistical counterpart of the power rule for limiting errors: relative uncertainties weighted by the exponents, squared, added, square-rooted.

$k=2$ gives about 95% coverage and $k=3$ about 99.7% when the combined distribution is close to normal with many degrees of freedom. State $k$ with the result.

A junction irradiated at microwave frequency $f$ shows constant-voltage steps at integer $n$, each $f/K_J$ high: about $2.07\mu$V per GHz. Since 2019 $K_J$ is exact. $N$ junctions in series, each on step $n$, give $V=Nnf/K_J$; arrays of thousands of junctions give 1 V and 10 V.

In a two-dimensional electron gas at low temperature and high magnetic field, the Hall resistance takes plateau values for integer $i$, independent of the sample's dimensions and material. The $i=2$ plateau, $R_K/2\approx 12906.4\Omega$, is commonly used.

Every point equally reliable. The line passes through the centroid $(\bar{x},\bar{y})$ and the residuals sum to zero.

Same slope as above with smaller numbers. $S_{xy}=\sum (x_i-\bar{x})(y_i-\bar{y})$ and $S_{xx}=\sum (x_i-\bar{x}{)}^2$.

Only when the model is known to have zero intercept. This line does not in general pass through the centroid.

The line passes through the weighted centroid $\left(\sum w_i{x}_i/\sum w_i,\sum w_i{y}_i/\sum w_i\right)$.

Combines independent measurements of the same quantity made with different precision. The combined uncertainty is smaller than the smallest individual one; with equal ${\sigma }_i=\sigma$ it reduces to $\bar{x}$ and $\sigma /\sqrt{n}$.