When Two Calculations Almost Agree

Botie converts a report, then reverses the conversion to check the original readings. The recovered numbers may differ by a tiny rounding amount. An exact comparison can reject a perfectly useful calculation, but accepting every small-looking difference would also be careless.

We need to choose what agreement means for the task. That includes the arrangement of the result, the kind of numbers it stores, whether its entries are finite, and how much numerical error we allow.

Choose an example, then change a value. Follow the calculation below to see which result changes and why. These are small teaching examples; the construction later asks you to choose the operations yourself.

1

The dtype controls the representation used for every entry in an ordinary numerical array. Assigning a fraction into integer storage discards its fractional part. Converting the array afterwards cannot recover information that has already been lost.

Fixed-width integer types also have limits, unlike Python's arbitrary-precision integers. Choose a type that can represent the required range. Narrowing a type merely to save memory can change the calculation.

import numpy as np

integer_report = np.array([12,18,24], dtype=int)
integer_report[0] = 12.5
float_report = np.array([12,18,24], dtype=float)
float_report[0] = 12.5
print(integer_report.tolist())
print(float_report.tolist())
print(np.array([12,18]).astype(float).tolist())
print(np.iinfo(np.int8).max)

[12, 18, 24]
[12.5, 18.0, 24.0]
[12.0, 18.0]
127

NaN means not a number. It can mark an absent numerical observation or result from invalid arithmetic. Infinity can result from an unbounded or invalid numerical calculation. Neither is an ordinary finite thermometer reading.

NaN does not compare equal to itself. Use isnan to identify it and isfinite to require an ordinary finite value. Replacing these values is a modelling decision, so detecting them must come before silently filling them.

import numpy as np

report = np.array([0., np.nan, 18., np.inf])
print(np.isnan(report).tolist())
print(np.isfinite(report).tolist())
print(report[np.isfinite(report)].tolist())
print(np.nanmean(np.array([0., np.nan, 18.])))

[False, True, False, False]
[True, False, True, False]
[0.0, 18.0]
9.0

An absolute tolerance permits a fixed error in the result's units. A relative tolerance permits an error proportional to the reference value. np.isclose compares the difference with their combined allowance, using the second argument as the reference.

Use an explicit absolute tolerance near zero, and choose it for the required precision. allclose asks whether every aligned comparison passes. array_equal instead checks exact shape and value equality.

import numpy as np

print(0.1 + 0.2 == 0.3)
print(np.isclose(0.1 + 0.2, 0.3, rtol=0, atol=1e-12))
actual = np.array([20.0001, 18.01])
expected = np.array([20., 18.])
print(np.isclose(actual, expected, rtol=0, atol=0.001).tolist())
print(np.allclose(actual, expected, rtol=0, atol=0.001))

False
True
[True, False]
False

Approximate comparisons can broadcast compatible inputs. That is useful for some numerical checks but dangerous if a wrong result shape is supposed to fail. Confirm shape and finiteness first, then compare values.

Python's assert raises an AssertionError when a condition is false. It is useful in these learning checks and tests. For validation that must run in a deployed program, use an explicit condition and raise an error, because Python can disable assertions when run with optimisation.

import numpy as np

actual = np.array([20., 20.])
wrong_shape = np.array([[20.], [20.]])
print(np.allclose(actual, wrong_shape))
print(actual.shape == wrong_shape.shape)
expected = np.array([20., 20.])
assert actual.shape == expected.shape
assert np.isfinite(actual).all()
np.testing.assert_allclose(actual, expected, rtol=0, atol=0.001)
print("Shape, finite values, and tolerance checked.")

True
False
Shape, finite values, and tolerance checked.