Keep Enough of a Calculation to Run It Again

Keeping Numerical Results Usable by the Next Program

BOTIE

Scientific programs need to exchange data and repeat earlier calculations. Plain text is inspectable, but it does not automatically preserve every array's type, shape, labels, or full numerical precision.

NumPy's NPY format records an array's dtype and shape with its values. NPZ archives collect named arrays, while text readers and writers serve simpler interchange needs. These formats preserve stored data, not the scientific meaning we forget to record.

Sources for this history

What Will Tomorrow's Program Need From Today's Work?

Botie has prepared a feature table, fitted coefficients, and kept the greenhouse names. Tomorrow's program must load those pieces in the same order. Saving only a screenshot or a rounded list of predictions would not be enough to repeat the calculation.

NumPy can save one array in its native format, keep several arrays in a named archive, or exchange numbers as text. We will try each route and check both the values and the information needed to interpret them.

Keep the model inputs

Observation rows must remain attached to their feature columns.

North features

10
40

South features

20
60

An invented example for this lesson.

1 of 3

What we can now calculate

The restored coefficients produce one score per row, and the restored names retain the row identity.

Explore the full example

Choose an example, then change a value. Follow the calculation below to see which result changes and why. These are small teaching examples; the construction later asks you to choose the operations yourself.

BOTIE

Botie has already loaded the saved feature table, coefficient vector, and greenhouse names. We need to check that those pieces still work together.

Choose an example

Restored feature table

1

2

1

2

Restored coefficients

1

2

Restored names in row order

1

2

Result from these values

[[24,46],["north","south"]]

Follow the operations

1. Restored feature table
10 40
20 60
2. Restored coefficients
1. 02
2. 10.1
3. Matrix times vector
1. 024
2. 146
4. Restored names in row order
1. 0north
2. 1south
5. Collect two results
```
[[24,46],["north","south"]]
```
6. Result
```
[[24,46],["north","south"]]
```

The workshop starts after loading and checks how the restored data is used. The Python examples and challenges perform the actual file-format round trips in memory. Editing this example does not change the workshop tests.

1. Save the Array, Not Only Its Printed Appearance

np.save writes one array in NPY format. np.load reads it back, including the dtype and shape. A printed string is intended for people; a saved numerical array is intended for another calculation.

These browser examples use BytesIO, a binary file held in memory. It follows the same read/write interface as a file without writing to your computer. Its contents are temporary and disappear when the Python runtime is reset. For a local Python program, a file path can be supplied instead.

Write an array and read it back

BytesIO comes from Python's io module. Writing advances the file position, so seek(0) returns to the beginning before loading. We disallow pickle because these ordinary numerical arrays do not need Python-object deserialisation.

Python

import numpy as np

from io import BytesIO

readings = np.array([[12.,18.],[14.,22.]], dtype=np.float64)
file = BytesIO()
np.save(file, readings, allow_pickle=False)
file.seek(0)
restored = np.load(file, allow_pickle=False)
print(restored.tolist())
print(restored.shape)
print(restored.dtype)
print(np.array_equal(restored, readings))

Output

[[12.0, 18.0], [14.0, 22.0]]
(2, 2)
float64
True

The restored array has the same values, shape, and numerical representation. That does not tell it whether a column was temperature or humidity; metadata must supply those meanings.

2. Keep Related Arrays Under Explicit Names

An NPZ archive can hold several arrays. Keyword arguments to savez choose names by which they can be retrieved. This is useful when a feature table, a coefficient vector, and observation labels belong to the same calculation.

The archive is a resource that should be closed after reading. A with statement keeps it open for the indented block and closes it when the block ends. Keep any arrays needed later by assigning them within that block.

Restore the pieces before recomputing the result

The labels are a regular NumPy string array, not arbitrary Python objects. Dictionary-like key lookup retrieves each named array from the archive.

Python

import numpy as np

from io import BytesIO

X = np.array([[10.,40.],[20.,60.]])
weights = np.array([2.,0.1])
names = np.array(["north","south"])
file = BytesIO()
np.savez(file, features=X, weights=weights, names=names)
file.seek(0)
with np.load(file, allow_pickle=False) as saved:
    restored_X = saved["features"]
    restored_weights = saved["weights"]
    restored_names = saved["names"]
print(restored_names.tolist())
print((restored_X @ restored_weights).tolist())

Output

['north', 'south']
[24.0, 46.0]

Names remain aligned with the rows. savez_compressed writes a compressed NPZ archive when reducing stored size is useful, with additional work for compression and decompression.

3. Read a Small Text Table Deliberately

Text is convenient for inspecting and sharing rows, but a reader must know the delimiter, which lines are headers, and what type to use. loadtxt is suitable for a regular numeric table with the stated format. genfromtxt provides additional handling for missing entries.

StringIO is the text counterpart to BytesIO. It lets this example read a tiny CSV entirely in memory. On disk, the reader can take a path instead.

Keep the header separate from the numerical rows

Three quotes enclose a string spanning several lines. The delimiter says that commas separate fields. skiprows skips the one header line; it does not turn those names into automatic labels on the returned array.

Python

import numpy as np

from io import StringIO

csv = StringIO("""temperature,humidity
12,50
18,60
""")
features = np.loadtxt(csv, delimiter=",", skiprows=1, ndmin=2)
print(features.tolist())
print(features.shape)

Output

[[12.0, 50.0], [18.0, 60.0]]
(2, 2)

The feature names still need to be stored separately. ndmin=2 keeps a table-shaped result even if a future report contains just one numerical row.

4. Record What a File Format Cannot Infer

A round-trip check should compare the loaded shape, dtype, values, and label ordering with the originals. Units, feature meanings, random settings, and library versions belong in the experiment record too. A saved array cannot reconstruct assumptions that were never recorded.

When writing text with savetxt, the format controls precision. A small number of decimal places may be fine for a human report, but it deliberately discards information needed for an exact numerical replay.

Show the difference between a report and an exact record

We write two decimal places to a text file in memory, then reload it. allclose can accept the stated display error, while exact equality correctly reports that the original values were not fully preserved.

Python

import numpy as np

from io import StringIO

original = np.array([[12.345,50.],[18.765,60.]])
text_file = StringIO()
np.savetxt(text_file, original, delimiter=",", fmt="%.2f")
text_file.seek(0)
restored = np.loadtxt(text_file, delimiter=",", ndmin=2)
print(restored.shape == original.shape)
print(np.array_equal(restored, original))
print(np.allclose(restored, original, rtol=0, atol=0.01))

Output

True
False
True

That loss of precision was a choice made by the writer. Use a native array file when exact preservation of the stored numerical entries is required.

Check yourself

Questions on Parts 1 to 4

Question 1 of 4

What must happen before reading this in-memory file from its start?

Use this setup. The code is given; its result is for you to work out.

Given

import numpy as np

from io import BytesIO
file = BytesIO()
np.save(file, np.array([12.,18.]), allow_pickle=False)

Try it with Botie

Build with Botie

Write and run Python