Turn Several Measurements Into Several Predictions
A prediction can combine temperature and humidity with separate coefficients. Matrix operations let us apply that same rule to every greenhouse, then check how a fitted rule relates to its observations.
10 sections · 4 questions · 2 coding challenges
The arrow keys move between sections too.
Reusing Numerical Linear Algebra for Many Observations
BOTIE
NumPy's linear algebra interface draws on established BLAS and LAPACK routines. We use that interface after making the small matrix calculation visible, and distinguish solving an exact square system from fitting an overdetermined one.
Sources for this history
How Can One Rule Read a Whole Table?
Botie has two measurements for every greenhouse. A simple illustrative score gives each measurement a coefficient, adds its contributions, then adds an offset. We can calculate one greenhouse's score by hand before doing the same for a whole table.
This is where the distinction between elementwise multiplication and matrix multiplication matters. We will keep the observations, feature columns, coefficients, and output shapes visible at every step.
Keep one observation per row
These are invented temperature and humidity features.
North
- 10
- 40
South
- 20
- 60
What we can now calculate
Matrix times vector collects each row's weighted contributions. The offset applies to every resulting score.
Explore the full example
Choose an example, then change a value. Follow the calculation below to see which result changes and why. These are small teaching examples; the construction later asks you to choose the operations yourself.
BOTIE
Result from these values
- 027
- 149
Follow the operations
1. Observation rows
10 40 20 60 2. Feature coefficients
- 02
- 10.1
3. Matrix times vector
- 024
- 146
4. Shared offset
3
5. Add
- 027
- 149
6. Result
- 027
- 149
The coefficients are supplied. Python separately introduces exact solving and least-squares fitting. Editing this example does not change the workshop tests.
1. Inspect the Contributions Before Summing
Elementwise multiplication combines matching entries without adding them. A dot product multiplies matching entries and then adds those products. For one observation, the feature and coefficient vectors must have equal lengths.
The coefficients in this example are supplied solely to illustrate the calculation. They are not a trained rule or a physical greenhouse control law.
Calculate one greenhouse's score
The temperature contributes through its own coefficient, and humidity through another. We print those contributions before the sum and the offset.
import numpy as np
features = np.array([10.,40.])
weights = np.array([2.,0.1])
contributions = features * weights
print(contributions.tolist())
print(np.dot(features, weights))
print(np.dot(features, weights) + 3)[20.0, 4.0]
24.0
27.0The elementwise product has two entries. The dot product reduces them to one weighted sum. Adding the offset completes the score.
2. Apply the Same Rule to Every Row
The @ operator performs matrix multiplication. For a two-dimensional feature table and a one-dimensional coefficient vector, it computes one dot product per row. The feature count must match the coefficient count.
For matrix times matrix, the left matrix's column count must match the right matrix's row count. The surviving outer dimensions describe the output.
Keep one prediction per greenhouse
Our table has two observations and two features. Multiplying by a two-entry coefficient vector leaves one score per observation. Multiplying by the identity leaves the feature table unchanged.
import numpy as np
X = np.array([[10.,40.],[20.,60.]])
weights = np.array([2.,0.1])
scores = X @ weights + 3
print(scores.tolist())
print(scores.shape)
print((X @ np.eye(2)).tolist())[27.0, 49.0]
(2,)
[[10.0, 40.0], [20.0, 60.0]]Using X * weights would instead leave a contribution for each feature, with two columns still present. It is useful intermediate data, but it is not the same result as @.
3. Check Distance With a Norm
A norm measures the size of a vector. The Euclidean norm squares its coordinates, adds them, and takes the square root. np.linalg.norm performs this calculation; naming an axis lets us calculate it separately for each row.
When coordinates are different kinds of measurement, a raw distance can be dominated by their units. Later scaling lessons explain how to make such comparisons meaningful.
One length for each error vector
These invented vectors are already on comparable scales. Reducing along the feature axis gives one length for each observation.
import numpy as np
errors = np.array([[3.,4.],[0.,5.]])
print(np.linalg.norm(errors, axis=1).tolist())
print(np.sqrt(np.square(errors).sum(axis=1)).tolist())[5.0, 5.0]
[5.0, 5.0]The second expression exposes the steps performed by the norm. Its five-unit result describes length, not an average coordinate error.
4. Solve an Exact System Before Fitting an Approximation
If a square coefficient matrix has an independent constraint for each unknown, solve can find values satisfying that system. Do not explicitly compute an inverse merely to apply it to one right-hand side. A singular matrix does not supply a unique solution.
With more observations than unknowns, exact agreement may be impossible. lstsq finds coefficients minimising the sum of squared residuals. It also returns information about the fit and the rank, the number of independent directions supported by the matrix.
Recover two settings and check the equations
The supplied equations require the sum of two settings to be thirty and their difference to be ten. We check the computed solution by putting it back into the equations.
import numpy as np
A = np.array([[1.,1.],[1.,-1.]])
b = np.array([30.,10.])
settings = np.linalg.solve(A, b)
print(settings.tolist())
print((A @ settings).tolist())[20.0, 10.0]
[30.0, 10.0]Recomputing the left-hand side confirms what the settings do. Floating-point systems generally need a tolerance check, which the numerical-checks lesson will introduce.
Fit a line when there are more observations than coefficients
The first column is all ones for the intercept; the second is the input. lstsq returns four items, which we unpack into separate names. We will inspect the coefficients, predictions, and rank.
import numpy as np
x = np.array([0.,1.,2.])
y = np.array([1.,2.,2.])
X = np.column_stack([np.ones_like(x), x])
weights, residual_sums, rank, singular_values = np.linalg.lstsq(X, y, rcond=None)
print(np.round(weights, 4).tolist())
print(np.round(X @ weights, 4).tolist())
print(rank)[1.1667, 0.5]
[1.1667, 1.6667, 2.1667]
2The line cannot pass through all three observations. It balances their squared residuals. The returned residual_sums is not the array of individual errors; calculate those from predictions and observations when you need to inspect them. Near-dependent features can make fitted coefficients sensitive even when the program returns a result.
Check yourself
Questions on Parts 1 to 4
Question 1 of 4
Which result has one weighted score per row?
Use this setup. The code is given; its result is for you to work out.
import numpy as np
X = np.array([[10.,40.],[20.,60.]])
weights = np.array([2.,0.1])Try it with Botie
Build with Botie
Write and run Python