← Back to FitLab
Work through the problems yourself
Botie's workshop
We will follow the problems behind machine learning with Botie, starting with the mathematics. Each lesson explains why a method was needed and what its pieces do. Then you will choose tools, connect them, and build something that solves a related problem.
Later lessons reuse those tools and introduce the next pieces. You can inspect the values moving through your construction, test different deliveries, and write the same ideas in Python.
Introductory Mathematics
Python and NumPy
- Convert the greenhouse thermometersGiving Measurements a Place
- Keep each greenhouse's averageWhich Direction Are We Adding?
- Prepare the greenhouse target sheetPreparing the Arrays We Need
- Retrieve the requested morningsChoose the Readings Without Losing Their Place
- Recover the greenhouse columns from a logger streamPut the Same Readings Into the Right Arrangement
- Compare greenhouse readings with their targetsOne Reference for Many Rows
- Find the greenhouse alertsAsk Each Row a Question
- Order the inspection report without losing its namesSummarise a Report Without Losing the Question
- Inspect every error before summarising the reportOne Calculation for a Whole Report
- Apply a rule to the greenhouse feature tableTurn Several Measurements Into Several Predictions
- Check the largest recovery errorWhen Two Calculations Almost Agree
- Replay a recorded greenhouse sampleCan We Repeat the Same Experiment?
- Reuse the training reference on another reportGive a Reusable Calculation a Clear Contract
- Recompute scores from a restored reportKeep Enough of a Calculation to Run It Again
Start with a Prediction
Data Preparation
- Audit repeated greenhouse eventsWhat Does One Row Represent?
- Fill a greenhouse report from its training referenceWhen a Measurement Is Missing
- Move the zero pointCentring on the Mean
- Give the ruler a useful unitThe Standard Score
- Make the two rulers agreeFeature Scaling
- Measure the missing featurePolynomial Features
- Describe the cloud along its main directionPrincipal Component Analysis
- Represent the greenhouse roof materialsCategories Without an Invented Ranking
- Give the greenhouse clock two coordinatesWhen the End Is Beside the Beginning
- Reconstruct the prediction-time reportWhat Could We Know at Prediction Time?
- Keep the rare sensor failures visibleWhen the Important Class Is Rare
Classical Machine Learning
- Compare people on a common scaleWhat Near Means
- When does warmer help the process?Multiple & Polynomial Regression
- Take a useful step downhillFitting by Walking
- Which coefficient can disappear?Ridge & Lasso
- Ask a few neighboursk-Nearest Neighbours
- Classify a specimen with two questionsDecision Trees
- Which digit do the scores support?More Than Two Classes
- Let the estimators share the loadBagging
- Grow trees with different viewsRandom Forests
- Repair what the first model missedGradient Boosting
- Recalculate the group centresk-Means Clustering
- Separate the inner and outer groupsThe Kernel Trick
- Keep the kernel coefficients in checkKernel Ridge Regression
- Centre the comparisonsKernel Principal Components
- Predict the next letter's categoryMarkov Chains
- Find where the light changesFilters and Edges
- Locate the reference patternTemplate Matching
- Count edges instead of brightnessHistogram of Oriented Gradients
- Is the closest match convincing?Keypoints and Descriptors
- Put the camera views togetherImage Alignment
- Give a window two checks to passHaar Cascades
Choosing and Checking a Model
- Measure the greenhouse baselineWhat Must a Model Improve On?
- Compare the two greenhouse error curvesWould More Examples Help?
- Audit the held-out greenhousesHave We Really Held Out a New Example?
- Keep preparation tied to the training dataPipelines
- Choose a setting without peekingSearching for a Setting
- Does the model depend on height?Which Feature Mattered
- Turn failure probabilities into inspection decisionsWhen Should a Probability Become an Action?
- Audit one group of failure probabilitiesDoes Eighty Percent Mean Eighty Percent?
- Compare the greenhouse population mixWhat Changed After the Model Was Fitted?
- Recalculate a resampled greenhouse meanHow Much Does Our Estimate Depend on This Sample?
Neural Networks
- Build a light-pattern detectorA Neuron
- Give two neurons different jobsA Dense Layer and the Forward Pass
- Keep each picture togetherThe Shape Guarantee
- Measure the prediction errorsLoss Functions
- Follow the effect backwardsBackpropagation
- Teach the weight from one mistakeTraining a Small Network
- Keep the average signal steadyDropout
- Normalise within each exampleNormalisation Layers
- Repair a noisy memoryHopfield Networks
- Pass a pattern through hidden unitsRestricted Boltzmann Machines
- Move a prototype towards an observationSelf-Organising Maps
- Stop a useful weight growing foreverHebbian Principal Components
- Carry something into the next stepRecurrent Layers
- Build a detector with a neighbourhoodRadial Basis Networks
- Let the layer make a correctionResidual Connections
Computer Vision with Neural Networks
- Use the same detector everywhereConvolution
- Keep the strongest local signalPooling
- Assemble the picture inspectorConvolutional Networks
- Bring the missing detail backU-Net
- Give the picture a small addressA Vector for a Picture
- Make a useful difference countLearning the Metric Itself
- Find the matching pictureSearching a Collection of Pictures
Natural Language Processing
- Give Botie a vocabularyWhat a Token Is
- Count what actually goes on the wireBytes and Characters
- What happens to two spaces?Splitting on Spaces
- Notice the punctuation in another scriptUnicode Word Boundaries
- Keep the contraction convention straightPenn Treebank Rules
- Separate the punctuation before translationMoses Rules
- Keep the word, number and punctuation apartThe Pattern Language Models Use
- Try the longest piece firstMaximum Matching
- Find the cheaper complete routeThe Word Lattice
- Recover the boundaries behind the charactersSegmenting With a Hidden Model
- Turn boundary probabilities into piecesLearning Boundaries From Examples
- Let Botie see adjacent pairsN-Grams
- Apply the merge the vocabulary learnedByte Pair Encoding
- Use a continuation pieceWordPiece
- Compare complete subword pathsThe Unigram Language Model
- Make the space visibleSentencePiece
- Choose the piece that covers more new groundGreedy Coverage
- Keep the IDs and embeddings togetherMoving a Vocabulary
- Look for reusable word partsMorfessor
- Turn an analysis into a surface wordFinite-State Morphology
- Fit characters into a small tableHashing Characters
- Group the bytes before processingPatching Without a Vocabulary
- Pick the nearest stored prototypeCodebook Quantisation
- Snap each coordinate to a levelFinite Scalar Quantisation
- Give the word more than an addressA Vector for a Word
- Compare direction without rewarding lengthDistance and Similarity
- Compress a table of word useLatent Semantic Analysis
- Ask whether the pair is surprisingPointwise Mutual Information
- Accumulate the context cluesRandom Indexing
- Retrieve and combine the word vectorsThe Embedding Layer
- Learn from the nearby wordsWord2vec
- Build a word from shared fragmentsFastText
- Fit the log of a co-occurrence countGloVe
- Tell two identical tokens apartPositional Encoding
- Let the query choose what contributesAttention
- Keep the padding out of the summaryPooling a Text
- Give a document its own memoryParagraph Vectors