How Much Does Our Estimate Depend on This Sample?

One average hides how it might change if different observations had been collected. Resampling the observed data gives us a way to study that dependence under explicit assumptions.

10 sections · 3 questions · 2 coding challenges

1 / 10

The arrow keys move between sections too.

Using Computation to Recreate the Act of Sampling

BOTIE

Statisticians often need the uncertainty of an estimate even when a convenient formula is unavailable. Repeating the entire real-world data collection may be impractical.

Efron's 1979 bootstrap paper developed a computational approach based on resampling the observed data. Our invented greenhouse readings show how replacement and repeated estimates work. Three displayed resamples make the mechanism inspectable; they are far too few to justify a useful confidence interval.

Sources for this history

Efron: Bootstrap Methods: Another Look at the Jackknife (1979)

Would Another Set of Readings Give the Same Average?

Botie estimates the average greenhouse temperature from a small collection of readings. The arithmetic gives one answer, but another collection could give a different one. We want some sense of how dependent the estimate is on the sample.

The bootstrap repeatedly samples from the observations we already have, with replacement, and recalculates the estimate. It treats the observed collection as a stand-in for the population we could sample again. That assumption is useful, but it is also a limitation.

Start with the observed readings

We have three observations. Each bootstrap resample will also contain three entries.

Observed readings

An invented example for this lesson.

1 of 3

What we can now calculate

The resample changes which observed values contribute and how often. Recalculating the mean turns that changed composition into one estimate.

Explore the full example

Choose an example, then change a value. Follow the calculation below to see which result changes and why. These are small teaching examples; the construction later asks you to choose the operations yourself.

BOTIE

Botie has a recorded bootstrap draw. Retrieve its observations with replacement and calculate the estimate for that draw.

Choose an example

Original observations

Recorded bootstrap indices

Result from these values

16.6667

Follow the operations

1. Original observations
1. 010
2. 120
3. 230
2. Recorded bootstrap indices
1. 00
2. 10
3. 22
3. Gather selected entries
1. 010
2. 110
3. 230
4. Mean
```
16.6667
```
5. Result
```
16.6667
```

The construction receives a fixed draw and performs no random sampling. A few illustrative means are not a confidence interval. Editing this example does not change the workshop tests.

1. Treat the Observed Sample as a Temporary Population

For an ordinary nonparametric bootstrap, draw as many observations as the original sample contains, with replacement. Each original observation has equal chance on each draw.

Replacement is essential. Sampling every observation once without replacement would only reorder the same sample and leave its mean unchanged.

Original sample: [10, 20, 30]
A valid resample: [10, 10, 30]
Another resample: [20, 20, 20]

Repeated values are repeated selections of the same observed value. They are not new sensor measurements.

2. Recompute the Complete Statistic

For each resample, repeat the statistic of interest. Here that statistic is the mean. For more complicated estimates, the required procedure can include refitting a model or preprocessing rather than merely resampling an already summarised number.

First mean = (10 + 10 + 30) / 3 ≈ 16.6667
Second mean = (20 + 20 + 20) / 3 = 20
Third mean = (10 + 30 + 30) / 3 ≈ 23.3333

These three fixed examples explain the mechanism. A real bootstrap uses many random resamples and checks whether the summary is stable enough for its purpose.

3. Summarise Variation Without Overclaiming

The spread of bootstrap estimates can estimate a standard error under appropriate conditions. Their quantiles can support a percentile interval, although different interval methods have different assumptions and accuracy.

Do not call the range of three hand-picked resample means a 95 percent confidence interval. Neither their number nor their selection supports that interpretation.

A confidence interval is a property of a repeated-sampling procedure. It is not a guarantee that every future individual greenhouse reading lies inside it.

4. Resample the Independent Units

Ordinary row resampling assumes the observed rows are suitable independent sampling units. Consecutive readings from one greenhouse may be dependent. Group or block resampling can preserve some of that structure when justified.

Resampling also cannot repair a biased collection process or create examples of conditions absent from the original data. If every original reading came from one season, bootstrap copies do not supply evidence about another season.

The workshop gathers one supplied resample and calculates its mean. Python then processes several recorded draws, making the distinction between a draw and its estimate explicit.