Why statistics, and why simulate

Every dataset is a sample of something larger. The 2,000 users in this month's experiment stand in for all future users. The 300 survey responses stand in for all customers. Statistics answers one central question: how much can we trust what this sample tells us about the bigger picture?

A small example

You flip a coin ten times and get seven heads. Is the coin biased? Your intuition probably says "not necessarily, that can happen by chance". Statistics lets us say how often it happens by chance, and therefore how surprised we should be.

We could look up a formula. Or we can simulate:

import numpy as np

rng = np.random.default_rng(42)
flips = rng.integers(0, 2, size=(100_000, 10))
heads = flips.sum(axis=1)
(heads >= 7).mean()

This performs 100,000 experiments of ten fair flips and asks what share produced seven or more heads. The answer is about 0.17. Roughly one experiment in six gives a result at least that extreme with a perfectly fair coin. Seven heads in ten flips is weak evidence of bias.

Why simulation helps

• It makes abstract ideas concrete. You can see a distribution form.
• It works when formulas do not exist or their assumptions fail.
• It lets you check your understanding: if your formula and your simulation disagree, one of them is wrong, and finding out which teaches you something.

Modern computers make this cheap. A hundred thousand simulated experiments take milliseconds.

Random numbers in NumPy

We use NumPy's Generator API, created with np.random.default_rng(seed). Setting a seed makes results reproducible, so your notebook gives the same numbers every time you run it and others can check your work. Use one generator per notebook and pass it around, rather than reseeding repeatedly.

What this course will not do

We will not turn every question into a test with a p value. Many of the most useful statistical skills are simpler: plotting the data, looking at spread as well as averages, and asking how the data was collected. A biased sample cannot be fixed by any formula.

In the next lesson we start with describing data well, the foundation for everything else.