Don't Peek

In a real A/B test visitors arrive day by day, and it is tempting to look at the p-value every evening and stop as soon as it drops below 0.05. This is peeking. Each look gives luck one more chance to cross the line, so peeking finds 'winners' between identical pages far more often than 5%.

To simulate the days, you test only the visitors seen so far: the slice a[:seen] gives back the first seen values of the array a. To stop at the first 'winner', break leaves the loop it sits in at once:

for n in [3, 8, 12, 5, 20]:
    if n > 10:
        print("stopped at", n)   # stopped at 12
        break

The starting code has p_value(a, b) and runs 200 experiments on identical pages: groups a and b get 2,000 visitors each (100 a day for 20 days), and both buy with a 10% chance. It counts in once how many look significant when checked once, after day 20. You add a counter daily for the experiments a peeker would stop: each day it tests the first day * 100 visitors of each group and stops at the first p below 0.05. The program must print:

Checked once, after 20 days: 12 of 200
Checked every day: 44 of 200

The starting code prints 8 for once, not 12: the daily tests take random numbers from the same rng, so the later experiments get different visitors. Both counts are close to 5%, while 44 of 200 is 22%.

Your task: add the daily checks to each experiment and count how many experiments a peeker would have stopped with a 'winner'.

Assignment
Follow the Coding Tutorial to check every day of each experiment and count the false winners peeking finds.

Hint
range(1, 21) gives the days 1 to 20. Put break inside the if, below daily += 1, so each experiment counts at most once.