An honest event study

“The stock rallies after X” is the easiest claim in finance to manufacture and the hardest to make stick. This guide builds an event study that could survive a referee: events defined by a mechanical rule, abnormal returns via ek.factors.event_study, a bootstrap null band from pseudo-events, and a multiple-testing correction for every definition you tried along the way.

→The two ways event studies lie

First: event selection. If a human picked the event dates — or picked the rule after eyeballing the chart — the study is measuring hindsight. Second: a missing null. Any average of returns over any set of dates produces a wiggly CAR curve; without knowing what random dates produce on the same series, the wiggle means nothing. This guide closes both holes, in that order.

Step 1 — define events mechanically#

The rule must be writable before looking at outcomes, and computable from data available at the event time. Here: volatility shocks in a single name, detected by the CUSUM filteron standardised return moves — a symmetric, parameter-light way to say “something happened” without a human in the loop. (Scheduled events — earnings dates, index rebalances — are even better, because the calendar defines them for you.)

event_study.py

import numpy as np
import pandas as pd
import edgekit as ek
from edgekit.core import bootstrap_rng

bars = ek.data.fetch("stooq", "aapl.us", interval="d", start="2012-01-01")
r = bars["close"].pct_change().dropna()

events = ek.timeseries.cusum_breaks(r, threshold=5.0)   # 5-sigma cumulative moves
print(f"{len(events)} events over {len(r)} bars")
# 23 events -> roughly two per year: rare enough to mean something

threshold is your one honest degree of freedom, and it is spent now. Decide it here, write it down, and if you later try other values, that is a fact the correction in step 4 must know about.

!Causality: the event date must not know the outcome

Whatever the rule, check it only reads data up to the event bar. A detector that uses the full sample to standardise (a global mean or vol) leaks the future into every event date — and post-event returns measured from leaked dates are the subtlest form of look-ahead. If you will trade the effect, the estimation window must also end before day 0, which is exactly how event_study positions it.

Step 2 — the event study itself#

event_studyaligns a window of returns around every event, subtracts each event’s own estimation-window mean (the “normal” return — so a stock that drifts up 10 bps a day does not fake an event effect), and averages across events into the AAR and its cumulative sum, the CAR.

res = ek.factors.event_study(r, events, window=(-5, 10), estimation=60)

print(f"n_events {res['n_events']}   CAR({res['car'].index[-1]:+d}) "
      f"{res['car'].iloc[-1]:+.2%}   t_car {res['t_car']:+.2f}")
# n_events 23   CAR(+10) +1.84%   t_car +2.31
res["aar"]   # average abnormal return by relative day (-5..+10)
res["car"]   # its cumulative sum — the headline curve

Output	Meaning
`aar`	Average abnormal return at each relative day, market-adjusted by the estimation window.
`car`	Cumulative AAR across the window — the curve everyone plots.
`t_car`	t-statistic of the final CAR across events — assumes independent, well-behaved events.
`n_events`	Events actually used (those with full windows). Below ~15, stop here.

A CAR of +1.8% with t_car = 2.3 looks done. It is not. The t-stat leans on assumptions your data laughs at — independent events, homogeneous variance — and it knows nothing about how many rules you tried. Two more gates.

Step 3 — the bootstrap null band#

The cleanest null for “do these dates matter?” is other dates: draw the same number of pseudo-event dates at random from the same eligible range, run the identical event_study, and repeat until you have the distribution of CAR curves that pure chance produces on this series — same fat tails, same vol clustering, same drift. The real CAR earns belief only where it exits that band.

rng = bootstrap_rng()
eligible = r.index[60:-10]                     # room for estimation + window
n_boot = 1000

null_cars = np.empty((n_boot, len(res["car"])))
for b in range(n_boot):
    pseudo = list(rng.choice(eligible, size=res["n_events"], replace=False))
    null_cars[b] = ek.factors.event_study(
        r, pseudo, window=(-5, 10), estimation=60)["car"].to_numpy()

lo, hi = np.percentile(null_cars, [2.5, 97.5], axis=0)   # pointwise 95% band
outside = res["car"].to_numpy() > hi                      # one-sided: rally claim
print(f"days outside band: {outside.sum()} of {len(outside)}")
print(f"final CAR {res['car'].iloc[-1]:+.2%} vs null 97.5% {hi[-1]:+.2%}")

And extract a proper p-value for the final CAR — the fraction of pseudo-studies that beat it, with the +1 correction so p can never be exactly zero:

p_boot = (np.sum(null_cars[:, -1] >= res["car"].iloc[-1]) + 1) / (n_boot + 1)
print(f"bootstrap p = {p_boot:.4f}")   # e.g. 0.021

!Read the band before the p-value

Two curves with the same final CAR are not the same evidence. A CAR that steps up on days 0–2 and holds outside the band is an event effect; one that drifts out on day +9 after hugging the null the whole way is the tail of the band doing the work. And if the CAR is already outside the band on days -5 to -1, your “events” are detected after the move they claim to predict — a look-ahead in the event definition, and with CUSUM-detected events the default suspicion.

Step 4 — pay the multiple-testing bill#

The quiet killer. You tried threshold=4 before 5; you looked at (-5, 5) before (-5, 10); you ran three tickers and are showing the good one. Each variant was a lottery ticket against the null, and the p-value must be charged for all of them — including the ones you abandoned. Count every combination you evaluated and Bonferroni-correct:

n_tried = 6          # 2 thresholds x 3 windows actually evaluated — count honestly
p_adj = min(1.0, p_boot * n_tried)
print(f"adjusted p = {p_adj:.3f}")
# 0.021 * 6 = 0.126  ->  NOT significant. This is the honest answer.

This is the step that reclassifies most event studies from “finding” to “noise” — and it is not pedantry. A p of 0.02 from one pre-registered test and a p of 0.02 from the best of six are different objects; the second is expected under the null about one run in eight. If the correction kills it, the fix is not a smaller n_tried — it is fresh data: freeze the exact rule, run it on a holdout period or sibling tickers it has never seen, and let a single confirmatory test carry the claim (see why backtests lie).

The verdict checklist#

Believe the effect only when all four hold:

Mechanical events — a rule (or a calendar), written before outcomes were seen, causal at event time.
Clears the band — the CAR exits the bootstrap null envelope, promptly after day 0, not before it.
Survives the correction — p_boot * n_tried stays under your alpha, with n_tried counted against yourself.
Replicates — the frozen rule works on data that had no vote in choosing it.

Our worked example passed three gates and failed the fourth-to-last: mechanically defined, outside the band, dead on the correction. That is not a failed study — that is the method working: an honest “not proven, here is exactly what confirmatory data would settle it” beats a publishable artifact every time you intend to trade the result.

Next#

Event studies — AAR/CAR theory, estimation windows, and the classic designs.
Why backtests lie — selection bias and multiple testing, the deep dives.
API · factors — event_study alongside fama_macbeth and newey_west.
API · timeseries — cusum_breaks and the rest of the event-detection toolkit.