The OHLC contract

Every module in edgekit agrees on one bar-frame shape. Fix that shape once — a UTC DatetimeIndex, float OHLC columns, monotone and de-duplicated — and every indicator, engine, and validator downstream can assume it without re-checking. This page is that contract and the data plumbing that enforces it.

The shape#

A bar frame that edgekit will accept is:

• A pandas.DataFrame indexed by a DatetimeIndex.
• Timestamps that are UTC and tz-naive by convention — the frame carries no tzinfo, but every timestamp means UTC (matching the HistoryExporter CSVs the project uses).
• Float columns open, high, low, close; volume (or tick_volume from the loaders) optional.
• Sorted ascending in time with no duplicate timestamps.

The single gate that normalises anything into this shape is as_ohlc. It validates the required columns, checks the index type, sorts, and drops duplicate timestamps — but deliberately does notresample or fill, because inventing bars is the caller's decision, not a silent side effect:

OHLC = ("open", "high", "low", "close")

def as_ohlc(df):
    """Validate and normalise a bar frame to the OHLC contract.
    Sorts the index, drops duplicate timestamps (keep first), checks columns.
    Does not resample or fill — that is the caller's decision."""
    missing = [c for c in OHLC if c not in df.columns]
    if missing:
        raise ValueError(f"bars missing required columns {missing}; have {list(df.columns)}")
    if not isinstance(df.index, pd.DatetimeIndex):
        raise TypeError(f"bars must have a DatetimeIndex, got {type(df.index).__name__}")
    out = df.sort_index()
    out = out[~out.index.duplicated(keep="first")]
    return out

Both backtest engines call as_ohlc on entry, so the monotone-and-unique invariant every indicator assumes is guaranteed at the boundary, not hoped for.

Loading bars#

load_bars reads a HistoryExporter CSV (time_utc,open,high,low,close,tick_volume) or a processed parquet split. Its schema is stricter than the core contract — it requires tick_volume, because the source loaders and the downstream integrity/profile code consume it:

REQUIRED_COLUMNS = ["open", "high", "low", "close", "tick_volume"]

def load_bars(path):
    if str(path).endswith(".parquet"):
        df = _read_parquet(path)                 # needs the optional pyarrow extra
    else:
        df = pd.read_csv(path, parse_dates=["time_utc"], index_col="time_utc")
    missing = [c for c in REQUIRED_COLUMNS if c not in df.columns]
    if missing:
        raise ValueError(f"{path}: missing columns {missing}")
    df = df.sort_index()
    df = df[~df.index.duplicated(keep="first")]
    return df

import edgekit as ek

bars = ek.data.load_bars("BTCUSDT_5m.csv")   # tz-naive UTC index, float OHLC + tick_volume
bars = ek.as_ohlc(bars)                       # belt-and-braces: sorted, unique, validated

Resampling#

Downsampling to a swing timeframe has exactly one correct aggregation: first open, max high, min low, last close, sum volume. Anything else — a mean close, say — fabricates prices that never traded. edgekit encodes this as OHLCV_AGG and exposes named timeframes through TIMEFRAME_RULES:

OHLCV_AGG = {"open": "first", "high": "max", "low": "min", "close": "last",
             "tick_volume": "sum"}

TIMEFRAME_RULES = {"M15": "15min", "M30": "30min", "H1": "1h", "H4": "4h",
                   "D1": "1D", "W1": "W-FRI"}

Two conventions in that table are load-bearing. The weekly bar is anchored on Friday (W-FRI) so a "week" closes at the Friday session end and keeps the weekend gap out of the bar body. And resample_ohlcv defaults to label="left" / closed="left" — each bar is stamped with the timestamp of its open and includes the left edge. That is the causalconvention: a strategy acting at a bar's timestamp only ever sees bars that have already closed.

import edgekit as ek

m5 = ek.data.load_bars("BTCUSDT_5m.csv")
h4 = ek.data.resample_ohlcv(m5, "H4")          # named rule -> "4h", causal left-labelled

# empty weekend buckets are dropped by requiring positive volume on D1/W1
weekly = ek.data.resample_ohlcv(m5, "W1")      # Friday-anchored weekly bars

Sessions#

Intraday work needs session masks, and the one rule that cost the research project real money is: never anchor a session on a fixed UTC hour. The US cash open drifts an hour twice a year with daylight saving. edgekit converts through zoneinfo so 09:30 is always 09:30 in New York:

import edgekit as ek

bars = ek.data.load_bars("US100_M1.csv")

ny = ek.data.to_ny(bars.index)                       # DST-correct America/New_York clock
mask = ek.data.rth_mask(bars.index,                  # regular trading hours 09:30–16:00 NY
                        start="09:30", end="16:00")
rth = bars[mask]                                     # cash-session bars only

rth_mask is left-inclusive / right-exclusive (start ≤ t < end), so the 16:00 close bar is excluded — matching the ORB and volume-profile session idiom — and it handles overnight windows (e.g. a Sydney session where start > end) by OR-ing the wrap-around. All of it is built on minute_of_day, which reads the wall clock in the target timezone rather than a fixed offset.

Integrity#

Before trusting a file, ask what it actually is. integrity_report describes the span, classifies gaps, and flags suspect bars — crucially separating weekend gaps (normal for FX: Friday → Sunday/Monday) from intraweek gaps, which mean missing data:

import edgekit as ek

rep = ek.data.integrity_report(bars, bar_minutes=5)
print(rep["bars"], rep["span_days"])
print(rep["weekend_gaps"], rep["intraweek_gaps"])          # normal vs. suspicious
print(rep["intraweek_gap_bars_missing"])                    # how many bars are actually absent
print(rep["bad_ohlc_bars"])                                 # high<low, high<open, etc.
print(rep["spike_bars_range_gt_20x_median"])                # candidate bad ticks

Sealing the holdout#

The data layer also owns the discipline that keeps validation honest. chronological_split carves the frame into train / validation / holdout by position (never by shuffling), so the arrow of time is preserved and the holdout is a contiguous final slice you read exactly once, at the very end:

import edgekit as ek

sp = ek.data.chronological_split(bars, train=0.6, val=0.2)
# sp.train, sp.val, sp.holdout — the last 20% is sealed; tuning on it is forbidden

See also#

• edgekit.data — load_bars, resample_ohlcv, sessions, integrity, splits, fetch, cache.
• edgekit.core — as_ohlc and the contract every module agrees on.
• Causality — why the resample defaults are left-closed.