Out-of-Sample Order-Flow Hypotheses

design = 2025 · test = 2026 · 58 crypto perps (51 cross-market) · 1h bars · generated 2026-10-10

Headline

NO EDGE (H0) All four order-flow hypotheses were re-tested on a genuine temporal split and all four fail. Every nominated effect is indistinguishable from zero — or inverts — out of sample.

This is the decisive comparison: the earlier runs used a short 2026-only design window, which manufactured small positive design-ICs. Replacing that window with a real 2025 design collapses every one of them to ~0 (or negative). The signals were in-sample artifacts.

The design-window artifact, in one table

Study	2026-only design IC	2025 design IC	2026 test IC (OOS)	Test 95% CI	OOS verdict
Spot–perp divergence	+0.0248	+0.0026	+0.0019	[−0.0091, +0.0120]	NO EDGE
H1 — price absorption	+0.0049	−0.0097	−0.0103	[−0.0206, +0.0014]	NO EDGE
H2 — failed sweeps	+0.0495	−0.0024	+0.0062	[−0.0201, +0.0354]	NO EDGE
H3 — flow-confirmed breakouts	−0.0296	+0.0203	−0.0206	[−0.0551, +0.0105]	NO EDGE

Read down the “2026-only design” → “2025 design” columns: divergence +0.0248→+0.0026, sweeps +0.0495→−0.0024. The apparent edge lives entirely in the short design slice and vanishes the moment the design window is real. (H1 and H3 were negative or unstable to begin with.)

Method (unchanged from the frozen H-series)

Signals at the close of bar t; outcomes from t+1 (1-bar delay); prior-range levels exclude the current bar.
Pooled rank IC of signal vs oriented forward return, with block-bootstrap CIs; matched controls; control regression; chronological split.
Universe: 58 crypto perps with 2025+2026 history (51 also spot, for divergence). 1h bars from 1m. Cost bar 12 bps.
Split: design = 2025-01-01 → 2026-01-01, test = 2026-01-01 → 2026-10-01.

1 · Spot–Perp CVD Divergence

51 cross-marketNO EDGE

Signal. D = z(spot flow) − z(perp flow), 60-bar window, 720-bar normalisation; predicted sign(D) → forward return. Outcome: market-neutral, oriented 4h return.

Period	Pooled IC	Cross-sectional IC	n
Design (2025)	+0.0026	+0.0001	415,741
Test (2026)	+0.0019	−0.0014	327,600

Test CI [−0.0091, +0.0120]. Decile monotonicity −0.21; decile spread −0.44 bps; IC ex-BTC/ETH +0.0010.

Preregistered criteria

C1 IC>0.02 & CI>0	C2 sign-stable	C3 monotone	C5 cost-clearing	C6 leave-one-out	C7 ex-megacap
✗	✓	✗	✗	✓	✓

Sign is stable and survives leave-one-out/megacap — but the magnitude is ~0.002 (10× below the 0.02 bar) and the CI spans zero. Partial pass, overall H0.

2 · H1 — Price Absorption

58 perpsNO EDGE

Signal. Residual of a rolling OLS r ~ f (720-bar); A = −sign(f)·s, z-scored; strong-flow events |f| ≥ q90. Predicts reversal against the aggressor.

Period	Pooled IC(A, y)	n
Design (2025)	−0.0097	428,326
Test (2026)	−0.0103	379,248

Horizons (test IC)

1h −0.0134 · 4h −0.0103 · 24h −0.0039 · 3d −0.0006 — negative everywhere.

Matched control

absorbed − control: +5.8 bps, CI [−5.4, +17.2] — includes zero.

Incremental regression

A coefficient ≈ 0.00002, CI [−0.00024, +0.00033] — zero.

Ex-BTC/ETH IC

−0.0101. Strong-flow-only test IC −0.0161.

Criteria P1–P7: all fail. Note the IC is now negative in both windows — not merely absent.

3 · H2 — Failed Liquidity Sweeps

58 perps · 17,109 eventsNO EDGE

Signal. Wick beyond the trailing 48-bar extreme, close back inside (24-bar cooldown); aligned flow af = −d·F/activity, z-scored. Predicts stronger reversal against the sweep.

Period	Event IC(af, y)	n
Design (2025)	−0.0024	9,353
Test (2026)	+0.0062	7,756

Horizons (test IC)

1h +0.0018 · 4h +0.0062 · 24h −0.0103 · 3d +0.0090 — sign flips by horizon.

Matched control

high − low af: −0.8 bps, CI [−23.1, +23.5].

Incremental regression

af coefficient +0.00020, CI [−0.00031, +0.00081] — zero.

Ex-BTC/ETH IC

+0.0056.

Criteria Q1–Q5 fail; Q6/Q7 pass trivially (both windows ≥ one horizon), but the headline IC (0.0062) is far below the 0.02 bar with a CI spanning zero → H0.

4 · H3 — Flow-Confirmed Breakout Continuation

58 perps · 15,467 eventsNO EDGE

Signal. Close beyond the trailing 48-bar extreme (24-bar cooldown); aligned, activity-normalised flow over 4 bars, z-scored. Predicts continuation with the breakout.

Period	Event IC(flow, y)	n
Design (2025)	+0.0203	8,518
Test (2026)	−0.0206	6,949

Horizons (test IC)

1h −0.0224 · 4h −0.0206 · 24h −0.0254 · 3d −0.0134 — negative everywhere.

Matched control

high − low flow: +11.2 bps, CI [−17.1, +38.7] — includes zero.

Incremental regression

flow coefficient −0.00124, CI [−0.00250, −0.00006] — significantly negative after controls.

Ex-BTC/ETH IC

−0.0184.

Criteria R1–R7: all fail. A design IC of +0.0203 flips to −0.0206 out of sample — the cleanest example of the selection artifact. Directionally consistent with the earlier breakout_regime_study negative (cvd_20>0, lift 1.07 ns).

Synthesis

Four for four fail out-of-sample. No hypothesis clears its preregistered bar on the 2026 holdout.
The design window was the whole story. With a real 2025 design, the two “promising” signals (divergence +0.0248, sweeps +0.0495) collapse to +0.0026 and −0.0024. The edge was an artifact of a short, in-sample design slice — precisely the failure mode the earlier single-regime caveat warned about.
Where anything appears, it is often inverted: H1's test IC is negative in both windows, and H3's incremental coefficient is significantly negative. Not zero-with-noise — actively wrong-signed.
No horizon rescues them. Across 1h/4h/24h/3d, H1 and H3 are negative everywhere; H2 flips sign by horizon.

Interpretation. At 1-minute→1-hour resolution, net taker flow (CVD) did not carry incremental predictive information for forward returns in 2025–2026. The proxy nature of CVD and the two-regime sample remain the standing limitations — but this is now a temporal holdout, not an in-sample fit.

Honest caveat

2025 vs 2026 is two regimes, not many. This is a far stronger test than the single-regime runs, but it is not a universal claim. The binding constraint remains regime breadth; more years (2023–2024, a real bear/high-vol stretch) and market-factor neutralisation would strengthen any future conclusion.

Reproducibility

# runs all four studies with design=2025 / test=2026, checkpointed per study
python scripts/oos_hypotheses.py

# faster (skip parameter sweeps; keeps IC + matched + regression)
python scripts/oos_hypotheses.py --no-sweep

# force a clean re-run
python scripts/oos_hypotheses.py --force

Artifacts

Study	Checkpoint	Spec
Spot–perp divergence	`experiments/reports/oos/divergence.{md,json}`	51 cross-market · w=60 · h=4h
H1 absorption	`experiments/reports/oos/absorption.{md,json}`	58 perps · W=720 · q90 · h=4h
H2 sweeps	`experiments/reports/oos/sweeps.{md,json}`	58 perps · L=48 · cooldown 24 · h=4h
H3 breakouts	`experiments/reports/oos/breakouts.{md,json}`	58 perps · L=48 · w=4 · h=4h

Combined report: reports/oos_hypotheses_results.md. Prior (single-regime) work: reports/cvd_hypotheses_report.html. No prior reports were overwritten.

quant-lab · personal crypto research environment · research artifact, not investment advice.