P-1 — the falsification battery
Every projection on this site links here. P-1 was pre-registered before any number was computed and asked one question: does this project's information beat the market, out of sample? It does not. The gate's pre-registered kill condition — all three tests failing on both universes — was not met, so the verdict is PROCEED; but the only evidence that survives out of sample is decision-level ranking, not player projection.
What that means for anything you read here
The Marcel projections shown on player pages are descriptive. Marcel measures persistence: how much of a player's recent npxG/90 carries into next season, regressed to the positional league average and moved along an age curve. It beats “just use last season” and it beats the positional mean. That is all it claims.
It is not evidence that this project can value a player better than the transfer market can. That claim was tested and failed. No copy anywhere on this site should be read as asserting predictive skill over the market, and none is intended to.
The base rate — the denominator every headline is quoted against
34.3% of fee-bearing big-5 signings clear the frozen bar. Two thirds fail. Ian Graham's published estimate was half. A zero-skill selector drawing uniformly from the same matched cohort reproduces the base rate to within 0.005 on every cut.
| Cut | Success rate |
|---|---|
| U1 headline (fee-bearing, big-5, development loans carved out) | 34.3% |
| Minutes limb alone | 44.8% |
| Rate limb alone | 54.7% |
| Fee band Q1 → Q4 | 24.9% → 45.5% |
| Destination tier low → top | 27.3% → 37.5% |
| AM · W · CF | 24.3% · 27.0% · 27.9% |
| CM · FB · DM · CB | 31.4% · 29.7% · 36.8% · 37.4% |
| GK (minutes limb only — no goalkeeping rate in this source) | 63.0% |
| U2 (market-value banded, includes loans and frees) | 20.8% |
Test A — linear incremental information
| Universe / tier | n | ΔR² | zero-skill ΔR² (p95) | threshold | verdict |
|---|---|---|---|---|---|
| U1 / T3 | 1,399 | 0.044 | 0.020 | 0.02 | PASS |
| U1 / T1 | 993 | 0.074 | 0.020 | 0.02 | PASS |
| U2 / T3 | 3,580 | 0.066 | 0.009 | 0.02 | PASS |
| U2 / T1 | 2,414 | 0.064 | 0.011 | 0.02 | PASS |
Test A is an in-sample statistic, and the gate's own prose says out-of-sample. The walk-forward analogue — same model, same features, same folds — disagrees with it:
| Universe / tier | in-sample ΔR² | out-of-sample ΔR² | skill vs market-only, 95% CI |
|---|---|---|---|
| U1 / T3 | 0.044 | −0.018 | −0.025 [−0.080, 0.022] |
| U1 / T1 | 0.074 | +0.002 | +0.002 [−0.085, 0.080] |
| U2 / T3 | 0.066 | +0.029 | +0.036 [0.005, 0.066] |
| U2 / T1 | 0.064 | +0.020 | +0.025 [−0.020, 0.067] |
On U1 the in-sample gain does not survive. Test A's pass is not evidence of predictive skill.
Test B — nonlinear, strictly temporal out-of-sample
| Universe / tier | n out of sample | skill | 95% CI | zero-skill row | verdict |
|---|---|---|---|---|---|
| U1 / T3 | 1,190 | −0.021 | [−0.105, 0.056] | −0.040 | FAIL |
| U1 / T1 | 854 | −0.026 | [−0.134, 0.069] | −0.022 | FAIL |
| U2 / T3 | 3,140 | +0.022 | [−0.026, 0.068] | −0.026 | FAIL |
| U2 / T1 | 2,134 | +0.004 | [−0.058, 0.066] | −0.021 | FAIL |
Every confidence interval straddles zero. Not a hyperparameter artefact: three fixed GBM configurations were run and all three are reported. The pre-registered configuration is the weakest of the three on U2/T3, so the failure is not over-flexibility — and no configuration produces a CI excluding zero either.
Test C — decision-level ranking
| Selector (U2 / T3, 193 cohorts, 3,048 transfers) | mean Spearman | precision@3 | success@3 |
|---|---|---|---|
| Random | 0.000 | 0.303 | 0.191 |
| Market price (fee) | 0.115 | 0.347 | 0.238 |
| Simple-stat (highest Marcel) | 0.110 | 0.363 | 0.238 |
| Ours | 0.205 | 0.413 | 0.275 |
Ours − fee: Spearman +0.087 [0.027, 0.147], precision@3 +0.066 [0.022, 0.111]. Both CIs exclude zero, and it also beats the simple-stat rung.
The caveat that matters. That pass does not survive removing loan returns. On U2 excluding loan returns the same comparison is +0.032 on precision@3 with CI [−0.018, 0.083]. 1,274 of the U2 rows carry the loan-return signature, and a loan return is not a recruitment decision. The single Test C pass is carried in material part by rows that are not decisions. U2 was not redefined after seeing this — changing a pre-registered universe on inspection is what the gate exists to prevent — so both are reported.
Why C can pass while B fails, without either being wrong: the skill score is an MSE ratio dominated by the right tail — a handful of Salah/Haaland-scale signings — while within-cohort ranking is a different and easier task. The pre-registration wrote this shape down in advance: “A can be mediocre while C succeeds.”
Per-product verdict
| Product | Verdict |
|---|---|
| A — player projection | NOT ESTABLISHED No out-of-sample predictive skill with a CI excluding zero on any cut except the U2/T3 linear diagnostic. |
| B — translation / portability | Not tested here (P1b). |
| C — decision engine | PROMISING, NOT SECURED One clean pass, including over the simple-stat rung, with a loan-return confound. |
| D — retrospective evaluator | Not tested here; needs P0a's opportunity-set reconstruction. |
Tier consistency (Spearman 0.925 between the two outcome definitions on their 993-row overlap) says T3 is a noisier ruler, not a different one. 54 of 1,453 U1 rows have no scoreable outcome because clubs.domestic_competition_id is a current snapshot — a club now in the Premier League is labelled big-5 for a season it spent in the Championship. That is the vintage-vs-snapshot trap appearing inside the universe definition itself, and it is reported rather than quietly fixed after the fact.
Full write-up: P1-RESULTS.md in the repository. Pre-registration: experiments/p-1-premise.json, commit b29a1ef, 2026-08-30T06:37:32Z — every timestamp precedes the result it governs, and that ordering is the whole evidentiary value of the exercise.