creatine.twin: external validation
Tested on abstracts PubMed returned for a query fixed before any prediction, against the module as frozen (creatine.twin version 1, SHA-256 3e58be5d…ce4770). Protocol: commit 118c7e450, task twin-creatine-validation. This is science about a module, not a product, and no figure here is a prediction for any person. validation.csv · validation.json
Result, as first run
0 observations were scorable, so the primary endpoint has no value on the first run. The screen found 86 candidate abstracts; two readers agreed on 35 values; 34 passed the verbatim check; the protocol’s scope kept 1, and that one is printed in a unit spelling the frozen unit reader does not convert (“mmol x l(-1)”), so it was excluded at simulation, as the protocol says an unrecognised unit is.
Amendment A1, 2026-11-16 (unit spelling, not a change of rule)
The protocol allows a printed spelling of a unit already converted to be learned by a dated amendment, checked against a control value, reporting both results. “mmol x l(-1)” was read as mmol/L; control: 2.17 mmol/L of creatine is 284.6 mg/L. Under A1 there is 1 observation, so the primary endpoint is one paper and one summary, 1 of 1 within twofold. It supports no claim about the model as a whole.
| Run | Observations | Within 2-fold | Within 1.5-fold | Within 1.25-fold | AFE | AAFE | In 5th–95th | In 90% of virtual trials |
|---|---|---|---|---|---|---|---|---|
| First run | 0 | 0 | 0 | 0 | not defined | not defined | 0 | 0 |
| Amendment A1 | 1 | 1 | 0 | 0 | 1.82 | 1.82 | 0 | 0 |
Primary metrics: Cmax, tmax, AUC0–∞, AUC0–t, t½. CL/F and V/F are scored apart: 0 observations.
The observation, with its quote
| PMID | Arm | Summary | Reported (model units) | Predicted median | 5th–95th | Fold | Quote |
|---|---|---|---|---|---|---|---|
| 10622230 | single Cr 20 g, 20.0 g creatine | Cmax | 2.17 mmol x l(-1) (284.6 mg·L⁻¹) | 517.8 | 420–613 | 1.82 | “peak value of 2.17 +/- 0.66 mmol x l(-1)” |
Cause of the miss
The one scorable value was a single 20 g dose. The module’s rise above the basal level is linear in dose and was built from 4.4 g (its listed omission: no saturable absorption), so it over-predicts the peak by a factor of 1.82, outside its virtual subjects’ 5th–95th percentile. That is one of the protocol’s named suspects, read from one row; it is not an estimate of the model’s bias.
Why so few observations
The query returned 504 PMIDs (2026-11-16T02:38Z); 504 abstracts were fetched, 22 had no abstract, and the fixed text screen passed 86. PMID 17997838, which built the model, is not among them. Most abstracts that report creatine kinetics do so for other groups or forms than the protocol’s, as the exclusions show.
| Exclusion reason | Values |
|---|---|
| analyte is phosphocreatine, not creatine | 6 |
| children and adolescents | 6 |
| modified formulation | 5 |
| matrix is blood, not plasma or serum | 3 |
| disease or patient population | 3 |
| dose cannot be read in g or mg | 3 |
| dosing is repeated, not a single dose | 3 |
| analyte is other, not creatine | 2 |
| matrix is not_stated, not plasma or serum | 1 |
| analyte is creatinine, not creatine | 1 |
1 value failed the verbatim check, and 29 were reported by one reader only or disagreed on scope. The readers transcribed 53 and 50 values; 35 agreed. 1 agreed value was excluded for its unit at first run. Every list, each item with its reason, is in validation.json.
Source 1, the study records
The first run and A1 did not read this source, for want of database access, and said so. Amendment A2 reads it late: the study records for creatine recorded from 2026-11-03 to the protocol’s commit are 2 rows, both PMID 41511610 (a background sentence giving plasma half-lives of about 3 h for creatine and 2.5 h for HMB). Two independent readers classified each row’s scope from the row alone and agree: no dose, form, route or population is stated, and the second row measures HMB. Neither is in scope, so source 1 adds no observation and the result above stands: first run 0, A1 1. Nothing was re-run or tuned.
Development data, beside and never pooled
The module’s own six rows (PMID 35268011 and 34445003), seen before the protocol: 6 comparisons, 6 within twofold, 3 inside the 5th–95th percentile.
Method, as written
Two readers transcribed each candidate abstract without seeing each other or any prediction. A value is kept when both report the same PMID, analyte, metric, value, unit and arm structure (dose amount, unit, form and dosing; the free-text label is not compared), their scope fields agree, and the quote is verbatim in the abstract and contains the value. Scope is applied mechanically. 1,000 virtual subjects per arm, seed 1; a dose printed as monohydrate is converted at 131.13/149.15. Results are reported as first run and not tuned.