Skip to content
Catalyst

creatine.twin

creatine.twin: external validation

Tested on abstracts PubMed returned for a query fixed before any prediction, against the module as frozen (creatine.twin version 1, SHA-256 3e58be5d…ce4770). Protocol: commit 118c7e450, task twin-creatine-validation. This is science about a module, not a product, and no figure here is a prediction for any person. validation.csv · validation.json

Result, as first run

0 observations were scorable, so the primary endpoint has no value on the first run. The screen found 86 candidate abstracts; two readers agreed on 35 values; 34 passed the verbatim check; the protocol’s scope kept 1, and that one is printed in a unit spelling the frozen unit reader does not convert (“mmol x l(-1)”), so it was excluded at simulation, as the protocol says an unrecognised unit is.

Amendment A1, 2026-11-16 (unit spelling, not a change of rule)

The protocol allows a printed spelling of a unit already converted to be learned by a dated amendment, checked against a control value, reporting both results. “mmol x l(-1)” was read as mmol/L; control: 2.17 mmol/L of creatine is 284.6 mg/L. Under A1 there is 1 observation, so the primary endpoint is one paper and one summary, 1 of 1 within twofold. It supports no claim about the model as a whole.

RunObservationsWithin 2-foldWithin 1.5-foldWithin 1.25-foldAFEAAFEIn 5th–95thIn 90% of virtual trials
First run0000not definednot defined00
Amendment A111001.821.8200

Primary metrics: Cmax, tmax, AUC0–∞, AUC0–t, t½. CL/F and V/F are scored apart: 0 observations.

The observation, with its quote

PMIDArmSummaryReported (model units)Predicted median5th–95thFoldQuote
10622230single Cr 20 g, 20.0 g creatineCmax2.17 mmol x l(-1) (284.6 mg·L⁻¹)517.8420–6131.82“peak value of 2.17 +/- 0.66 mmol x l(-1)”

Cause of the miss

The one scorable value was a single 20 g dose. The module’s rise above the basal level is linear in dose and was built from 4.4 g (its listed omission: no saturable absorption), so it over-predicts the peak by a factor of 1.82, outside its virtual subjects’ 5th–95th percentile. That is one of the protocol’s named suspects, read from one row; it is not an estimate of the model’s bias.

Why so few observations

The query returned 504 PMIDs (2026-11-16T02:38Z); 504 abstracts were fetched, 22 had no abstract, and the fixed text screen passed 86. PMID 17997838, which built the model, is not among them. Most abstracts that report creatine kinetics do so for other groups or forms than the protocol’s, as the exclusions show.

Exclusion reasonValues
analyte is phosphocreatine, not creatine6
children and adolescents6
modified formulation5
matrix is blood, not plasma or serum3
disease or patient population3
dose cannot be read in g or mg3
dosing is repeated, not a single dose3
analyte is other, not creatine2
matrix is not_stated, not plasma or serum1
analyte is creatinine, not creatine1

1 value failed the verbatim check, and 29 were reported by one reader only or disagreed on scope. The readers transcribed 53 and 50 values; 35 agreed. 1 agreed value was excluded for its unit at first run. Every list, each item with its reason, is in validation.json.

Source 1, the study records

The first run and A1 did not read this source, for want of database access, and said so. Amendment A2 reads it late: the study records for creatine recorded from 2026-11-03 to the protocol’s commit are 2 rows, both PMID 41511610 (a background sentence giving plasma half-lives of about 3 h for creatine and 2.5 h for HMB). Two independent readers classified each row’s scope from the row alone and agree: no dose, form, route or population is stated, and the second row measures HMB. Neither is in scope, so source 1 adds no observation and the result above stands: first run 0, A1 1. Nothing was re-run or tuned.

Development data, beside and never pooled

The module’s own six rows (PMID 35268011 and 34445003), seen before the protocol: 6 comparisons, 6 within twofold, 3 inside the 5th–95th percentile.

Method, as written

Two readers transcribed each candidate abstract without seeing each other or any prediction. A value is kept when both report the same PMID, analyte, metric, value, unit and arm structure (dose amount, unit, form and dosing; the free-text label is not compared), their scope fields agree, and the quote is verbatim in the abstract and contains the value. Scope is applied mechanically. 1,000 virtual subjects per arm, seed 1; a dose printed as monohydrate is converted at 131.13/149.15. Results are reported as first run and not tuned.