AI / ML · Evaluation infrastructure

Quebec French drift detection

Three clocks move Metropolitan Drift Rate: the instrument, the models, and the language. Frozen controls separate them. Ollama-served open weights, matched-pair corpus, McNemar on discordant pairs. Select any node.

Pipeline flow from trigger through a control gate to scoring

01 TRIGGER 02 PIN 03 CONTROL 04 MEASURE 05 GATE Change event digest poll · prompt commit · adapter push · cron Resolve + pin sha256 digests, corpus manifest, prompt hash Control run frozen model, corpus and prompt McNemar vs baseline Run matrix regression and longitudinal, T=0, fixed seed Gate block or alert ref lock ok rows b+c > 0 HALT instrument drift, emit environment FROZEN ARTEFACTS, CONTENT ADDRESSED Golden corpus 9 categories, matched pairs, neutral controls Metric layer MDR, CLR, QFCR, matched-pair asymmetry Run store per-pair JSONL, lockfile, trend series
control gate inputs measurement halt path

Run matrix crossing frozen and rotating model against frozen and rotating corpus

CORPUS FROZEN CORPUS ROTATING MODEL FROZEN MODEL ROTATING Control is the instrument stable? every trigger Longitudinal did the language move? monthly Regression did a model or adapter drift? on tag change Uninterpretable both clocks moving, no attribution never
gate, not a measurement attributable result excluded structurally

Data contracts and statistical thresholds

CORPUS CATEGORIES lexical/ meals/ anglicism-accept/ anglicism-reject/ syntax/ morphology/ feminisation/ typography/ neutral-control/ two anglicism directions, one false-positive detector STATISTICS McNemar, continuity corrected chi2 = (|b - c| - 1)^2 / (b + c) discordant pairs required p=0.70 n_disc = 49 p=0.65 n_disc = 87 p=0.60 n_disc = 196 exact binomial when b+c < 25 total pairs = n_disc / discordance rate GATES BLOCK control b+c > 0 neutral_drift > 0 corpus_sha mismatch digest unpinned n_disc < 49 ALERT ONLY MDR delta by category CLR / QFCR trend asymmetry drift
deterministic, blocks statistical, alerts corpus structure

Select a node

Three clocks, one metric

MDR moves when Ollama ships a build, when a model is released, and when Quebec French changes. Benchmarking cannot tell these apart. The frozen control can.