Publish gliclass-std-base-v3-daecore-5facet-qint8-v2 (qualified ONNX export and complete attribution)

MODIFICATIONS.md

@@ -21,6 +21,11 @@ What changed:
 - **Calibration.** Per-facet temperature scaling and two frozen threshold
   tables (`recall_leaning`, `contract`) travel in `classifier-metadata.json`
   and are part of the artifact's identity.
+- **Vulkan graph preparation.** Bounded integer/boolean mask calculations
+  use exact FP32 equivalents before their original output types are restored.
+  Learned parameters, tokenizers, label prompts, calibration and thresholds
+  are unchanged. `vulkan-derivation.json` binds the source and derived graph;
+  `classifier-metadata.json` records the new graph's byte identity.
 
 Unchanged: the tokenizer vocabulary and the `<<LABEL>>` / `<<SEP>>` prompt
 convention of the upstream model.

README.md

@@ -1,511 +1,126 @@
----
-license: apache-2.0
-base_model: knowledgator/gliclass-base-v3.0
-pipeline_tag: text-classification
-tags:
-  - gliclass
-  - deberta-v3
-  - multi-label-classification
-  - fine-tuned
-  - onnx
-  - int8
----
-
-# GLiClass Base v3, five-facet memory classifier
-
-## Summary
-
-This model tags passages of working notes and documentation with five
-independent facets: `trap`, `decision`, `constraint`, `mechanism`, and
-`procedure`. It is a supervised fine-tune of
-[`knowledgator/gliclass-base-v3.0`](https://huggingface.co/knowledgator/gliclass-base-v3.0)
-trained on 59,886 labeled passages and shipped as a 452.8 MB ONNX graph inside a
-461.5 MB bundle, qualified on CPU, CUDA, and DirectML.
-
-On a frozen 1,300-row held-out panel drawn from three unseen document families,
-it reached macro average precision 0.9676 against 0.8526 for the previous
-classifier, a paired gain of 0.1150 with every facet improving. At 90% recall it
-is 0.902 precise on average against 0.719 before. The gain concentrates on the
-two hard facets: decision rose from 0.678 to 0.914 average precision and
-procedure from 0.817 to 0.958.
-
-Two caveats travel with every number below. All labels, for training and for
-evaluation, are frontier-model judgments under a frozen protocol, not human
-annotations. All evaluation panels are generated documents, chosen because they
-span far more domains and document shapes than the operator's own corpus; how
-the model behaves on unrelated real documents has not been measured.
-
-## Model details
-
-| Item | Value |
-|---|---|
-| Architecture | GLiClass single-pass label-conditioned classifier on a DeBERTa-v3 base backbone |
-| Parameters | 186.5M |
-| Input contract | 768 tokens per passage, five label prompts |
-| Output | five independent posteriors in [0, 1], temperature-scaled per facet |
-| Serialized artifact | `model.onnx`, 452,812,018 bytes, opset 17 |
-| Quantization | signed INT8 on the token-embedding matrix only; the transformer body stays FP32 |
-| Threshold tables | two frozen operating views, `contract` and `recall_leaning` |
-| Deployment state | the installed default since 2026-09-11; published 2026-09-06 |
-
-The artifact size follows from the quantization choice: the 128k-entry
-embedding table, about 98M parameters, is stored at one byte per weight while
-the 88M-parameter body stays at four. Broader quantization was measured, not
-skipped: on the predecessor fit of the same recipe, six arms were scored
-against the FP32 reference on a 5,298-row panel,
-and every one beyond the embedding table cost quality — embedding plus attention
-−0.0013, full per-channel reduced-range −0.0531, and full per-channel −0.0599
-macro average precision.
-
-## Intended use
-
-The model stamps passages inside a local-first memory system so that retrieval
-can weight evidence by kind. The full posterior map is the product; thresholds
-are advisory views for consumers that need a hard label. It is not a safety
-classifier, a factuality judge, or an authority detector, and it should not be
-used to filter what a retrieval system is allowed to return.
-
-## Task and labels
-
-Each facet answers one question about a passage:
-
-- `trap`: it names a specific mistake, hazard, pitfall, or failure mode.
-- `decision`: it makes or records a choice among alternatives.
-- `constraint`: it states a rule, policy, invariant, or restricting condition.
-- `mechanism`: it explains how something works.
-- `procedure`: it gives sequenced steps for carrying out a task.
-
-Facets overlap freely. The hardest boundary is between a passage that makes a
-decision and one that merely discusses or mentions one. Labels distinguish
-"performs" from "mentions only", and mentions-only rows count as negatives for
-both training and evaluation.
-
-**Label provenance.** Every label was produced by a language model applying the
-same one-sentence facet definitions the classifier sees. Training rows received
-two independent judgments each with a higher-effort tiebreak on semantic
-disagreement, except for 3,750 inherited rows that received one primary judgment
-from GPT-5.6 (Sol, medium reasoning) with a frozen 400-row cross-model audit by
-GPT-5.6 (Terra, high) and a Sol-high tiebreak where the two disagreed.
-Calibration and promotion panels used two
-independent Sol-high sessions per row plus a third Sol-xhigh session that saw
-only the disputed fields. Unresolved fields stay unresolved and are masked per
-facet; nothing unresolved becomes a negative.
-
-Before adopting model judges, the operator labeled about fifty passages by
-hand and compared. The hand labels were not reliable enough to serve as an
-answer key: attention drifted toward one or two facets per passage, and rows
-came out either under- or over-labeled relative to the definitions. Frontier
-judgments applied the definitions more evenly, so they became the label
-instrument. The consequence is stated plainly in every result: scores measure
-agreement with that instrument, not with human ground truth.
-
-## Training data
-
-| Lane | Rows | Origin |
-|---|---:|---|
-| Final training set | 59,886 | 54,803 generated; 4,583 real rows from the operator's self-owned corpora and 500 real rows of public-domain US Federal Register text |
-
-The 59,886 passages come from 9,376 documents, about 6.4 passages each: 8,212
-generated documents spanning 143 fictional organizations, and 1,164 real
-documents — 1,042 from the operator's two workspaces and 122 from the Federal
-Register set.
-
-Generated documents come from fictional organizations written as complete
-documents (procedures, incident reports, decision records, design notes) and
-then passed through the production parser. They were preferred over the
-real lane for both training and evaluation because they cover many more domains,
-document shapes, and organizational settings. The real lane is narrower than its
-family count suggests: 315 of its 437 source families are directory groupings inside
-two workspaces belonging to one person, and the other 122 are single Federal
-Register dockets, not 437 organizational settings; the Federal Register rows are
-also regulatory prose from a single genre. Whole-family
-splitting is therefore a strong isolation guarantee in the generated lane, where
-a family is a whole organization, and a weak one in the real lane.
-
-Every real row is redistributable — the operator's corpora by ruling,
-the Federal Register text as public-domain US government work. The Federal
-Register rows are training supervision only: no panel that calibrates, gates, or
-reports on this model contains them, or any other real row.
-
-Splits are by whole source family, never by row. The seven families reserved
-for calibration and promotion were removed from training along with 3,875
-predecessor training rows that shared them. Calibration and promotion families
-are disjoint from each other and from training by family, document lineage, and
-exact passage identity. Assignment was label-blind and checked for support only
-afterward.
-
-The final fit used weighted binary cross-entropy with a 2.0 multiplier on
-mentions-only hard negatives, learning rate 2e-5, batch 4 with 8 accumulation
-steps, four epochs, and an equal-weight average of the checkpoints from epochs
-2 through 4. Generated and real rows were weighted equally.
-
-## Evaluation data
-
-| Panel | Rows | Families | Origin | Role |
-|---|---:|---:|---|---|
-| Calibration | 1,700 | 4 | generated | fit temperatures and threshold tables; no evaluation authority |
-| Promotion | 1,300 | 3 | generated | one sealed read; the numbers reported below |
-
-Promotion-panel support per facet, with mentions-only rows counted as
-negatives. Prevalence equals the average precision a random ranker would score.
-
-| Facet | Positive | Mentions only | Negative | Unresolved | Prevalence |
-|---|---:|---:|---:|---:|---:|
-| trap | 933 | 62 | 305 | 0 | 0.718 |
-| decision | 598 | 398 | 303 | 1 | 0.460 |
-| constraint | 1,187 | 32 | 81 | 0 | 0.913 |
-| mechanism | 892 | 62 | 343 | 3 | 0.688 |
-| procedure | 485 | 277 | 531 | 7 | 0.375 |
-
-The macro random floor is 0.631. Constraint is near saturation at 0.913
-prevalence, so its average precision carries little information and the macro
-average leans on the other four facets. The panel was deliberately enriched for
-hard decision and procedure rows; it is not a natural-traffic sample.
-
-## Evaluation protocol
-
-The promotion plan was frozen on 2026-08-12 before the candidate was scored:
-one read of the panel, bootstrap seed 20260812, 10,000 draws resampling whole
-source families, 95% one-sided bounds, and zero permitted stamp flips between
-the reference scorer and the production ONNX adapter. Pass criteria were
-pre-registered on the decision facet under its frozen contract threshold:
-precision at least 0.70 with a one-sided 95% lower bound at least 0.60, and
-recall at least 0.50. Threshold tables were fitted on the calibration panel and
-bound into the prediction receipt; nothing was tuned on the promotion panel.
-
-Average precision and precision at fixed recall are the primary metrics because
-they do not depend on a threshold. Brier score and 10-bin expected calibration
-error report probability quality separately from ranking quality.
-
-## Results
-
-Promotion panel, 1,300 rows, three families, single sealed read. "Previous"
-is the classifier this model replaces.
-
-| Facet | Prevalence | AP, this model | AP, previous | P@R90, this model | P@R90, previous | P@R95, this model | P@R95, previous |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| trap | 0.718 | 0.9864 | 0.8931 | 0.960 | 0.832 | 0.935 | 0.784 |
-| decision | 0.460 | 0.9135 | 0.6783 | 0.741 | 0.504 | 0.658 | 0.484 |
-| constraint | 0.913 | 0.9986 | 0.9884 | 0.996 | 0.956 | 0.995 | 0.942 |
-| mechanism | 0.688 | 0.9817 | 0.8865 | 0.926 | 0.777 | 0.891 | 0.736 |
-| procedure | 0.375 | 0.9578 | 0.8167 | 0.886 | 0.528 | 0.822 | 0.489 |
-| macro | 0.631 | 0.9676 | 0.8526 | 0.902 | 0.719 | 0.860 | 0.687 |
-
-The macro gain is 0.1150. The one-sided 95%
-lower bound on the macro precision gain under the frozen contract thresholds is
-0.009; the decision-facet precision gain has a negative lower bound, because the
-previous classifier's decision threshold was so strict that it predicted only 26
-positives at 92% precision and 4% recall.
-
-![Per-facet precision-recall curves on the promotion panel for the promoted model, the previous classifier, and the linear baseline](figures/promotion_precision_recall.svg)
-
-**Frozen-threshold operating points on the promotion panel.** The `contract`
-table is the stricter view; the pre-registered gates apply to decision.
-
-| Facet | Threshold | Precision | Recall | Predicted positive |
-|---|---:|---:|---:|---:|
-| trap | 0.976 | 0.995 | 0.636 | 596 |
-| decision | 0.485 | 0.890 | 0.703 | 473 |
-| constraint | 0.993 | 1.000 | 0.802 | 952 |
-| mechanism | 0.900 | 0.992 | 0.655 | 591 |
-| procedure | 0.578 | 0.946 | 0.715 | 371 |
-
-The recall denominators in this table do not all match the support table above.
-Where a facet has unresolved rows, the promoted model's recall is computed over a
-slightly larger set: decision 599 against 598 positives, mechanism 895 against
-892, procedure 491 against 485. The previous classifier is scored over the
-support counts, except procedure at 487. The gaps track the per-facet unresolved
-rows, which the sealed read did not mask identically for the two models. The
-effect is conservative for the promoted model — a larger denominator lowers its
-recall — and no gate outcome changes.
-
-Decision passed all three gates: precision 0.890 against a 0.70 floor, a
-one-sided 95% lower bound of 0.872 against 0.60, and recall 0.703 against 0.50.
-The operating point encodes a product judgment: surfacing a non-decision as a
-decision is treated as worse than missing one. The `recall_leaning` table
-(decision threshold 0.193) exists for consumers with the opposite preference.
-
-**Calibration.** Per-facet temperatures fitted on the calibration panel were
-1.89 (trap), 2.40 (decision), 1.69 (constraint), 1.50 (mechanism), and 2.05
-(procedure). On the promotion panel the calibrated posteriors score macro Brier
-0.072 and macro expected calibration error 0.039, against 0.349 and 0.374 for
-the previous classifier.
-
-![Per-facet reliability diagrams of the promoted model's calibrated probabilities on the promotion panel](figures/promotion_reliability.svg)
-
-Held-family slices are diagnostic only: macro average precision was 0.962,
-0.967, and 0.974 across the three families for this model and 0.851, 0.850, and
-0.859 for the previous one.
-
-## Baselines and comparisons
-
-Ordered by how directly each comparison can be checked.
-
-**Previous classifier.** A ModernBERT-base sequence-classification head used as
-a zero-shot entailment judge: one pass per facet, five passes per passage. Its
-staged bundle declares `ModernBertForSequenceClassification` over the two
-labels `entailment` and `not_entailment`, 22 layers at hidden size 768, and a
-599,027,211-byte FP32 `model.onnx` — about 1.3 times the promoted artifact for
-five times the work per passage. Its frozen thresholds were fitted on an older
-population and artifact, which is why its contract recall collapsed to 0.001 on
-trap and 0.040 on decision on this panel; the threshold-free columns above are
-the fair comparison.
-
-**Lexical baseline.** A word-level TF-IDF (1-2 grams, 200k features) with one
-balanced logistic-regression head per facet, trained on the same 59,886 rows
-and scored on the same 1,300-row panel under the same label rules.
-
-| Facet | Floor | TF-IDF + LR | This model | Previous |
-|---|---:|---:|---:|---:|
-| trap | 0.718 | 0.970 | 0.986 | 0.893 |
-| decision | 0.460 | 0.847 | 0.914 | 0.678 |
-| constraint | 0.913 | 0.997 | 0.999 | 0.988 |
-| mechanism | 0.688 | 0.963 | 0.982 | 0.886 |
-| procedure | 0.375 | 0.914 | 0.958 | 0.817 |
-| macro AP | 0.631 | 0.938 | 0.968 | 0.853 |
-| macro P@R90 | | 0.834 | 0.902 | 0.719 |
-| macro P@R95 | | 0.786 | 0.860 | 0.687 |
-
-The linear model lands 0.029 macro average precision below this model and well
-above the previous classifier. Much of the task is lexical on generated text.
-The fine-tune earns its size at high recall on the hard facets: at 95% recall it
-is 9 points more precise on decision and 17 points more precise on procedure
-than the linear model, and those are the operating regions the memory system
-runs in.
-
-![Average precision by facet on the promotion panel against the random floor](figures/promotion_average_precision.svg)
-
-**Zero-shot frontier models.** A score-blind 300-row subsample of the promotion
-panel, 100 rows per family, scored under one shared prompt, output schema, and
-label set; each external model returned one confidence per facet. This sample
-is not independent of the promotion read, and each external configuration was
-run once, so run-to-run variance is not estimated. Subsample support: trap
-213/87, decision 137/163, constraint 271/29, mechanism 194/105 with one
-unresolved, procedure 106/194 (positive/negative).
-
-| Model | Setup | Macro AP | Decision AP | Procedure AP | Macro P@R90 |
-|---|---|---:|---:|---:|---:|
-| This model | supervised fine-tune | 0.9671 | 0.9140 | 0.9500 | 0.8872 |
-| GPT-5.4 | zero-shot, high reasoning | 0.9242 | 0.7885 | 0.9177 | 0.8016 |
-| GPT-5.5 | zero-shot, high reasoning | 0.9162 | 0.7691 | 0.9210 | 0.8064 |
-| GPT-5.5 | zero-shot, low reasoning | 0.9082 | 0.7127 | 0.9209 | 0.7924 |
-| GPT-5.4 | zero-shot, low reasoning | 0.8734 | 0.6985 | 0.8337 | 0.6705 |
-| Previous classifier | prior local model | 0.8368 | 0.5992 | 0.8387 | 0.7007 |
-| GPT-5.3-Codex-Spark | zero-shot, low reasoning | 0.8086 | 0.5387 | 0.7684 | 0.7156 |
-| Nemotron 3 Ultra 550B-A55B | zero-shot, reasoning off | 0.7356 | 0.5474 | 0.7163 | 0.6144 |
-| Gemma 4 26B-A4B IT | zero-shot, reasoning off | 0.7103 | 0.4970 | 0.6707 | 0.6144 |
-
-Paired macro average-precision differences from this model, with family-grouped
-95% intervals: GPT-5.4 high −0.0428 [−0.0523, −0.0339]; GPT-5.5 high −0.0508
-[−0.0548, −0.0366]; GPT-5.5 low −0.0588 [−0.0666, −0.0442]; GPT-5.4 low
-−0.0937 [−0.1015, −0.0827]. Every interval excludes parity. Raising reasoning
-effort improved GPT-5.4 by 0.0509 [0.0456, 0.0589] and GPT-5.5 by 0.0080
-[0.0014, 0.0204]; it narrows the gap without closing it. Three accepted runs
-of the Spark configuration spanned 0.8086 to 0.8496 — the reported 0.8086 plus
-repeats at 0.8180 and 0.8496 — a spread of 0.041 that nearly
-matches the smallest GPT gap and exceeds every one of the four interval widths,
-which is the reason single runs are flagged above.
-
-The open-weight rows are kept for completeness but carry a scoring caveat:
-their confidences were much coarser than the classifier's posteriors, and once
-tied zero scores had to be admitted at high recall their P@R90 fell to the
-sample's macro prevalence. Their deltas (−0.23 and −0.26) are partly an artifact
-of that coarseness.
-
-This comparison shows task specialization, not general capability. The
-classifier was trained on roughly 60,000 examples of how the label instrument
-applies the definitions; the external models saw the definitions once. Where a
-frontier model and this classifier disagree on a row, the panel cannot say which
-is right.
-
-## How the recipe was chosen
-
-Selection ran on generated development panels held out by whole document
-family; none of them carried promotion authority. Numbers are macro average
-precision unless stated.
-
-1. **Data first.** On a 1,000-row eight-family development panel (858 fully
-   resolved rows, random floor 0.654) with the larger GLiClass backbone, training
-   on the predecessor's data alone scored 0.929 and adding more of the operator's
-   real documents scored 0.929; adding twelve generated document families scored
-   0.968. Two loss-side branches on top of that baseline were rejected: a data
-   curriculum tied it (+0.0005, interval spanning zero) with worse calibration,
-   and a robust-loss variant lost (−0.0018, interval entirely below zero).
-2. **Targeted reserve.** Appending the independently reviewed generated reserve
-   added +0.0025 [+0.0009, +0.0039].
-3. **Context window.** Widening the input from 416 to 768 tokens added +0.0048
-   [+0.0028, +0.0068] on development and +0.0033 [+0.0011, +0.0065] on a
-   separate held-out set. A counterfactual-projector branch on minimal pairs
-   changed nothing against its own replay control (+0.0001) and was dropped; the
-   run's window also did not match its draft plan, so it could not have
-   authorized a recipe change either way.
-4. **Backbone size.** On the ModernBERT GLiClass pair, with 44,049 training
-   rows and a matched recipe on a 5,298-row eight-family panel, the large arm beat
-   the base by 0.0042 average precision and 0.0152 at 90% recall [+0.0015,
-   +0.0451] — at 95% recall the interval spans zero — but cost 2.6 times
-   the artifact bytes, 2.4 times the training time, and 1.7 times peak GPU
-   memory, while the base scored 1.8 times as many rows per second. The base was
-   selected as the product architecture.
-5. **Backbone variant and window.** On the standard GLiClass Base v3
-   (DeBERTa-v3) backbone, 768 tokens scored 0.928 against 0.914 at 512, with
-   procedure moving from 0.765 to 0.821. The backbone declares a 512-position
-   limit; the 768-token contract is a measured improvement, retained because
-   every trial confirmed it.
-6. **Final fit.** The frozen recipe was trained once on all 59,886 rows,
-   calibrated, compressed, and read once on the promotion panel.
-
-What did not help: more real documents without generated diversity, the data
-curriculum, the robust loss, the counterfactual projector, and the larger
-backbone at its cost. A distribution-balanced loss, R-Drop, SMART smoothness, a
-three-state auxiliary head, and a 512-token window were also measured and
-rejected, and the experiment log carries each with its numbers. Two further
-branches closed without a scored comparison: a frozen large-NLI recipe, stopped
-after 23.2 hours with no completed epoch, and a decision-only rationale
-auxiliary, which trained four epochs and reduced all three training losses but
-failed artifact assembly on incomplete checkpoint-average candidates, so no
-panel was mounted and no per-row scores were emitted. It is retired as
-inconclusive. Better and more diverse examples, boundary cases, and a wider
-context did the work.
-
-## Runtime
-
-The shipped artifact is the INT8 bundle. Quantization was qualified as a compression step, not
-a retrain: the FP32 export and the INT8 export were scored on the same 5,298-row eight-family
-panel and compared directly.
-
-| Item | Value |
-|---|---|
-| FP32 export | 747,319,179 bytes |
-| INT8 export | 452,812,018 bytes, 39.4% smaller |
-| Quantization | signed INT8 weights, per-channel off, reduce-range off, one operator type; 24 constant-identity nodes folded |
-| Toolchain | onnx 1.21.0, onnxruntime 1.24.4, opset 17 |
-| Macro average-precision change, INT8 minus FP32 | +0.000119, family-grouped 95% interval [-0.000038, +0.000187] |
-| Posterior absolute error, INT8 against FP32 | mean 0.00088, p95 0.00483, maximum 0.0516 |
-| Production adapter against the reference scorer | maximum posterior delta 9.06e-6 against an allowed 1e-4; zero stamp flips under both threshold tables across 1,300 rows |
-| Local CPU scoring rate | 3.74 passages per second at batch 16 over 5,298 rows, peak resident set 4.18 GB |
-
-The compression interval spans zero, so INT8 is not measurably worse than FP32 on that panel.
-That read is a diagnostic: it carries no promotion authority and set no thresholds.
-
-The scoring rate above comes from the run that produced the compression score receipt
-`5505e117…`, on this artifact, through the production CPU path, as an aggregate over a whole
-panel.
-
-Provider qualification (2026-09-06, ONNX Runtime 1.24.4, one NVIDIA RTX 3060 Ti with 8 GB)
-ran the same graph through the production adapter on three execution providers, with CUDA's
-TensorFloat-32 disabled (enabled, posteriors drifted 7.5e-4 from the CPU reference). Every
-matrix product stays on the accelerator; the only CPU-resident work under CUDA or DirectML
-is shape bookkeeping and the rank-0 scalars of the attention scale.
-
-| Provider | Posterior delta vs CPU (418 passages) | Stamp flips | Single passage p50 | Batched rate |
-|---|---|---|---|---|
-| CPU | reference | 0 | 331 ms | 1.5 passages/s |
-| CUDA | 2e-6 | 0 | 19 ms | 56 passages/s |
-| DirectML | 3e-6 | 0 | 74 ms | 33 passages/s |
-
-Batched rates use eight-row batches planned against the device's free memory: throughput is
-flat beyond four rows on both accelerators, and larger batches at the 768-token ceiling exceed
-8 GB and page silently on Windows. Device memory per row at 768 tokens is about 208 MiB on
-CUDA and 277 MiB on DirectML above a resident session of 657 MiB and 481 MiB respectively.
-
-## Evidence status
-
-| Evidence | Status |
-|---|---|
-| Promotion results, gates, and calibration | as recorded from the sealed 2026-08-12 read |
-| Artifact identity, size, and quantization settings | verified 2026-09-04 against the quantization receipt |
-| Compression equivalence | as recorded; diagnostic authority only |
-| Production-adapter parity | as recorded from the promotion read |
-| Linear baseline | re-run 2026-09-04 from the recorded inputs; byte-identical |
-| Figures | regenerated 2026-09-04 from the recorded inputs; byte-identical |
-| Previous-classifier identity | verified 2026-09-04 against the staged bundle's own config |
-| Frontier-model comparisons | as recorded; single runs on a subsample that is not independent of the promotion read |
-| Serving throughput | 3.74 passages per second, as recorded from the scoring run bound to the compression score receipt |
-| Per-passage latency percentiles | single-passage p50 per provider only; no p95 or p99 |
-
-## Limitations, ranked
-
-1. **Labels are model judgments.** All scores measure agreement with a frozen
-   frontier-model instrument. There is no human-adjudicated answer key, by the
-   operator's explicit decision after the hand-labeling trial described above.
-   Resolving evidence would be a human-adjudicated sample; none is planned.
-2. **Panels are generated text.** Absolute performance on unrelated real
-   documents is unmeasured for this artifact. Resolving evidence would be a
-   multi-domain real-document panel labeled under the same protocol.
-3. **Three families in the promotion panel.** The reported intervals capture
-   sampling noise within three similar generated families, not variation across
-   domains. More families would widen and honest-size the intervals.
-4. **No seed replicate.** Every winner was trained once. Hardware and compute
-   budget, not design, set that limit; the smallest selection margins above
-   (0.0042 between backbones, 0.0005 for the curriculum) are within a plausible
-   seed effect.
-5. **Lexical headroom is thin.** A linear model sits 0.029 macro average
-   precision below this model. The advantage is real but lives at high recall
-   on decision and procedure, and should be read that way.
-6. **Frontier comparisons are single runs** on a non-independent subsample with
-   coarse external confidences.
-7. **Decision stays the hardest facet.** 398 of the 1,300 promotion rows mention
-   a decision without making one; that boundary is where the remaining error
-   concentrates.
-8. **Serving cost is measured on one machine.** The provider table in the runtime
-   section comes from a single 8 GB NVIDIA card and a four-core CPU budget; other
-   hosts will land elsewhere, and cold-start cost is not separated from warm batches.
-9. **Second sealed campaign in its family.** An earlier fit was read on a
-   sealed panel, returned NO-GO on a per-origin recall gate, and the contract was
-   then amended so origin slices are diagnostics rather than independent vetoes.
-   That fit's calibration and promotion rows were folded into this model's own
-   training set, and this campaign therefore grants its panel promotion authority
-   while explicitly declining a claim to a pristine sealed alpha-spending draw.
-   The earlier panel carried 333 operator-workspace rows; this one is entirely
-   generated, so the per-origin recall that failed on the earlier read cannot be
-   measured on this model at all. Resolving evidence would be a promotion panel
-   drawn from unspent reserve that includes real rows.
-
-## Reproducibility
-
-- Base model: `knowledgator/gliclass-base-v3.0`, revision pinned in
-  `gliclass_final_fit_campaign.json`.
-- Training rows: 59,886; manifest and lane digests in
-  `gliclass_final_fit_campaign.json` and `history/gliclass_std_base_v3_5facet_promotion_results.json`.
-- Promotion protocol: `gliclass_std_base_v3_5facet_promotion.json` (frozen
-  2026-08-12, seed 20260812, 10,000 family-cluster draws). Three digests in the
-  results record do not reproduce from anything published or retained.
-  `promotion_plan_sha256` describes the private sealed plan, not this tracked
-  public-safe copy. `panel_sha256` and `predictions_sha256` were transcribed
-  faithfully from the sealed promotion report, but the private panel and
-  predictions files that survive today hash to different values, so the exact
-  bytes the sealed read scored are gone. The results themselves are the record.
-- Calibration and thresholds: `gliclass_std_base_v3_5facet_calibration.json`,
-  `gliclass_std_base_v3_5facet_thresholds.json`; fitted values in the promotion
-  results record.
-- Artifact: `model.onnx` SHA-256
-  `690e50920e7780db5ecd14e4b209f0d3c86199214063e8bb59d399c93861324c`,
-  452,812,018 bytes; candidate manifest
-  `ed8c1f8febe961624a698c97a3035e8ca5caa582718fd5a8f93f962c8106b942`.
-- Baselines: `history/linear_baseline_results.json`, reproduced byte-identical
-  from the promoted training rows and the frozen evaluation lane with
-  `python -m classifier.scripts.qualification.linear_baseline --train
-  <private>/classifier/campaigns/gliclass_standard_base_v3_final_fit_v2_training/train.jsonl
-  --panel <private>/source_allocations/classifier_final_repartition_v1/lanes/classifier-evaluation.jsonl`.
-  The external-model protocols and results sit under `history/`. Those
-  comparison plans pin the SHA-256 of the four scripts that produced them, and
-  two of the four — `eval/scripts/decision_grade_runner.py` and
-  `eval/scripts/isolated_codex.py` — have changed since. The plans therefore no
-  longer load and are sealed records of what was run, not re-runnable commands.
-- Figures: all three regenerate byte-identical with
-  `python eval/classifier/scripts/qualification/figures.py` against the promotion
-  run's private `predictions.json`, `panel.json`, and `linear-baseline-scores.json`.
-- Entry point: `python -m classifier.scripts.promotion.evaluation`, which
-  reproduces the plan's live bindings and verifies each against its recorded
-  digest.
-
-## License and attribution
-
-This derivative is published under the Apache License, Version 2.0, following the
-upstream model. It is a modified derivative of
-[`knowledgator/gliclass-base-v3.0`](https://huggingface.co/knowledgator/gliclass-base-v3.0)
-(Apache-2.0, revision `77a70e6cd52e602ed18184ef37d18bdd3741e3d5`), which builds on
-[`microsoft/deberta-v3-base`](https://huggingface.co/microsoft/deberta-v3-base) (MIT).
-Neither upstream author endorses this derivative. The changes — a supervised five-facet
-fine-tune, an ONNX export, INT8 quantization of the token-embedding table, and per-facet
-calibration — are listed in the `NOTICE` file that accompanies this model.
+---
+license: apache-2.0
+base_model: knowledgator/gliclass-base-v3.0
+pipeline_tag: text-classification
+language:
+  - en
+tags:
+  - gliclass
+  - deberta-v3
+  - multi-label-classification
+  - fine-tuned
+  - onnx
+  - int8
+---
+
+# Five-facet passage classifier
+
+A GLiClass Base v3 fine-tune that describes what an English passage contains: a **trap, decision, constraint, mechanism or procedure**. A passage can have several facets. The model produces all five scores in one pass and runs locally through ONNX Runtime.
+
+This revision's ONNX graph adds Vulkan compatibility. Learned weights, prompts, calibration and thresholds are unchanged.
+
+On 1,300 generated passages from three document families excluded from training, macro average precision is **0.9676** for the fine-tune, 0.9384 for a TF-IDF baseline trained on the same rows, and 0.7709 for the zero-shot upstream checkpoint.
+
+## Quick start
+
+Install `onnxruntime==1.24.4`, `tokenizers` and `numpy`. This CPU example assumes the package is in `downloaded-model` and uses its published prompts and calibration rather than new label wording:
+
+```python
+import json
+from pathlib import Path
+import numpy as np
+import onnxruntime as ort
+from tokenizers import Tokenizer
+
+root = Path("downloaded-model")
+metadata = json.load(open(f"{root}/classifier-metadata.json", encoding="utf-8"))
+facets = metadata["facet_order"]
+prep = metadata["preprocessing"]
+prefix = "".join(
+    prep["label_token"] + prep["label_prompts"][facet] for facet in facets
+) + prep["separator_token"]
+tokenizer = Tokenizer.from_file(f"{root}/tokenizer.json")
+tokenizer.enable_truncation(max_length=prep["max_length"])
+encoded = tokenizer.encode(prefix + "We chose weekly releases to reduce rollout risk.")
+session = ort.InferenceSession(
+    f"{root}/model.onnx", providers=["CPUExecutionProvider"]
+)
+logits = session.run(["logits"], {
+    "input_ids": np.array([encoded.ids], dtype=np.int64),
+    "attention_mask": np.array([encoded.attention_mask], dtype=np.int64),
+})[0][0].astype(np.float64)
+calibration = metadata["probability_calibration"]
+clip = calibration["probability_clip"]
+raw = np.clip(1 / (1 + np.exp(-np.clip(logits, -80, 80))), clip, 1 - clip)
+temperatures = np.array([calibration["temperatures"][f] for f in facets])
+probabilities = 1 / (1 + np.exp(-np.log(raw / (1 - raw)) / temperatures))
+print(dict(zip(facets, probabilities.tolist())))
+```
+
+The output is five independent calibrated scores, not a distribution that sums to one. For applications that need labels, the metadata supplies two threshold tables: `contract` favors precision and `recall_leaning` retains more candidates. Choose between them by the relative cost of missed and incorrect labels.
+
+## What the labels mean
+
+| Facet | The passage… |
+|---|---|
+| `trap` | Warns about a specific mistake, hazard or failure mode |
+| `decision` | Records a choice or commitment among alternatives |
+| `constraint` | States a rule or condition that materially restricts action |
+| `mechanism` | Explains how something is organized, connected or works |
+| `procedure` | Gives sequenced actions for carrying out a task |
+
+Mentioning a decision is different from making one; training labels treat “mentions only” as negative for that facet. Facets overlap: a procedure may also contain a constraint and warn about a trap. The scores help organize passages or inspect what a search returns. They do not establish relevance, truth or authority and should not serve as a hard retrieval filter.
+
+## Before and after fine-tuning
+
+“Upstream” is the exact `knowledgator/gliclass-base-v3.0` checkpoint used to start training, applied zero-shot with the same label definitions and **no Daecore fine-tuning**. The word-feature baseline is TF-IDF plus logistic regression, trained on the same labeled rows as the fine-tune. All three use the same 1,300-passage panel; unresolved labels are excluded per facet for every model.
+
+| Metric, macro average over five facets | Upstream GLiClass | Word-feature baseline | Daecore fine-tune |
+|---|---:|---:|---:|
+| Average precision | 0.7709 | 0.9384 | **0.9676** |
+| Precision at 90% recall | 66.11% | 83.35% | **90.21%** |
+| Precision at 95% recall | 64.51% | 78.62% | **86.02%** |
+
+Average precision summarizes how well scores rank positives across thresholds. Precision at 90% recall is the cleanest measured prediction set that still keeps at least 90% of positive labels; it comes from this evaluation's curve, not from a threshold chosen in advance.
+
+| Facet | Resolved passages | Positive prevalence | Upstream AP | Fine-tuned AP |
+|---|---:|---:|---:|---:|
+| Trap | 1,300 | 71.77% | 0.9028 | **0.9864** |
+| Decision | 1,299 | 46.04% | 0.5219 | **0.9135** |
+| Constraint | 1,300 | 91.31% | 0.9544 | **0.9986** |
+| Mechanism | 1,297 | 68.77% | 0.7692 | **0.9817** |
+| Procedure | 1,293 | 37.51% | 0.7064 | **0.9578** |
+
+![Upstream, word-feature and fine-tuned classifier results](figures/classifier-comparison.svg)
+
+As the baseline shows, much of this task can be learned from word cues; the fine-tune's clearest benefit is at high recall. Constraint is common in this panel, so its near-perfect AP says less than the decision and procedure results. The panel was deliberately enriched for difficult decision and procedure cases and is not a sample of natural traffic. The [evaluation companion](evaluation/README.md) contains the text-free labels and scores, model identities and calculation code.
+
+## Data and training choices
+
+The final fit used **59,886 passages from 9,376 documents**: 54,803 generated and 5,083 real. The real portion is 4,583 passages from self-owned workspaces and 500 from public-domain US Federal Register documents; all real rows were used for training, none for evaluation. Workspace documents may also be AI-written; real describes their source, not human authorship. Generated material was written as complete organizational documents and then parsed into passages, covering more settings and document types than the available workspaces.
+
+Models applied fixed facet definitions to produce the labels. The main label campaigns used independent judgments with conflict resolution; 3,750 inherited rows used a single primary judge with a separate audit. Splits separate whole source families: the three evaluation families and four calibration families are disjoint from training and from each other. Calibration uses 1,700 passages to set per-facet temperatures and the two threshold tables.
+
+| Training choice | Value and reason |
+|---|---|
+| Starting model | GLiClass Base v3 on DeBERTa-v3, 186.5M parameters |
+| Input length | 768 tokens including label prompts; context-length experiments favored it over 512 |
+| Objective | Weighted binary cross-entropy; mentions-only negatives receive 2× weight |
+| Optimizer schedule | Learning rate 2e-5; batch 4, accumulated over 8 steps |
+| Duration | Four fixed epochs; equal-weight average of epochs 2, 3 and 4, chosen before the final fit |
+| Serving compression | INT8 token embeddings with the transformer body in FP32 |
+
+Quantizing only the token embeddings shrinks the large vocabulary table while keeping transformer calculations in FP32; broader quantization reduced quality. The ONNX graph is **452.8 MB**, and the package is about **461.5 MB** excluding runtime libraries.
+
+## Runtime and limits
+
+On the 1,300 evaluation passages, the derived graph produced the same labels under both threshold tables on CPU, CUDA and Vulkan as the previous graph on CPU, with maximum score differences below 5.7e-6. Vulkan uses `onnxruntime==1.24.4` with the native WebGPU plugin [`onnxruntime-ep-webgpu==0.4.0`](https://pypi.org/project/onnxruntime-ep-webgpu/), registered explicitly, with `dawnBackendType=Vulkan` and no providers list at session creation. Checks used Windows x64 with an NVIDIA RTX 3060 Ti (8 GB); other GPUs and Linux are untested, even where Vulkan is available.
+
+- The model is trained and evaluated only on English text.
+- Labels are model judgments without human validation.
+- The evaluation covers generated text from three held-out families and was reused during development; performance on unrelated real documents is unmeasured.
+- Training was run once; variation across seeds is unmeasured.
+
+## License
+
+Apache-2.0. The package includes upstream attribution and a modification notice.

classifier-metadata.json

@@ -1,78 +1,78 @@
-{
-  "architecture": "gliclass-single-pass",
-  "classifier_version": "gliclass-std-base-v3-daecore-5facet-qint8-v2",
-  "facet_order": [
-    "trap",
-    "decision",
-    "constraint",
-    "mechanism",
-    "procedure"
-  ],
-  "files": {
-    "config.json": {
-      "sha256": "9bad60c87ac3e047bc838ca7e6897a6fe265180eeffce4e62aabc1e9c828d606",
-      "size": 2118
-    },
-    "model.onnx": {
-      "sha256": "690e50920e7780db5ecd14e4b209f0d3c86199214063e8bb59d399c93861324c",
-      "size": 452812018
-    },
-    "tokenizer.json": {
-      "sha256": "519648948c4c59da1af88f2cf2c8b4f84417b5c673981bc9809abf84cda1b7cc",
-      "size": 8649234
-    },
-    "tokenizer_config.json": {
-      "sha256": "9d5220b355d2cb9a7df69deccca45509ce1c9a857990fb1047c48403b0f6ddfb",
-      "size": 1692
-    }
-  },
-  "preprocessing": {
-    "label_prompts": {
-      "constraint": "This text states a rule, policy, convention, invariant, or factual condition that materially restricts action.",
-      "decision": "This text records a decision or commitment that chooses one option over alternatives.",
-      "mechanism": "This text explains how a system, process, component, or structure is organized, connected, or works.",
-      "procedure": "This text gives sequenced actions or steps for carrying out a task or operation.",
-      "trap": "This text warns about a specific mistake, hazard, pitfall, or failure mode."
-    },
-    "label_token": "<<LABEL>>",
-    "max_length": 768,
-    "prompt_first": true,
-    "schema": "daecore.gliclass-preprocessing.v1",
-    "separator_token": "<<SEP>>"
-  },
-  "probability_calibration": {
-    "method": "per-facet-temperature-scaling",
-    "probability_clip": 1e-06,
-    "temperatures": {
-      "constraint": 1.6869790276494951,
-      "decision": 2.4036524709979896,
-      "mechanism": 1.501092485229896,
-      "procedure": 2.0450110042733725,
-      "trap": 1.8898163065802225
-    }
-  },
-  "schema_version": 3,
-  "source_candidate_sha256": "ed8c1f8febe961624a698c97a3035e8ca5caa582718fd5a8f93f962c8106b942",
-  "threshold_tables": {
-    "contract": {
-      "values": {
-        "constraint": 0.9928175,
-        "decision": 0.48485,
-        "mechanism": 0.8998575,
-        "procedure": 0.5776285,
-        "trap": 0.9759885
-      },
-      "version": "gliclass-std-base-v3-daecore-5facet-qint8-v2-contract"
-    },
-    "recall_leaning": {
-      "values": {
-        "constraint": 0.9059865,
-        "decision": 0.192514,
-        "mechanism": 0.5053545,
-        "procedure": 0.201766,
-        "trap": 0.715311
-      },
-      "version": "gliclass-std-base-v3-daecore-5facet-qint8-v2-recall"
-    }
-  }
-}
+{
+  "architecture": "gliclass-single-pass",
+  "classifier_version": "gliclass-std-base-v3-daecore-5facet-qint8-v2",
+  "facet_order": [
+    "trap",
+    "decision",
+    "constraint",
+    "mechanism",
+    "procedure"
+  ],
+  "files": {
+    "config.json": {
+      "sha256": "9bad60c87ac3e047bc838ca7e6897a6fe265180eeffce4e62aabc1e9c828d606",
+      "size": 2118
+    },
+    "model.onnx": {
+      "sha256": "7e9b7e37afc06a23a89ba064e1255363cdc3140ff4b2823f5dae5865945621dc",
+      "size": 452831012
+    },
+    "tokenizer.json": {
+      "sha256": "519648948c4c59da1af88f2cf2c8b4f84417b5c673981bc9809abf84cda1b7cc",
+      "size": 8649234
+    },
+    "tokenizer_config.json": {
+      "sha256": "9d5220b355d2cb9a7df69deccca45509ce1c9a857990fb1047c48403b0f6ddfb",
+      "size": 1692
+    }
+  },
+  "preprocessing": {
+    "label_prompts": {
+      "constraint": "This text states a rule, policy, convention, invariant, or factual condition that materially restricts action.",
+      "decision": "This text records a decision or commitment that chooses one option over alternatives.",
+      "mechanism": "This text explains how a system, process, component, or structure is organized, connected, or works.",
+      "procedure": "This text gives sequenced actions or steps for carrying out a task or operation.",
+      "trap": "This text warns about a specific mistake, hazard, pitfall, or failure mode."
+    },
+    "label_token": "<<LABEL>>",
+    "max_length": 768,
+    "prompt_first": true,
+    "schema": "daecore.gliclass-preprocessing.v1",
+    "separator_token": "<<SEP>>"
+  },
+  "probability_calibration": {
+    "method": "per-facet-temperature-scaling",
+    "probability_clip": 1e-06,
+    "temperatures": {
+      "constraint": 1.6869790276494951,
+      "decision": 2.4036524709979896,
+      "mechanism": 1.501092485229896,
+      "procedure": 2.0450110042733725,
+      "trap": 1.8898163065802225
+    }
+  },
+  "schema_version": 3,
+  "source_candidate_sha256": "ed8c1f8febe961624a698c97a3035e8ca5caa582718fd5a8f93f962c8106b942",
+  "threshold_tables": {
+    "contract": {
+      "values": {
+        "constraint": 0.9928175,
+        "decision": 0.48485,
+        "mechanism": 0.8998575,
+        "procedure": 0.5776285,
+        "trap": 0.9759885
+      },
+      "version": "gliclass-std-base-v3-daecore-5facet-qint8-v2-contract"
+    },
+    "recall_leaning": {
+      "values": {
+        "constraint": 0.9059865,
+        "decision": 0.192514,
+        "mechanism": 0.5053545,
+        "procedure": 0.201766,
+        "trap": 0.715311
+      },
+      "version": "gliclass-std-base-v3-daecore-5facet-qint8-v2-recall"
+    }
+  }
+}

evaluation/README.md

@@ -0,0 +1,83 @@
+# Model evaluation records
+
+These files let readers recompute the comparisons in the Daecore model cards without model weights. Task-specific records contain anonymized row and group IDs, labels, and scores or ranked grades; none contains query or document text, private source identifiers or workspace paths. Model identities are recorded inside each file.
+
+## Recompute
+
+Python 3.11 or later is enough; no packages or network access are needed. From the directory that holds the records:
+
+```sh
+python metrics.py --directory .
+```
+
+The command summarizes every record present and stops with an error if there is none. Each entry in its output equals the matching `*-summary.json`. The classifier and Ettin packages also include `figures.py`; `python figures.py --output <directory>` redraws their card chart. Gemma uses a table. Each model package contains only its own records and summaries; the source repository holds all four record types. Fractions are stored at full precision; cards round percentages to two decimals and ranking metrics to four.
+
+The records support metric recomputation. Re-running inference would need the private text and corpus, which are not included. They come from task-specific development evaluations, not a new untouched test.
+
+## Records
+
+| Record | Inputs and denominator | Comparison |
+|---|---|---|
+| `classifier.json` | 1,300 generated passages from three held-out source families; resolved labels only, per facet | Upstream GLiClass, a trained word-feature baseline and the fine-tune |
+| `reranker.json` | 970 fixed pools of 50 passages, all 48,500 pairs judged; 873 pools have useful evidence | Upstream Ettin and the fine-tune, plus the expected result of random ordering |
+| `promotion.json` | 970 hybrid-search queries over 82,719 passage texts, with reviewed labels and declared exclusions; five public dense-retrieval panels | Previous and updated Gemma with unchanged Ettin; the public panels also include upstream Gemma |
+| `serving.json` | The same 970 queries with updated-Gemma candidate pools; reference extended by 32 grades | Ettin through CUDA FP16 and Vulkan FP32, with identical weights and score mapping |
+
+`serving-qualification.json` summarizes provider and recovery checks by receipt hash and keeps the aggregate FiQA and SciFact results for upstream and fine-tuned Ettin.
+
+“Upstream” means no Daecore fine-tuning, not an untrained network; upstream Ettin is already a trained reranker. Interim training checkpoints are not included.
+
+## How each comparison was run
+
+**Classifier.** Evaluation families were excluded from training. Upstream uses the same five label definitions and 768-token input limit as the fine-tune. Its raw logits and the fine-tune's calibrated probabilities are used only to rank within each facet; their scales are not compared. All 1,300 saved predictions matched fresh CPU inference of the released graph within 5.62e-6. The baseline uses word unigram and bigram TF-IDF with one balanced logistic-regression model per facet, fitted on the same 59,886 training passages; it sees full passage text, while the neural models apply their 768-token limit.
+
+**Ettin, fixed pools.** Candidates and grades are held fixed. Both models run in PyTorch FP16 with the same 1,153-token pair construction, Transformers 5.2.0 and Sentence Transformers 5.5.1; upstream's metadata names a newer library version, but both use the same reference runtime here. The fine-tune's saved scores were checked against fresh inference on three complete pools. This evaluates the trained models, not ONNX serving or latency. The public FiQA and SciFact results rerank fixed 50-candidate pools from upstream Gemma.
+
+**Gemma update.** Only Gemma changes; BM25, fusion and Ettin's weights stay fixed. All 970 queries are retained, and seven reviewed grade corrections apply to both models. Within each original top-k or selected prefix, 65 unresolved query–passage abstentions are excluded without backfilling: `ranked_grades` keeps their positions as `null`, with the required `excluded` flag. Precision is pooled over retained positions; nDCG reindexes them and takes its ideal ordering at the retained depth from `reference_grade_counts`. Selected depths refer to the original score-selected prefixes.
+
+| Metric, 970 queries | Previous Gemma | Updated Gemma |
+|---|---:|---:|
+| Hit@3 | 83.92% | 85.36% |
+| Hit@5 | 85.88% | 88.45% |
+| Hit@10 | 87.84% | 89.90% |
+| Hit@20 | 90.10% | 91.03% |
+| nDCG@10 | 0.6764 | 0.6611 |
+| Selected-prefix precision | 80.41% | 80.34% |
+
+The same record holds per-query dense nDCG@10 for the five public panels. The previous fine-tune was measured only on FiQA (0.4009) and SciFact (0.7679); missing baselines stay absent. These sets supplied no training examples but informed development, and recomputing their means is different from rerunning retrieval on the public corpora.
+
+**Ettin providers.** Updated-Gemma candidate pools for the same 970 queries were scored through CUDA FP16 and Vulkan FP32 with unchanged weights and score mapping. The two paths' top-20 results included 32 query–passage pairs without a grade. Of these, 21 reuse grades from the hard-contrast Ettin evaluation. The other 11 received two independent GPT-6 Sol judgments plus a resolution step and were then reviewed against the full passage text by GPT-6 Astra; no person reviewed them. No existing grade changed, and the 65 abstentions remain excluded. The added grades slightly change nDCG's ideal ordering, so the promotion record keeps its original reference.
+
+| Metric, 970 queries | CUDA FP16 | Vulkan FP32 |
+|---|---:|---:|
+| Hit@3 | 85.36% | 85.26% |
+| Hit@5 | 88.45% | 88.45% |
+| Hit@10 | 89.90% | 89.90% |
+| Hit@20 | 91.03% | 91.03% |
+| nDCG@10 | 0.6611 | 0.6615 |
+| Selected-prefix precision | 80.34% | 80.28% |
+
+Small numerical differences between the paths can reorder close scores. On 20 matched pools replayed twice, second-pass reranking took 1.56 s median and 2.11 s at the 95th percentile with Vulkan, against 1.92 s and 2.70 s with the previous package's DirectML graph. All provider measurements come from one Windows x64 machine with an NVIDIA RTX 3060 Ti (8 GB) and do not transfer to other GPUs or platforms.
+
+## Metric definitions
+
+Relevance grades are 0–3, and grades **2 and 3** count as useful. An “answerable” query has at least one useful labeled passage in the specified pool; the absence of a useful judgment does not prove that no answer exists.
+
+- **Hit@k:** fraction of queries with at least one useful passage in the first k positions.
+- **Precision@k:** useful passages among the first k. Fixed-pool tables average it over queries; the promotion and serving records pool it over retained positions. Selected-prefix precision applies the same pooling to all returned passages.
+- **Recall@k:** useful passages retrieved divided by the query's known useful passages, averaged over queries. It counts labeled passages, not every fact an answer needs.
+- **nDCG@k:** gain `2**grade - 1`, discount `1/log2(rank + 1)`, divided by the ideal ordering at the same cutoff. Grade 1 adds a small gain although it is not useful for Hit, precision or recall. Each cutoff has its own ideal, so values need not change monotonically with k.
+- **Average precision (AP):** area under the stepwise precision–recall curve, with tied scores grouped at one threshold; macro AP weights the five facets equally.
+- **Precision at 90% or 95% recall:** the best measured precision at any threshold reaching that recall, taken from the evaluation curve rather than a threshold chosen in advance.
+
+Ties keep the original candidate order. Classifier fields marked `null` are excluded identically for every model. Conditional reranker nDCG and recall use the 873 answerable pools; all-query Hit and precision use all 970. The random-order reference is exact within each pool: with N candidates and R useful passages, expected precision is R/N and Hit@k is `1 - C(N-R, k)/C(N, k)`. Random ordering already reaches 97.30% Hit@20 on the answerable reranker pools.
+
+## Limits
+
+Task-specific labels are language-model judgments without a human-adjudicated reference. Much of the source material is generated. Project-document sources are narrow and often AI-written; they do not establish generalization across users. The panels were reused during development, and each released model was trained once. Several relevant passages can repeat one fact, so passage-level scores do not measure unique-fact coverage, answer completeness or downstream agent success.
+
+## Cards
+
+- [Five-facet classifier](https://huggingface.co/Daecore/gliclass-std-base-v3-5facet-qint8-v2)
+- [EmbeddingGemma retriever](https://huggingface.co/Daecore/embeddinggemma-300m-memory-ft-v1)
+- [Ettin reranker](https://huggingface.co/Daecore/ettin-150m-memory-reranker-ft-v1)

evaluation/classifier-summary.json

@@ -0,0 +1,138 @@
+{
+  "rows": 1300,
+  "families": 3,
+  "models": {
+    "upstream": {
+      "facets": {
+        "trap": {
+          "rows": 1300,
+          "average_precision": 0.9027959606064969,
+          "precision_at_recall_90": 0.7787037037037037,
+          "precision_at_recall_95": 0.7485232067510549,
+          "prevalence": 0.7176923076923077
+        },
+        "decision": {
+          "rows": 1299,
+          "average_precision": 0.5219192438473815,
+          "precision_at_recall_90": 0.46072874493927124,
+          "precision_at_recall_95": 0.46072874493927124,
+          "prevalence": 0.4603541185527329
+        },
+        "constraint": {
+          "rows": 1300,
+          "average_precision": 0.954374886008778,
+          "precision_at_recall_90": 0.9130769230769231,
+          "precision_at_recall_95": 0.9130769230769231,
+          "prevalence": 0.9130769230769231
+        },
+        "mechanism": {
+          "rows": 1297,
+          "average_precision": 0.7691508918347779,
+          "precision_at_recall_90": 0.6880804953560371,
+          "precision_at_recall_95": 0.6880804953560371,
+          "prevalence": 0.6877409406322282
+        },
+        "procedure": {
+          "rows": 1293,
+          "average_precision": 0.7063681153993947,
+          "precision_at_recall_90": 0.4648936170212766,
+          "precision_at_recall_95": 0.4153153153153153,
+          "prevalence": 0.3750966744006187
+        }
+      },
+      "macro": {
+        "average_precision": 0.7709218195393658,
+        "precision_at_recall_90": 0.6610966968194424,
+        "precision_at_recall_95": 0.6451449370877204
+      }
+    },
+    "linear": {
+      "facets": {
+        "trap": {
+          "rows": 1300,
+          "average_precision": 0.9704508234286988,
+          "precision_at_recall_90": 0.9231613611416026,
+          "precision_at_recall_95": 0.895959595959596,
+          "prevalence": 0.7176923076923077
+        },
+        "decision": {
+          "rows": 1299,
+          "average_precision": 0.8471593175324644,
+          "precision_at_recall_90": 0.6304093567251462,
+          "precision_at_recall_95": 0.571,
+          "prevalence": 0.4603541185527329
+        },
+        "constraint": {
+          "rows": 1300,
+          "average_precision": 0.9969713734939275,
+          "precision_at_recall_90": 0.9916743755781684,
+          "precision_at_recall_95": 0.9791666666666666,
+          "prevalence": 0.9130769230769231
+        },
+        "mechanism": {
+          "rows": 1297,
+          "average_precision": 0.9633161957308005,
+          "precision_at_recall_90": 0.8865638766519823,
+          "precision_at_recall_95": 0.8330058939096268,
+          "prevalence": 0.6877409406322282
+        },
+        "procedure": {
+          "rows": 1293,
+          "average_precision": 0.914259121155341,
+          "precision_at_recall_90": 0.7356902356902357,
+          "precision_at_recall_95": 0.652050919377652,
+          "prevalence": 0.3750966744006187
+        }
+      },
+      "macro": {
+        "average_precision": 0.9384313662682464,
+        "precision_at_recall_90": 0.833499841157427,
+        "precision_at_recall_95": 0.7862366151827083
+      }
+    },
+    "finetuned": {
+      "facets": {
+        "trap": {
+          "rows": 1300,
+          "average_precision": 0.9863618253424686,
+          "precision_at_recall_90": 0.96,
+          "precision_at_recall_95": 0.934668071654373,
+          "prevalence": 0.7176923076923077
+        },
+        "decision": {
+          "rows": 1299,
+          "average_precision": 0.9135015910515543,
+          "precision_at_recall_90": 0.7414030261348006,
+          "precision_at_recall_95": 0.6581986143187067,
+          "prevalence": 0.4603541185527329
+        },
+        "constraint": {
+          "rows": 1300,
+          "average_precision": 0.9985879374608636,
+          "precision_at_recall_90": 0.9962825278810409,
+          "precision_at_recall_95": 0.9947183098591549,
+          "prevalence": 0.9130769230769231
+        },
+        "mechanism": {
+          "rows": 1297,
+          "average_precision": 0.9816644729408911,
+          "precision_at_recall_90": 0.9261822376009228,
+          "precision_at_recall_95": 0.8908709338929696,
+          "prevalence": 0.6877409406322282
+        },
+        "procedure": {
+          "rows": 1293,
+          "average_precision": 0.9578217870935172,
+          "precision_at_recall_90": 0.8868686868686869,
+          "precision_at_recall_95": 0.822380106571936,
+          "prevalence": 0.3750966744006187
+        }
+      },
+      "macro": {
+        "average_precision": 0.967587522777859,
+        "precision_at_recall_90": 0.9021472956970902,
+        "precision_at_recall_95": 0.8601672072594281
+      }
+    }
+  }
+}

evaluation/figures.py

@@ -0,0 +1,89 @@
+"""Regenerate the classifier and reranker SVGs from the public labels and scores.
+
+Run with Python 3.11+: python figures.py --output ../figures
+The repository shares its renderer from eval/lib; HF packages include that
+same renderer next to this script.
+"""
+
+from __future__ import annotations
+
+import argparse
+import importlib.util
+import json
+import sys
+from pathlib import Path
+
+import metrics
+
+HERE = Path(__file__).resolve().parent
+renderer = HERE / 'svg_figures.py'
+if not renderer.is_file():
+    renderer = HERE.parents[1] / 'lib/svg_figures.py'
+spec = importlib.util.spec_from_file_location('model_card_svg', renderer)
+svg = importlib.util.module_from_spec(spec)
+sys.modules[spec.name] = svg
+spec.loader.exec_module(svg)
+
+UPSTREAM, FIT, REFERENCE = '#0072B2', '#D55E00', '#777777'
+
+
+def classifier(data: dict) -> str:
+    summary = metrics.summarize_classifier(data)
+    facets = [*metrics.FACETS, 'macro']
+    def values(model):
+        result = summary['models'][model]
+        return [result['facets'][f]['average_precision'] for f in metrics.FACETS] + [result['macro']['average_precision']]
+    return svg.render_bars(
+        facets, [('Upstream', values('upstream'), UPSTREAM),
+                 ('TF-IDF + logistic regression', values('linear'), REFERENCE),
+                 ('Daecore fine-tune', values('finetuned'), FIT)],
+        title='Classifier: matched before and after fine-tuning',
+        subtitle='1,300 generated passages · 3 held-out families · resolved labels only',
+        ylabel='average precision', ylim=(0, 1.05), separator_before=5,
+        width=840, height=360,
+        notes=['Model-generated labels; these results do not establish accuracy on unrelated real documents.'],
+    )
+
+
+def reranker(data: dict) -> str:
+    summary = metrics.summarize_reranker(data)
+    definitions = [('hit', 'At least one useful passage'), ('precision', 'Useful passages / returned passages'), ('ndcg', 'Graded ranking quality')]
+    panels = []
+    legend = [('Upstream', UPSTREAM, None), ('Daecore fine-tune', FIT, None), ('Random pool order', REFERENCE, '5 3')]
+    for field, title in definitions:
+        series = []
+        for model, (label, color, dash) in zip(('upstream', 'finetuned', 'random'), legend, strict=True):
+            values = summary['models'][model]['answerable']
+            series.append(svg.Series(label, list(range(5)), [values[str(k)][field] for k in metrics.CUTOFFS], color, dash=dash))
+        panels.append(svg.Panel(title, series, xlabel='rank cutoff',
+                                ylabel={'hit': 'Hit@k', 'precision': 'Precision@k', 'ndcg': 'nDCG@k'}[field],
+                                xlim=(0, 4), ylim=(0, 1.02), xticks=list(enumerate(map(str, metrics.CUTOFFS)))))
+    return svg.render_grid(panels, columns=3, title='Ettin: identical candidates, different ordering',
+                           subtitle='873 answerable pools of 50 · 970 queries in total · useful = grade 2 or 3',
+                           legend=legend, panel_width=300, panel_height=270)
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--directory', type=Path, default=HERE)
+    parser.add_argument('--output', type=Path, required=True)
+    parser.add_argument('--model', choices=('classifier', 'reranker'))
+    args = parser.parse_args()
+    args.output.mkdir(parents=True, exist_ok=True)
+    definitions = {'classifier': ('classifier', 'classifier-comparison.svg'),
+                   'reranker': ('reranker', 'ettin-comparison.svg')}
+    names = [args.model] if args.model else [
+        name for name, (record, _) in definitions.items()
+        if (args.directory / f'{record}.json').is_file()
+    ]
+    if not names:
+        parser.error('No model comparison records found in the selected directory')
+    for name in names:
+        record, filename = definitions[name]
+        data = json.loads((args.directory / f'{record}.json').read_text(encoding='utf-8'))
+        (args.output / filename).write_text(globals()[name](data), encoding='utf-8', newline='\n')
+
+
+
+if __name__ == '__main__':
+    main()

evaluation/metrics.py

@@ -0,0 +1,219 @@
+"""Recompute the published model comparisons from text-free evaluation records.
+
+Python 3.11+, standard library only. Run: python metrics.py --directory .
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import math
+from pathlib import Path
+from statistics import mean
+
+CUTOFFS = (1, 3, 5, 10, 20)
+FACETS = ("trap", "decision", "constraint", "mechanism", "procedure")
+
+
+def classification_metrics(labels: list[int], scores: list[float]) -> dict[str, float]:
+    """Threshold-grouped AP and best precision at or above each recall target."""
+    if len(labels) != len(scores) or not labels or set(labels) - {0, 1}:
+        raise ValueError("Classification labels and scores must be aligned binary rows")
+    if not all(math.isfinite(s) for s in scores) or sum(labels) == 0:
+        raise ValueError("Classification scores must be finite with positive support")
+    order = sorted(range(len(labels)), key=lambda i: -scores[i])
+    positives = sum(labels)
+    true_positives = 0
+    previous_recall = 0.0
+    ap = 0.0
+    points = []
+    for rank, index in enumerate(order, 1):
+        true_positives += labels[index]
+        if rank < len(order) and scores[order[rank]] == scores[index]:
+            continue
+        precision = true_positives / rank
+        recall = true_positives / positives
+        ap += (recall - previous_recall) * precision
+        previous_recall = recall
+        points.append((precision, recall))
+    return {
+        "average_precision": ap,
+        "precision_at_recall_90": max(p for p, r in points if r >= 0.9),
+        "precision_at_recall_95": max(p for p, r in points if r >= 0.95),
+        "prevalence": positives / len(labels),
+    }
+
+
+def dcg(grades: list[int], k: int) -> float:
+    return sum((2**g - 1) / math.log2(i + 2) for i, g in enumerate(grades[:k]))
+
+
+def ranking_metrics(grades: list[int], scores: list[float], k: int) -> dict[str, float]:
+    if len(grades) != len(scores) or len(grades) < k or k < 1:
+        raise ValueError("Ranking inputs must be aligned and cover the cutoff")
+    if any(type(g) is not int or g not in range(4) for g in grades):
+        raise ValueError("Ranking requires fully judged integer grades 0 through 3")
+    if not all(math.isfinite(s) for s in scores):
+        raise ValueError("Ranking scores must be finite")
+    order = sorted(range(len(scores)), key=lambda i: (-scores[i], i))
+    ranked = [grades[i] for i in order]
+    useful = sum(g >= 2 for g in ranked[:k])
+    relevant = sum(g >= 2 for g in grades)
+    ideal = dcg(sorted(grades, reverse=True), k)
+    result = {"hit": float(useful > 0), "precision": useful / k, "useful": float(useful)}
+    if relevant:
+        result["recall"] = useful / relevant
+        result["ndcg"] = dcg(ranked, k) / ideal
+    return result
+
+
+def random_ranking_metrics(grades: list[int], k: int) -> dict[str, float]:
+    """Exact expectation under a uniform permutation of one fixed judged pool."""
+    n = len(grades)
+    if k < 1 or k > n or any(type(g) is not int or g not in range(4) for g in grades):
+        raise ValueError("Random reference requires judged grades and a valid cutoff")
+    relevant = sum(g >= 2 for g in grades)
+    misses = math.comb(n - relevant, k) if n - relevant >= k else 0
+    result = {"hit": 1 - misses / math.comb(n, k), "precision": relevant / n, "useful": k * relevant / n}
+    if relevant:
+        expected_dcg = mean(2**g - 1 for g in grades) * sum(1 / math.log2(i + 2) for i in range(k))
+        result["recall"] = k / n
+        result["ndcg"] = expected_dcg / dcg(sorted(grades, reverse=True), k)
+    return result
+
+
+def summarize_classifier(data: dict) -> dict:
+    rows = data["rows"]
+    result = {"rows": len(rows), "families": len({r["group"] for r in rows}), "models": {}}
+    for model in data["model_order"]:
+        by_facet = {}
+        for facet in FACETS:
+            resolved = [r for r in rows if r["labels"][facet] is not None]
+            if any(r["labels"][facet] not in (0, 1) for r in resolved):
+                raise ValueError("Invalid resolved classifier label")
+            by_facet[facet] = {"rows": len(resolved), **classification_metrics(
+                [r["labels"][facet] for r in resolved], [r["scores"][model][facet] for r in resolved]
+            )}
+        macro = {k: mean(v[k] for v in by_facet.values()) for k in ("average_precision", "precision_at_recall_90", "precision_at_recall_95")}
+        result["models"][model] = {"facets": by_facet, "macro": macro}
+    return result
+
+
+def summarize_reranker(data: dict) -> dict:
+    rows = data["rows"]
+    answerable = [r for r in rows if any(g >= 2 for g in r["grades"])]
+    output = {"queries": len(rows), "answerable": len(answerable), "models": {}}
+    for model in ["random", *data["model_order"]]:
+        scopes = {}
+        for scope, subset in [("answerable", answerable), ("all", rows)]:
+            cuts = {}
+            for k in CUTOFFS:
+                values = [random_ranking_metrics(r["grades"], k) if model == "random" else ranking_metrics(r["grades"], r["scores"][model], k) for r in subset]
+                fields = ("hit", "precision", "useful", "recall", "ndcg") if scope == "answerable" else ("hit", "precision", "useful")
+                cuts[str(k)] = {field: mean(v[field] for v in values) for field in fields}
+            scopes[scope] = cuts
+        output["models"][model] = scopes
+    return output
+
+
+def summarize_promotion(data: dict) -> dict:
+    """Hybrid search with reviewed exclusions inside each original prefix.
+
+    A null is permitted only for an explicitly excluded judging abstention.
+    Cut first, remove exclusions second, and never backfill from a deeper rank.
+    """
+    rows = data['rows']
+    if not rows or len({row['id'] for row in rows}) != len(rows):
+        raise ValueError('Promotion rows require unique nonempty query identities')
+    output = {'queries': len(rows), 'models': {}, 'public': {}}
+    for model in data['model_order']:
+        cutoffs = {}
+        for cutoff in (3, 5, 10, 20, 'selected'):
+            per_query = []
+            for row in rows:
+                counts = row['reference_grade_counts']
+                if set(counts) != {'0', '1', '2', '3'} or any(type(n) is not int or n < 0 for n in counts.values()):
+                    raise ValueError('Reference grade counts must cover grades zero through three')
+                ranked = row['ranked_grades'][model]
+                excluded = row['excluded'][model]
+                depth = row['selected_depth'][model] if cutoff == 'selected' else cutoff
+                if (len(ranked) != 20 or len(excluded) != 20 or type(depth) is not int
+                        or not 3 <= depth <= 20 or any(type(x) is not bool for x in excluded)):
+                    raise ValueError('Promotion rows require a bounded original top twenty')
+                if any((grade is not None if drop else type(grade) is not int or grade not in range(4))
+                       for grade, drop in zip(ranked, excluded, strict=True)):
+                    raise ValueError('Only declared abstentions may lack grades')
+                kept = [grade for grade, drop in zip(ranked[:depth], excluded[:depth], strict=True) if not drop]
+                if not kept:
+                    raise ValueError('Every scored prefix must retain judged passages')
+                ideal_grades = [g for g in (3, 2, 1, 0) for _ in range(min(counts[str(g)], len(kept)))][:len(kept)]
+                useful = sum(g >= 2 for g in kept)
+                positives = counts['2'] + counts['3']
+                ideal = dcg(ideal_grades, len(kept))
+                per_query.append({
+                    'hit': float(useful > 0), 'precision': useful / len(kept),
+                    'useful': useful, 'retained': len(kept), 'excluded': depth - len(kept),
+                    'ndcg': dcg(kept, len(kept)) / ideal if ideal else 0.0,
+                    'known_recall': useful / positives if positives else None,
+                })
+            useful = sum(row['useful'] for row in per_query)
+            retained = sum(row['retained'] for row in per_query)
+            recalls = [row['known_recall'] for row in per_query if row['known_recall'] is not None]
+            cutoffs[str(cutoff)] = {
+                'hit': mean(row['hit'] for row in per_query), 'precision': useful / retained,
+                'macro_precision': mean(row['precision'] for row in per_query),
+                'ndcg': mean(row['ndcg'] for row in per_query),
+                'known_positive_recall': mean(recalls) if recalls else None,
+                'recall_queries': len(recalls), 'useful': useful, 'retained': retained,
+                'excluded_positions': sum(row['excluded'] for row in per_query),
+                'mean_useful': useful / len(rows), 'mean_retained': retained / len(rows),
+            }
+        output['models'][model] = cutoffs
+    for dataset, panel in data['public'].items():
+        if not panel['rows'] or len({row['id'] for row in panel['rows']}) != len(panel['rows']):
+            raise ValueError('Public panel requires distinct query identities')
+        measured = panel['model_order']
+        for row in panel['rows']:
+            if set(row['ndcg@10']) != set(measured) or any(
+                isinstance(value, bool) or not isinstance(value, int | float)
+                or not math.isfinite(value) or not 0 <= value <= 1
+                for value in row['ndcg@10'].values()
+            ):
+                raise ValueError('Public nDCG values must be finite measured scores')
+        output['public'][dataset] = {
+            'queries': len(panel['rows']),
+            'ndcg@10': {model: mean(row['ndcg@10'][model] for row in panel['rows']) for model in measured},
+        }
+    return output
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--directory", type=Path, default=Path(__file__).resolve().parent)
+    parser.add_argument("--output", type=Path)
+    parser.add_argument("--model", choices=("classifier", "reranker", "promotion", "serving"), help="Recompute one comparison; default: every record in this package")
+    args = parser.parse_args()
+    summarizers = {"classifier": summarize_classifier, "reranker": summarize_reranker,
+                   "promotion": summarize_promotion, "serving": summarize_promotion}
+    names = [args.model] if args.model else [
+        name for name in summarizers if (args.directory / f"{name}.json").is_file()
+    ]
+    if not names:
+        parser.error("No evaluation records found in the selected directory")
+    result = {}
+    for name in names:
+        summarize = summarizers[name]
+        data = json.loads((args.directory / f"{name}.json").read_text(encoding="utf-8"))
+        ids = [r["id"] for r in data["rows"]]
+        if len(set(ids)) != len(ids):
+            raise ValueError(f"Repeated query/passage identity in {name}")
+        result[name] = summarize(data)
+    text = json.dumps(result, indent=2, allow_nan=False) + "\n"
+    if args.output:
+        args.output.write_text(text, encoding="utf-8")
+    else:
+        print(text, end="")
+
+
+if __name__ == "__main__":
+    main()

evaluation/serving-qualification.json

@@ -0,0 +1,134 @@
+{
+  "schema": "daecore.serving-preparation-summary.v1",
+  "date": "2026-09-28",
+  "status": "local-model-serving-checks-complete-release-transition-pending",
+  "hardware": "Windows x64, NVIDIA RTX 3060 Ti 8 GiB",
+  "unmeasured_targets": [
+    "AMD",
+    "Intel",
+    "Linux x64"
+  ],
+  "runtimes": {
+    "onnxruntime": "1.24.4",
+    "vulkan_plugin": "0.4.0",
+    "backend": "Vulkan",
+    "storage_buffer_cache": "lazyRelease"
+  },
+  "models": {
+    "gemma": {
+      "vectors": 74,
+      "providers": {
+        "cpu": {
+          "receipt_sha256": "31851791104fa95d6b7b25d35f950b25e239a3d6f8fa458addb2a44917c25cb4",
+          "maximum_vector_delta": 3.0174851417541504e-07,
+          "concurrent_delta": 0.0
+        },
+        "cuda": {
+          "receipt_sha256": "6977d0aa0ebf9c39ea0fd24078c47367d1d50e81184653225fcd10758961903e",
+          "maximum_vector_delta": 2.644956111907959e-07,
+          "concurrent_delta": 2.2351741790771484e-08
+        },
+        "vulkan": {
+          "receipt_sha256": "567fb89ea195e8b188203d2334f3eda82994ae8c7fc97f2126fac3e6f4766079",
+          "maximum_vector_delta": 5.438923835754395e-07,
+          "concurrent_delta": 0.0
+        }
+      },
+      "admission": {
+        "cases": 3841,
+        "semantic_fallbacks": 679,
+        "new_useful_exclusions": 0,
+        "additional_noise_retained": 2,
+        "receipt_sha256": "71550f606c95c1e1192dd5939488b280d5f2f2f091f99413f6b63f4408a5bc2c"
+      }
+    },
+    "classifier": {
+      "cpu": {
+        "rows": 1300,
+        "maximum_posterior_delta": 5.612167303103988e-06,
+        "changed_threshold_labels": {
+          "contract": 0,
+          "recall_leaning": 0
+        },
+        "receipt_sha256": "ea545fb0a30ff25bf3da178b60593e934e6025e789c2db67290ff5360e8669a4"
+      },
+      "cuda": {
+        "rows": 1300,
+        "maximum_posterior_delta": 5.612167303103988e-06,
+        "changed_threshold_labels": {
+          "contract": 0,
+          "recall_leaning": 0
+        },
+        "receipt_sha256": "5616891e9dfdc3b6615951901cb24aa31189d6a81ed84b0080f461b85c045065"
+      },
+      "vulkan": {
+        "rows": 1300,
+        "maximum_posterior_delta": 5.612167303103988e-06,
+        "changed_threshold_labels": {
+          "contract": 0,
+          "recall_leaning": 0
+        },
+        "receipt_sha256": "c7ec971e020d9771adf0756316f468629cc05e300d7edf9abb52612901adf6b0"
+      }
+    },
+    "ettin": {
+      "queries": 970,
+      "unjudged_top20": 0,
+      "quality_evidence": "serving.json",
+      "quality_receipt_sha256": "805a1b0acd7baaf248e182908bc23cd52171e40aa8f1281799a20d15b58c6905",
+      "full_replay_seconds_p50_p95": [
+        1.7189999999827705,
+        2.610000000044238
+      ],
+      "matched_20_pool_seconds_p50_p95": {
+        "vulkan": [
+          1.5565476999909151,
+          2.1063475799834124
+        ],
+        "predecessor_directml": [
+          1.9197023500164505,
+          2.701977299965802
+        ]
+      },
+      "pressure": "One preflight refusal after 322 queries with lazy release; resumed all remaining rows with zero further retries. An earlier default-cache run stopped after 424. No native OOM observed.",
+      "precision": "FP32 Vulkan versus FP16 CUDA; rankings not bit-exact",
+      "selector": "Unchanged CUDA mapping transferred for measurement; new package/provider binding pending"
+    }
+  },
+  "operator_runtime_changed": false,
+  "published": false,
+  "remaining": [
+    "immutable-publication-revisions",
+    "profile-and-calibration-release-bindings",
+    "isolated-package-update-and-index-rebuild-recovery",
+    "DirectML-current-route-retirement"
+  ],
+  "worker_recovery": {
+    "receipt_sha256": "46205d39521bde40d1228c837e57f88e1a941d42f6846f4a244c48b513347cc6",
+    "all_three_consumer_outputs_identical_after_owned_worker_crash": true,
+    "ettin_maximum_envelope": {
+      "tokens": 1153,
+      "max_abs_logit_delta_vs_cpu": 5.91278076171875e-05
+    }
+  },
+  "retained_reranker_public": {
+    "scope": "Retained matched public pools retrieved by upstream Gemma; upstream and production Ettin PyTorch FP16; not a new Vulkan public replay",
+    "source_sha256": "0814db560081af8376e4b1f3e3d2bcd5c4d9b8b208f7fb8b7172f113e8123ef3",
+    "panels": {
+      "fiqa": {
+        "queries": 648,
+        "ndcg@10": {
+          "upstream": 0.48603492061219766,
+          "finetuned": 0.452680922415071
+        }
+      },
+      "scifact": {
+        "queries": 300,
+        "ndcg@10": {
+          "upstream": 0.7487436478294831,
+          "finetuned": 0.7543833016238701
+        }
+      }
+    }
+  }
+}

evaluation/svg_figures.py

@@ -0,0 +1,576 @@
+"""Dependency-free SVG charts for tracked evaluation figures.
+
+The evaluation figures are generated from private per-row evidence into tracked SVG files.
+Keeping the renderer inside the repository, with no plotting dependency, means a figure can be
+regenerated by any contributor with the private inputs and compared byte for byte.
+
+Everything is emitted as presentation attributes rather than CSS. Markdown hosts sanitize
+embedded stylesheets and scripts out of SVG, so a figure that carries its styling in attributes
+renders the same in the repository, on a model-card host, and in a local viewer.
+
+Layout is measured rather than assumed: legend entries, panel titles, and reference-line labels
+are placed from an estimated text width, so a longer label reflows instead of overlapping its
+neighbour. ``text_width`` approximates a sans-serif advance table, which is enough to keep
+elements apart but is not a substitute for a real font metric.
+"""
+
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass, field
+from xml.sax.saxutils import escape
+
+# Okabe-Ito, chosen because it stays distinguishable under the common colour-vision
+# deficiencies and prints legibly in greyscale.
+PALETTE = ("#0072B2", "#D55E00", "#009E73", "#CC79A7", "#E69F00", "#56B4E9", "#000000")
+FONT_STACK = "system-ui, Segoe UI, Roboto, Helvetica, Arial, sans-serif"
+FONT = f'font-family="{FONT_STACK}"'
+
+INK = "#1a1a1a"
+MUTED = "#5c5c5c"
+AXIS = "#8a8a8a"
+GRID = "#e8e8e8"
+GRID_STRONG = "#d0d0d0"
+
+# Per-character advance as a fraction of font size, for a humanist sans at normal weight.
+_NARROW = set("iljft.,;:|!()[]{}I '")
+_WIDE = set("mwMW@%")
+_DIGIT = set("0123456789")
+
+
+def text_width(text: str, size: float, *, weight: str = "normal") -> float:
+    """Estimate rendered width in user units."""
+
+    total = 0.0
+    for character in text:
+        if character in _NARROW:
+            total += 0.30
+        elif character in _WIDE:
+            total += 0.86
+        elif character in _DIGIT:
+            total += 0.56
+        elif character.isupper():
+            total += 0.66
+        else:
+            total += 0.52
+    if weight in {"600", "700", "bold"}:
+        total *= 1.05
+    return total * size
+
+
+def _fit_lines(text: str, size: float, limit: float, *, weight: str = "normal") -> list[str]:
+    """Wrap to at most two lines, breaking on whitespace."""
+
+    if text_width(text, size, weight=weight) <= limit:
+        return [text]
+    words = text.split(" ")
+    line: list[str] = []
+    for index, word in enumerate(words):
+        candidate = " ".join([*line, word])
+        if line and text_width(candidate, size, weight=weight) > limit:
+            return [" ".join(line), " ".join(words[index:])]
+        line.append(word)
+    return [" ".join(line)]
+
+
+@dataclass
+class Series:
+    label: str
+    x: list[float]
+    y: list[float]
+    color: str = PALETTE[0]
+    dash: str | None = None
+    marker: bool = True
+    width: float = 1.9
+    marker_size: float = 2.6
+
+
+@dataclass
+class Band:
+    """A shaded interval drawn behind its series."""
+
+    x: list[float]
+    low: list[float]
+    high: list[float]
+    color: str = PALETTE[0]
+    opacity: float = 0.16
+
+
+@dataclass
+class Counts:
+    """A support strip under the plot: how many rows sit behind each x position."""
+
+    x: list[float]
+    values: list[float]
+    color: str = MUTED
+    label: str = "rows per bin"
+
+
+@dataclass
+class Panel:
+    title: str
+    series: list[Series] = field(default_factory=list)
+    bands: list[Band] = field(default_factory=list)
+    xlabel: str = ""
+    ylabel: str = ""
+    xlim: tuple[float, float] | None = None
+    ylim: tuple[float, float] | None = None
+    xticks: list[tuple[float, str]] | None = None
+    yticks: list[tuple[float, str]] | None = None
+    xscale: str = "linear"
+    hlines: list[tuple[float, str, str]] = field(default_factory=list)
+    diagonal: bool = False
+    notes: list[str] = field(default_factory=list)
+    counts: Counts | None = None
+
+
+def _fmt(value: float) -> str:
+    if abs(value) >= 1e6:
+        return f"{value:.3g}"
+    text = f"{value:.3f}".rstrip("0").rstrip(".")
+    return text or "0"
+
+
+def _auto_ticks(low: float, high: float, count: int = 5) -> list[tuple[float, str]]:
+    if high <= low:
+        high = low + 1.0
+    step = (high - low) / count
+    magnitude = 10 ** math.floor(math.log10(step)) if step > 0 else 1.0
+    for factor in (1, 2, 2.5, 5, 10):
+        if step <= factor * magnitude:
+            step = factor * magnitude
+            break
+    start = math.ceil(low / step) * step
+    ticks = []
+    value = start
+    while value <= high + 1e-9:
+        ticks.append((value, _fmt(0.0 if abs(value) < step * 1e-6 else value)))
+        value += step
+    return ticks
+
+
+def _text(
+    x: float,
+    y: float,
+    body: str,
+    *,
+    size: float,
+    fill: str = INK,
+    anchor: str = "start",
+    weight: str | None = None,
+) -> str:
+    weight_attr = f' font-weight="{weight}"' if weight else ""
+    return (
+        f'<text x="{x:.1f}" y="{y:.1f}" text-anchor="{anchor}" font-size="{size}" '
+        f'fill="{fill}"{weight_attr} {FONT}>{escape(body)}</text>'
+    )
+
+
+_TITLE_SIZE = 12.5
+
+# Everything below the plot box is stacked in fixed bands rather than placed at absolute
+# offsets, so an x-axis label, a support strip, and a note can coexist without overlapping.
+_TICK_BAND = 18.0
+_XLABEL_BAND = 16.0
+_COUNTS_BAND = 30.0
+_NOTE_BAND = 12.0
+_NOTE_LEAD = 6.0
+_FLOOR_SLACK = 8.0
+
+
+def _panel_bottom(panel: Panel) -> float:
+    bottom = _TICK_BAND + _FLOOR_SLACK
+    if panel.xlabel:
+        bottom += _XLABEL_BAND
+    if panel.counts:
+        bottom += _COUNTS_BAND
+    if panel.notes:
+        bottom += _NOTE_LEAD + _NOTE_BAND * len(panel.notes)
+    return bottom
+
+
+def _panel_svg(panel: Panel, width: float, height: float, *, title_rows: int | None = None) -> str:
+    left, right = 54.0, 14.0
+    plot_w = width - left - right
+    title_lines = _fit_lines(panel.title, _TITLE_SIZE, plot_w, weight="600")
+    top = 18.0 + 14.0 * (title_rows or len(title_lines))
+    plot_h = height - top - _panel_bottom(panel)
+
+    xs = [x for s in panel.series for x in s.x] or [0.0, 1.0]
+    ys = [y for s in panel.series for y in s.y] or [0.0, 1.0]
+    ys += [value for band in panel.bands for value in (*band.low, *band.high)]
+    ys += [level for level, _, _ in panel.hlines]
+    xlim = panel.xlim or (min(xs), max(xs))
+    ylim = panel.ylim or (min(ys), max(ys))
+    if ylim[0] == ylim[1]:
+        ylim = (ylim[0] - 0.5, ylim[1] + 0.5)
+
+    def tx(value: float) -> float:
+        if panel.xscale == "log":
+            lo, hi = math.log(xlim[0]), math.log(xlim[1])
+            return left + (math.log(value) - lo) / (hi - lo) * plot_w
+        return left + (value - xlim[0]) / (xlim[1] - xlim[0]) * plot_w
+
+    def ty(value: float) -> float:
+        return top + plot_h - (value - ylim[0]) / (ylim[1] - ylim[0]) * plot_h
+
+    parts: list[str] = []
+    for index, line in enumerate(title_lines):
+        parts.append(
+            _text(
+                left + plot_w / 2,
+                16.0 + 14.0 * index,
+                line,
+                size=_TITLE_SIZE,
+                anchor="middle",
+                weight="600",
+            )
+        )
+
+    xticks = panel.xticks or _auto_ticks(*xlim)
+    yticks = panel.yticks or _auto_ticks(*ylim)
+    for value, label in yticks:
+        if not ylim[0] - 1e-9 <= value <= ylim[1] + 1e-9:
+            continue
+        y = ty(value)
+        parts.append(
+            f'<line x1="{left}" y1="{y:.1f}" x2="{left + plot_w:.1f}" y2="{y:.1f}" '
+            f'stroke="{GRID}" stroke-width="1"/>'
+        )
+        parts.append(_text(left - 6, y + 3.4, label, size=10, fill=MUTED, anchor="end"))
+    for value, label in xticks:
+        if not xlim[0] - 1e-9 <= value <= xlim[1] + 1e-9:
+            continue
+        x = tx(value)
+        parts.append(
+            f'<line x1="{x:.1f}" y1="{top}" x2="{x:.1f}" y2="{top + plot_h:.1f}" '
+            f'stroke="{GRID}" stroke-width="1"/>'
+        )
+        parts.append(_text(x, top + plot_h + 13, label, size=10, fill=MUTED, anchor="middle"))
+
+    if panel.diagonal:
+        parts.append(
+            f'<line x1="{tx(xlim[0]):.1f}" y1="{ty(ylim[0]):.1f}" '
+            f'x2="{tx(xlim[1]):.1f}" y2="{ty(ylim[1]):.1f}" stroke="{AXIS}" '
+            f'stroke-width="1.1" stroke-dasharray="4 3"/>'
+        )
+
+    for band in panel.bands:
+        forward = " ".join(
+            f"{tx(x):.1f},{ty(y):.1f}" for x, y in zip(band.x, band.high, strict=True)
+        )
+        backward = " ".join(
+            f"{tx(x):.1f},{ty(y):.1f}"
+            for x, y in zip(reversed(band.x), reversed(band.low), strict=True)
+        )
+        parts.append(
+            f'<polygon points="{forward} {backward}" fill="{band.color}" '
+            f'fill-opacity="{band.opacity}" stroke="none"/>'
+        )
+
+    # A reference line carries its label at the left margin over a solid backing box, so the
+    # label never lands on the data it is a reference for.
+    for value, label, color in panel.hlines:
+        y = ty(value)
+        parts.append(
+            f'<line x1="{left}" y1="{y:.1f}" x2="{left + plot_w:.1f}" y2="{y:.1f}" '
+            f'stroke="{color}" stroke-width="1.2" stroke-dasharray="5 3"/>'
+        )
+        if not label:
+            continue
+        label_w = text_width(label, 9.5) + 8.0
+        parts.append(
+            f'<rect x="{left + 3:.1f}" y="{y - 12:.1f}" width="{label_w:.1f}" height="11.5" '
+            f'fill="white" fill-opacity="0.9" stroke="none"/>'
+        )
+        parts.append(_text(left + 7, y - 3.5, label, size=9.5, fill=color))
+
+    for series in panel.series:
+        points = " ".join(
+            f"{tx(x):.1f},{ty(y):.1f}" for x, y in zip(series.x, series.y, strict=True)
+        )
+        dash = f' stroke-dasharray="{series.dash}"' if series.dash else ""
+        parts.append(
+            f'<polyline points="{points}" fill="none" stroke="{series.color}" '
+            f'stroke-width="{series.width}" stroke-linejoin="round" '
+            f'stroke-linecap="round"{dash}/>'
+        )
+        if series.marker:
+            for x, y in zip(series.x, series.y, strict=True):
+                parts.append(
+                    f'<circle cx="{tx(x):.1f}" cy="{ty(y):.1f}" r="{series.marker_size}" '
+                    f'fill="{series.color}"/>'
+                )
+
+    parts.append(
+        f'<rect x="{left}" y="{top}" width="{plot_w:.1f}" height="{plot_h:.1f}" fill="none" '
+        f'stroke="{AXIS}" stroke-width="1"/>'
+    )
+
+    cursor = top + plot_h + _TICK_BAND
+    if panel.xlabel:
+        parts.append(
+            _text(
+                left + plot_w / 2,
+                cursor + 11,
+                panel.xlabel,
+                size=10.5,
+                fill=MUTED,
+                anchor="middle",
+            )
+        )
+        cursor += _XLABEL_BAND
+    if panel.counts:
+        counts = panel.counts
+        if not counts.values:
+            raise ValueError("a support strip needs at least one count")
+        strip_top = cursor + 2.0
+        strip_h = 15.0
+        peak = max(counts.values) or 1.0
+        slot = plot_w / max(len(counts.x), 1) * 0.7
+        for x, value in zip(counts.x, counts.values, strict=True):
+            bar_h = (value / peak) * strip_h
+            parts.append(
+                f'<rect x="{tx(x) - slot / 2:.1f}" y="{strip_top + strip_h - bar_h:.1f}" '
+                f'width="{slot:.1f}" height="{bar_h:.1f}" fill="{counts.color}" '
+                f'fill-opacity="0.5"/>'
+            )
+        parts.append(
+            f'<line x1="{left}" y1="{strip_top + strip_h:.1f}" x2="{left + plot_w:.1f}" '
+            f'y2="{strip_top + strip_h:.1f}" stroke="{GRID_STRONG}" stroke-width="1"/>'
+        )
+        parts.append(_text(left, strip_top + strip_h + 9, counts.label, size=9, fill=MUTED))
+        parts.append(
+            _text(
+                left + plot_w,
+                strip_top + strip_h + 9,
+                f"tallest {int(peak):,}",
+                size=9,
+                fill=MUTED,
+                anchor="end",
+            )
+        )
+        cursor += _COUNTS_BAND
+    for index, note in enumerate(panel.notes):
+        parts.append(
+            _text(left, cursor + _NOTE_LEAD + 9 + _NOTE_BAND * index, note, size=9.5, fill=MUTED)
+        )
+    if panel.ylabel:
+        parts.append(
+            f'<text transform="translate(13,{top + plot_h / 2:.1f}) rotate(-90)" '
+            f'text-anchor="middle" font-size="10.5" fill="{MUTED}" {FONT}>'
+            f"{escape(panel.ylabel)}</text>"
+        )
+    return "\n".join(parts)
+
+
+def _legend_svg(
+    entries: list[tuple[str, str, str | None]],
+    *,
+    x: float,
+    y: float,
+    max_width: float,
+) -> tuple[str, float]:
+    """Flow legend entries across as many rows as their measured widths need."""
+
+    swatch, gap, pad = 22.0, 7.0, 22.0
+    parts: list[str] = []
+    cursor_x, cursor_y, rows = x, y, 1
+    for label, color, dash in entries:
+        entry_w = swatch + gap + text_width(label, 11) + pad
+        if cursor_x > x and cursor_x + entry_w - pad > x + max_width:
+            cursor_x, cursor_y, rows = x, cursor_y + 16.0, rows + 1
+        dash_attr = f' stroke-dasharray="{dash}"' if dash else ""
+        parts.append(
+            f'<line x1="{cursor_x:.1f}" y1="{cursor_y:.1f}" x2="{cursor_x + swatch:.1f}" '
+            f'y2="{cursor_y:.1f}" stroke="{color}" stroke-width="2.4" '
+            f'stroke-linecap="round"{dash_attr}/>'
+        )
+        parts.append(_text(cursor_x + swatch + gap, cursor_y + 3.8, label, size=11))
+        cursor_x += entry_w
+    return "\n".join(parts), 16.0 * rows
+
+
+def _dedupe(entries: list[tuple[str, str, str | None]]) -> list[tuple[str, str, str | None]]:
+    seen: set[tuple[str, str, str | None]] = set()
+    unique = []
+    for entry in entries:
+        if entry not in seen:
+            seen.add(entry)
+            unique.append(entry)
+    return unique
+
+
+def _open_svg(width: float, height: float, title: str, description: str) -> list[str]:
+    return [
+        f'<svg xmlns="http://www.w3.org/2000/svg" width="{width:.0f}" height="{height:.0f}" '
+        f'viewBox="0 0 {width:.0f} {height:.0f}" role="img" aria-label="{escape(title)}">',
+        f"<title>{escape(title)}</title>",
+        f"<desc>{escape(description)}</desc>",
+        '<rect width="100%" height="100%" fill="white"/>',
+    ]
+
+
+def render_grid(
+    panels: list[Panel],
+    *,
+    columns: int,
+    title: str,
+    subtitle: str = "",
+    panel_width: float = 352.0,
+    panel_height: float = 256.0,
+    legend: list[tuple[str, str, str | None]] | None = None,
+    description: str = "",
+) -> str:
+    """Render panels on a grid with one shared title and a flowed legend.
+
+    A final short row is centred, so a five-panel figure on three columns has no empty cell.
+    """
+
+    if not panels:
+        raise ValueError("render_grid needs at least one panel")
+    if columns < 1:
+        raise ValueError("render_grid needs at least one column")
+    legend = _dedupe(legend or [])
+    rows = math.ceil(len(panels) / columns)
+    width = columns * panel_width
+    header = 26.0 + (16.0 if subtitle else 0.0)
+    legend_svg, legend_h = "", 0.0
+    if legend:
+        legend_svg, legend_h = _legend_svg(legend, x=18.0, y=header + 10.0, max_width=width - 36.0)
+        legend_h += 8.0
+    height = header + legend_h + rows * panel_height
+    parts = _open_svg(width, height, title, description or title)
+    parts.append(_text(width / 2, 19, title, size=14.5, anchor="middle", weight="700"))
+    if subtitle:
+        parts.append(_text(width / 2, 34, subtitle, size=11, fill=MUTED, anchor="middle"))
+    if legend_svg:
+        parts.append(legend_svg)
+    # One title row count for the whole grid, so a panel whose title wraps does not push its
+    # plot box below its neighbours'.
+    title_rows = max(
+        len(_fit_lines(panel.title, _TITLE_SIZE, panel_width - 68.0, weight="600"))
+        for panel in panels
+    )
+    for index, panel in enumerate(panels):
+        row, column = divmod(index, columns)
+        in_row = min(columns, len(panels) - row * columns)
+        offset = (columns - in_row) * panel_width / 2.0
+        px = offset + column * panel_width
+        py = header + legend_h + row * panel_height
+        parts.append(f'<g transform="translate({px:.1f},{py:.1f})">')
+        parts.append(_panel_svg(panel, panel_width, panel_height, title_rows=title_rows))
+        parts.append("</g>")
+    parts.append("</svg>")
+    return "\n".join(parts) + "\n"
+
+
+def render_bars(
+    groups: list[str],
+    series: list[tuple[str, list[float], str]],
+    *,
+    title: str,
+    ylabel: str,
+    subtitle: str = "",
+    ylim: tuple[float, float] = (0.0, 1.0),
+    reference: list[tuple[str, list[float], str]] | None = None,
+    notes: list[str] | None = None,
+    separator_before: int | None = None,
+    width: float = 780.0,
+    height: float = 350.0,
+    description: str = "",
+) -> str:
+    """Render grouped bars with per-group dashed reference levels.
+
+    Bars keep a zero baseline. Value labels are drawn only where a bar is wide enough to hold
+    one, because a crowded label is worse than none.
+    """
+
+    if not groups or not series:
+        raise ValueError("render_bars needs at least one group and one series")
+    if any(len(values) != len(groups) for _, values, _ in series):
+        raise ValueError("every bar series must carry one value per group")
+    notes = notes or []
+    left, right, top = 54.0, 16.0, 26.0 + (15.0 if subtitle else 0.0)
+    legend_entries = [(label, color, None) for label, _, color in series]
+    legend_entries += [(label, color, "5 3") for label, _, color in reference or []]
+    legend_svg, legend_h = _legend_svg(
+        _dedupe(legend_entries), x=18.0, y=top + 12.0, max_width=width - 36.0
+    )
+    top += legend_h + 10.0
+    bottom = 46.0 + 12.0 * len(notes)
+    plot_w, plot_h = width - left - right, height - top - bottom
+    group_w = plot_w / len(groups)
+    bar_w = group_w * 0.74 / len(series)
+
+    def ty(value: float) -> float:
+        return top + plot_h - (value - ylim[0]) / (ylim[1] - ylim[0]) * plot_h
+
+    parts = _open_svg(width, height, title, description or title)
+    parts.append(_text(width / 2, 19, title, size=14.5, anchor="middle", weight="700"))
+    if subtitle:
+        parts.append(_text(width / 2, 34, subtitle, size=11, fill=MUTED, anchor="middle"))
+    parts.append(legend_svg)
+    for value, label in _auto_ticks(*ylim):
+        y = ty(value)
+        parts.append(
+            f'<line x1="{left}" y1="{y:.1f}" x2="{left + plot_w:.1f}" y2="{y:.1f}" '
+            f'stroke="{GRID}" stroke-width="1"/>'
+        )
+        parts.append(_text(left - 6, y + 3.4, label, size=10, fill=MUTED, anchor="end"))
+
+    label_fits = bar_w - 2 >= text_width("0.000", 8.5) + 2
+    for g_index, group in enumerate(groups):
+        gx = left + g_index * group_w + group_w * 0.13
+        for s_index, (_, values, color) in enumerate(series):
+            value = values[g_index]
+            x = gx + s_index * bar_w
+            parts.append(
+                f'<rect x="{x:.1f}" y="{ty(value):.1f}" width="{bar_w - 2:.1f}" '
+                f'height="{ty(ylim[0]) - ty(value):.1f}" fill="{color}"/>'
+            )
+            if label_fits:
+                parts.append(
+                    _text(
+                        x + (bar_w - 2) / 2,
+                        ty(value) - 4,
+                        f"{value:.3f}",
+                        size=8.5,
+                        anchor="middle",
+                    )
+                )
+        for _, values, color in reference or []:
+            y = ty(values[g_index])
+            parts.append(
+                f'<line x1="{gx - 3:.1f}" y1="{y:.1f}" '
+                f'x2="{gx + group_w * 0.74 + 3:.1f}" y2="{y:.1f}" stroke="{color}" '
+                f'stroke-width="1.6" stroke-dasharray="5 3"/>'
+            )
+        parts.append(
+            _text(
+                left + g_index * group_w + group_w / 2,
+                top + plot_h + 16,
+                group,
+                size=11,
+                anchor="middle",
+            )
+        )
+    if separator_before is not None and 0 < separator_before < len(groups):
+        x = left + separator_before * group_w
+        parts.append(
+            f'<line x1="{x:.1f}" y1="{top}" x2="{x:.1f}" y2="{top + plot_h + 6:.1f}" '
+            f'stroke="{GRID_STRONG}" stroke-width="1.4"/>'
+        )
+    parts.append(
+        f'<rect x="{left}" y="{top}" width="{plot_w:.1f}" height="{plot_h:.1f}" fill="none" '
+        f'stroke="{AXIS}" stroke-width="1"/>'
+    )
+    parts.append(
+        f'<text transform="translate(13,{top + plot_h / 2:.1f}) rotate(-90)" '
+        f'text-anchor="middle" font-size="10.5" fill="{MUTED}" {FONT}>'
+        f"{escape(ylabel)}</text>"
+    )
+    for index, note in enumerate(notes):
+        parts.append(_text(left, top + plot_h + 34 + 12 * index, note, size=9.5, fill=MUTED))
+    parts.append("</svg>")
+    return "\n".join(parts) + "\n"

model.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:690e50920e7780db5ecd14e4b209f0d3c86199214063e8bb59d399c93861324c
-size 452812018
+oid sha256:7e9b7e37afc06a23a89ba064e1255363cdc3140ff4b2823f5dae5865945621dc
+size 452831012

publication-manifest.json

@@ -1,6 +1,6 @@
 {
   "schema": "daecore.classifier-publication-manifest",
-  "tool_sha256": "ec0f21b2c945a07b0a1f7be9895c91f30a985fb981a5fb83f409604b33879af4",
+  "tool_sha256": "5ac2ef9465fe9a3454bfd2e61c7aa651b08fbdfef9473fb5b3455585ba560a7b",
   "public_repo": "Daecore/gliclass-std-base-v3-5facet-qint8-v2",
   "classifier_version": "gliclass-std-base-v3-daecore-5facet-qint8-v2",
   "upstream": {
@@ -8,38 +8,39 @@
     "revision": "77a70e6cd52e602ed18184ef37d18bdd3741e3d5"
   },
   "nominee_receipt_sha256": "d463fc167b2a26f8f5b56e03c18e4c604b231eb25b5093064dd8a9bd7ffa6373",
-  "staged_at": "2026-09-18T18:58:21+00:00",
+  "derivation_preparation_sha256": "bfd902e06b7a984348283dc279f5e46d76398ff59633a9b5b426f33d872a67d3",
+  "source_nominee_binding": "unchanged learned model and calibration; graph bytes separately derived",
+  "staged_at": "2026-09-29T03:04:21+00:00",
   "files": {
+    "classifier-metadata.json": {
+      "sha256": "736d8b615fb8c4bd7cc36d1c2e0e86bbe096d63dc373d047ca75af9cc6119a11",
+      "size": 2674,
+      "binding": "pinned graph derivation after original nominee verification"
+    },
+    "config.json": {
+      "sha256": "9bad60c87ac3e047bc838ca7e6897a6fe265180eeffce4e62aabc1e9c828d606",
+      "size": 2118,
+      "binding": "pinned graph derivation after original nominee verification"
+    },
     "model.onnx": {
-      "sha256": "690e50920e7780db5ecd14e4b209f0d3c86199214063e8bb59d399c93861324c",
-      "size": 452812018,
-      "binding": "candidate manifest (file hash)"
+      "sha256": "7e9b7e37afc06a23a89ba064e1255363cdc3140ff4b2823f5dae5865945621dc",
+      "size": 452831012,
+      "binding": "pinned graph derivation after original nominee verification"
     },
     "tokenizer.json": {
       "sha256": "519648948c4c59da1af88f2cf2c8b4f84417b5c673981bc9809abf84cda1b7cc",
       "size": 8649234,
-      "binding": "candidate manifest (file hash)"
-    },
-    "config.json": {
-      "sha256": "9bad60c87ac3e047bc838ca7e6897a6fe265180eeffce4e62aabc1e9c828d606",
-      "size": 2118,
-      "binding": "candidate manifest (file hash)"
+      "binding": "pinned graph derivation after original nominee verification"
     },
     "tokenizer_config.json": {
       "sha256": "9d5220b355d2cb9a7df69deccca45509ce1c9a857990fb1047c48403b0f6ddfb",
       "size": 1692,
-      "binding": "candidate manifest (file hash)"
-    },
-    "classifier-metadata.json": {
-      "sha256": "00d8c3b36be47f96ddd113bd6ddbf525c3599de27064c68ef7a1aacf4c0408e5",
-      "size": 2596,
-      "canonical_sha256": "1918aa0c392e6582a7028a4a2482166714080958602f6234f145dd294fb01996",
-      "binding": "nominee receipt runtime_metadata_sha256 (canonical JSON hash)"
+      "binding": "pinned graph derivation after original nominee verification"
     },
-    "README.md": {
-      "sha256": "48cc3f3e7659463c7bd6031d923a96f3822e7ebd62a2d5f660d779d95612f76c",
-      "size": 29930,
-      "binding": "packaging record"
+    "vulkan-derivation.json": {
+      "sha256": "472be46b2c127c4bf38dcf1db3d5dfe65672a6ea66164677d91a2cbb2d05ee70",
+      "size": 5782,
+      "binding": "pinned graph derivation after original nominee verification"
     },
     "LICENSE": {
       "sha256": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4",
@@ -52,9 +53,55 @@
       "binding": "packaging record"
     },
     "MODIFICATIONS.md": {
-      "sha256": "fca339612182863205137d75b60cb23dc921090c0e91a4541a382c3889cd7e95",
-      "size": 1399,
+      "sha256": "429cfef847f3816fe9cc4715d2431f344d5b34376e03b214eae52c1626021f39",
+      "size": 1773,
+      "binding": "packaging record"
+    },
+    "evaluation/README.md": {
+      "sha256": "3215e6b0e302fdecd86ef1457e46e7d033fa18fb1ebcbd0dcb47c2b8abe7c174",
+      "size": 8825,
       "binding": "packaging record"
+    },
+    "evaluation/metrics.py": {
+      "sha256": "7c25f29a03e0f61d5f0e59f7781269489f80512ae6862920f91a2d333c058bc4",
+      "size": 11135,
+      "binding": "packaging record"
+    },
+    "evaluation/classifier.json": {
+      "sha256": "5498bb3f5f15f587bc93fc77c9a56308e6ccbc85976bd8e38530b643614628bc",
+      "size": 721988,
+      "binding": "packaging record"
+    },
+    "evaluation/classifier-summary.json": {
+      "sha256": "e89f48dff2e29f9d8f4c27b221b88c2d380f725a682d6792116bb0d3dff7bd8e",
+      "size": 4799,
+      "binding": "packaging record"
+    },
+    "evaluation/serving-qualification.json": {
+      "sha256": "bf7deda3f25306d15454798ad4c8e785c25fba970fc774efbe557225fc5d42e0",
+      "size": 4667,
+      "binding": "packaging record"
+    },
+    "evaluation/figures.py": {
+      "sha256": "9944765ee42148f2bb135f428ee1dacb822af0ca5c77be9f82b507eb8da236d4",
+      "size": 4097,
+      "binding": "packaging record"
+    },
+    "evaluation/svg_figures.py": {
+      "sha256": "c68b7e26e072e3875f111b592f8031322261272fcf285ae2a3edbe378e70740d",
+      "size": 21536,
+      "binding": "packaging record"
+    },
+    "figures/classifier-comparison.svg": {
+      "sha256": "a465b6ab22d60b50543c29dd7f16fe440b948eb0d12c38df7facc34fecfee509",
+      "size": 8642,
+      "binding": "packaging record"
+    },
+    "README.md": {
+      "sha256": "eec5fe5f0d257201d4d517f6a3edb97c5a6cfbe352cc8835c2be42c3baf0ad14",
+      "size": 8321,
+      "binding": "model card with upload-relative links",
+      "source_sha256": "f62d093e04a97c57080285d08d64c66906b500ee2124d86013b2dde0c120b592"
     }
   }
 }

vulkan-derivation.json

@@ -0,0 +1,227 @@
+{
+  "artifacts": [
+    {
+      "bytes": 452831012,
+      "path": "model.onnx",
+      "sha256": "7e9b7e37afc06a23a89ba064e1255363cdc3140ff4b2823f5dae5865945621dc"
+    }
+  ],
+  "changes": [
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/Squeeze",
+      "operation": "Squeeze"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/CumSum",
+      "operation": "CumSum"
+    },
+    {
+      "dtype": 7,
+      "name": "Sign_1892",
+      "operation": "Sign"
+    },
+    {
+      "dtype": 7,
+      "name": "Abs_1898",
+      "operation": "Abs"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/Slice_1",
+      "operation": "Slice"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.0/attention/self/Squeeze_3",
+      "operation": "Squeeze"
+    },
+    {
+      "dtype": 7,
+      "name": "Identity_2252",
+      "operation": "Identity"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.0/attention/self/Neg",
+      "operation": "Neg"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.0/attention/self/Squeeze_5",
+      "operation": "Squeeze"
+    },
+    {
+      "dtype": 7,
+      "name": "Identity_2696",
+      "operation": "Identity"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.1/attention/self/Neg",
+      "operation": "Neg"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.1/attention/self/Squeeze_3",
+      "operation": "Squeeze"
+    },
+    {
+      "dtype": 7,
+      "name": "Identity_3138",
+      "operation": "Identity"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.2/attention/self/Neg",
+      "operation": "Neg"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.2/attention/self/Squeeze_3",
+      "operation": "Squeeze"
+    },
+    {
+      "dtype": 7,
+      "name": "Identity_3580",
+      "operation": "Identity"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.3/attention/self/Neg",
+      "operation": "Neg"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.3/attention/self/Squeeze_3",
+      "operation": "Squeeze"
+    },
+    {
+      "dtype": 7,
+      "name": "Identity_4022",
+      "operation": "Identity"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.4/attention/self/Neg",
+      "operation": "Neg"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.4/attention/self/Squeeze_3",
+      "operation": "Squeeze"
+    },
+    {
+      "dtype": 7,
+      "name": "Identity_4464",
+      "operation": "Identity"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.5/attention/self/Neg",
+      "operation": "Neg"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.5/attention/self/Squeeze_3",
+      "operation": "Squeeze"
+    },
+    {
+      "dtype": 7,
+      "name": "Identity_4906",
+      "operation": "Identity"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.6/attention/self/Neg",
+      "operation": "Neg"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.6/attention/self/Squeeze_3",
+      "operation": "Squeeze"
+    },
+    {
+      "dtype": 7,
+      "name": "Identity_5348",
+      "operation": "Identity"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.7/attention/self/Neg",
+      "operation": "Neg"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.7/attention/self/Squeeze_3",
+      "operation": "Squeeze"
+    },
+    {
+      "dtype": 7,
+      "name": "Identity_5790",
+      "operation": "Identity"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.8/attention/self/Neg",
+      "operation": "Neg"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.8/attention/self/Squeeze_3",
+      "operation": "Squeeze"
+    },
+    {
+      "dtype": 7,
+      "name": "Identity_6232",
+      "operation": "Identity"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.9/attention/self/Neg",
+      "operation": "Neg"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.9/attention/self/Squeeze_3",
+      "operation": "Squeeze"
+    },
+    {
+      "dtype": 7,
+      "name": "Identity_6674",
+      "operation": "Identity"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.10/attention/self/Neg",
+      "operation": "Neg"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.10/attention/self/Squeeze_3",
+      "operation": "Squeeze"
+    },
+    {
+      "dtype": 7,
+      "name": "Identity_7116",
+      "operation": "Identity"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.11/attention/self/Neg",
+      "operation": "Neg"
+    },
+    {
+      "dtype": 7,
+      "name": "/inner/model/encoder_model/encoder/layer.11/attention/self/Squeeze_3",
+      "operation": "Squeeze"
+    }
+  ],
+  "max_sequence": 768,
+  "role": "gliclass",
+  "schema": "daecore.vulkan-mask-derivation.v1",
+  "source_graph_sha256": "690e50920e7780db5ecd14e4b209f0d3c86199214063e8bb59d399c93861324c",
+  "source_weights_sha256": null,
+  "weights_unchanged": true
+}