Safety record
Council Workbench System & Safety Card
What actually governs Council Workbench's outputs, what has been measured, and — just as important — what has not yet been measured. A governance company publishing an unverified safety claim about its own model would be the exact failure mode it exists to catch in others. This card is held to that standard.
Every response passes a care-floor check (threshold 0.95) before any vote or output logic runs — it is a pre-gate, not a vote-dependent filter. Measured across every governance topology tested (20 configurations × a 60-item ground-truth battery):
Reading this honestly: an obvious care-floor breach (score below 0.35) is hard-gated to reject regardless of votes — that part is a guaranteed, unconditional stop. A harder case — content that reads as confident but is actually harmful, with 2–3 compromised voting nodes attempting to force it through — is where the real signal lives: signed-vote verification (forged-vote rejection) measurably matters, but containment there is 58–79%, not 100%. The residual risk is backstopped by escalation to a human/central reviewer, not eliminated. We do not claim perfect containment under adversarial pressure.
| Model | Metric | Samples | Verdict |
|---|---|---|---|
| creativity_assessment_nn | r² 0.91 | 350 | strong |
| care_pattern_analyzer | mae 0.037 | 600 | strong |
| relationship_evolution_nn | mae 0.071 | 500 | strong |
| care_validation_nn | mae 0.19 | 19 | tiny sample |
| partnership_detection_ml | mae 0.22 | 19 | tiny sample |
| threat_detection_nn | acc 0.45 | 33 | weak — needs retrain |
| dependency_detection_nn | acc 0.22 | 50 | weak — needs retrain |
3 of 7 signal models are strong on real, adequately-sized held-out data. The other 4 — including threat detection and dependency detection, two of the more safety-relevant signals — are trained on too few labelled examples to trust yet. They are consulted for their measured reliability, not treated as ground truth; a weak signal's output is weighted down, never silently upgraded to a confident claim.
When Council Workbench runs as an ensemble (the multi-model council configuration), the safety-relevant finding is: lineage diversity dominates topology shape. Every diverse-lineage configuration outperformed every identical-lineage configuration in the measured battery — 5 identical copies of one model correlate their votes (ρ 0.33–0.58) and collapse toward roughly 1 effective independent vote ("quorum theatre"); 5 distinct model families (Qwen/Llama/DeepSeek/Gemma/Mistral) stay near-independent (ρ 0.04–0.19), keeping close to their full effective-vote count. This governs how the council is configured, not just a research curiosity.
- ◐Raw capability vs. frontier models (GSM8K / MMLU head-to-head) — the governance-topology results above measure decision-quality and safety, not language/reasoning capability. That comparison requires a dedicated benchmark run and has not been done.
- ◐Formal red-teaming by an external party — internal adversarial tests exist (see the laundered-harm figures above) but no independent red-team has evaluated Council Workbench.
- ◐Refusal-rate / false-positive-rate on benign edge cases at scale — measured on a 60-item battery, not yet on a large, diverse real-traffic sample.
Last reviewed 2026-07-12. Figures above come from internal governance-topology sweeps under a stated error model — re-run scripts and full config tables are referenced in the whitepaper.