The measurement
The gap between saying and doing.
Instruction benchmarks ask whether a model does what it's told. We ask whether it stays honest, fair, and safe while doing it. For each virtue we record what the model says about itself, then what it does when tested. The distance between them is dissonance.
Illustrative example, not a real audit
Bar: observed behavior. Marker: what the model says about itself. Shaded: the gap between them, dissonance.
The framework
Ten virtues, each measured separately.
A single score hides where a model fails. Measuring each virtue shows which values hold under pressure and which do not.
How an audit works
Four steps, one report.
- ScopeThe model or deployed assistant, its operating context, and the testing tier.
- TestStructured scenarios that probe each virtue, including cases where values pull against instructions with varying degrees of severity.
- JudgeResponses are scored against written rubrics by an independent judge panel and human reviewers.
- ReportThe report includes virtue scores, say-do gaps with examples and recommendations along with individualized statistics.
Products
Test it. Fix it. Keep watching.
Alignment Audit
A one-time assessment of your model or deployed assistant across all ten virtues, with a written report and recommendations.
Tiers: Quick, Standard, Comprehensive. Pricing on request.
Request Quick AuditFramework License
Continuous monitoring in your pipeline. Drift alerts when a model's behavior moves away from its audited baseline.
Chatbot Check
A test of the chatbot on your business’s website. We check it against your own policies and prices, and show you where it makes things up, caves under pressure or promises what you never approved.
$750 per chatbot, retest included.
Learn moreRequest an audit
Request Quick Audit.
Tell us what you'd like assessed. We'll reply by email to schedule a short scoping call and send a quote.