Key points
- Inkling 2 Small produced 316 instances of profanity across 2,655 benchmark runs.
- Most occurrences happened late in long-running tests that ultimately failed.
- Profanity was not a reliable failure signal, as 96 percent of failed runs contained none.
Context
If you assess AI models, this case shows why internal reasoning traces should not be treated as straightforward indicators of failure. Inkling 2 Small’s profanity appeared in only a small share of failed benchmark runs, making it a poor stand-alone diagnostic for developers and technical decision-makers.
What this means for you
Directly applicable
When you evaluate AI models, do not rely on profanity in reasoning traces as a failure signal, since 96 percent of failed runs showed none.
Who benefits: Developers and technical decision-makers benefit by avoiding a misleading profanity-based heuristic for detecting model failures.
Share article: