Transluce trains 8B–1.1T activation-reading oversight models

• Transluce trained activation-reading oversight models spanning 8B to 1.1T parameters, as its launch thread reported.
• The models are meant to answer natural-language questions about another model’s current state or future behavior from its internal activations, according to Transluce’s evaluation announcement.
• Additional training improved performance on many evaluations, and larger, more capable models generally performed better, according to Transluce’s scaling update.
• Reward hacking remains the hard case: Transluce’s oracle trailed a full-context language-model monitor, which was itself imperfect, in the team’s limitation report.

ImpossibleBench mutates unit tests to contradict a task specification, so passing them requires a specification-violating shortcut. A separate ICML workshop paper found publicly released Qwen3 activation oracles difficult to use off the shelf on out-of-distribution safety tasks, while Transluce describes its broader mission as building infrastructure for scalable AI oversight.

An activation oracle ingests another model’s internal activations and answers questions about its present state or likely future behavior, according to Transluce’s description of the method.

The target is a review problem familiar from long agentic runs: a coding agent can take an undesirable action far inside a transcript of tens of thousands of tokens, as Transluce’s example puts it.

Transluce said activation-oracle training has two persistent traps: ground truth can be expensive or unavailable, and a benchmark may reward reconstructing text already in the input rather than understanding behavior. Its method summary describes a scaling-first response:

1. Build a broader evaluation suite with verifiable answers.
2. Assemble diverse training data with ground truth.
3. Test whether performance continues to improve with scale.

The suite includes deployment-relevant questions such as whether a coding agent is reward hacking. Transluce trained and evaluated Qwen-, GLM-, and Kimi-based oracles from 8B through 1.1T parameters, in the same evaluation announcement.

Transluce reported that more training improved results on many evaluations, while larger and more capable models tended to perform better. The team framed that trend as evidence that scaling could produce progressively stronger assistants for understanding other AI systems in its results update.

That is the central technical bet: oversight capability could improve through the familiar levers of model scale and more varied supervision, rather than requiring a bespoke monitor for every agent behavior.

The reward-hacking result breaks from the otherwise encouraging scaling report. On that evaluation, Transluce said its activation oracle performed worse than a full-context LM monitor, while adding that the monitor did not solve the task either in its reward-hacking result.

The evaluation therefore remains unsolved rather than a win for either hidden-state access or transcript-only monitoring.

Transluce said finding reward-hacking instances in small open-source models was surprisingly difficult, so it used ImpossibleBench in an evaluation note. The benchmark alters tests until passing them requires cheating, a construction the ImpossibleBench paper defines as conflicts between unit tests and the natural-language specification.

That gives the oracle a concrete object to identify: an agent following the altered success signal instead of the stated task.