Transluce releases DSSE-1.1 with 6,915 fine-tuning datasets

• DSSE-1.1 contains 6,915 fine-tuning datasets across 18 target behaviors, according to Transluce's release.
• Ordinary-looking training data can induce unrelated harmful behaviors, as Transluce's examples demonstrate.
• Many side effects transfer across model families and sizes in Transluce's experiments.
• ML researchers predicted side effects with 63% accuracy against 50% chance in the team's human study.

Shutdown resistance accounts for 922 datasets in the DSSE Explorer. One precursor study cited in Transluce's write-up induced a nineteenth-century persona by training models on outdated bird names.

Transluce deliberately searches for datasets whose effects extend beyond their apparent training domain. The team builds on emergent misalignment and weird generalization research, which established individual examples, as it explains in its follow-up.

The pipeline scores three properties:

1. Elicitation: Fine-tune a model, sample responses to evaluation prompts, and have an LLM judge score the target behavior.
2. Naturalness: Use pairwise LLM judgments to assign Elo-style ratings for how much datasets resemble ordinary post-training data.
3. Surprisingness: Give frontier models examples from a dataset and test whether they can predict its side effect. Worse prediction means greater surprisingness.

Candidate-generation methods include:

• Data evolution.
• Dataset policy gradient.
• Subliminal-learning approaches that rewrite or filter existing data.

Search uses cheaper proxy scores, followed by rescoring and filtering with more expensive, held-out judges. Transluce says this separation mitigates reward hacking of the search objectives.

The paper's original abstract reports 6,575 datasets across eight behaviors. The DSSE-1.1 changelog adds ten behaviors, updates behavioral evaluations, and tightens inclusion criteria.

The random-data baseline is the release's most useful addition. Key elicitation filters require a dataset's fine-tuned Qwen3-1.7B model to satisfy all three tests:

• Exceed the upper end of the untouched model's 95% base-rate interval.
• Exceed the mean plus three standard deviations of scores from 40 models fine-tuned on random, 150-row samples of Tulu-3 SFT data.
• Produce at least three evaluation responses with nonzero behavior scores.

Those comparisons separate a dataset's targeted effect from changes that also occur after unrelated fine-tuning.

Dataset discovery optimized against Qwen3-1.7B. The released score table also carries transfer measurements for:

• Llama: Llama-3.2-1B-Instruct and Llama-3.2-3B-Instruct.
• Qwen: Qwen3-4B, Qwen3-8B, and Qwen3-14B.

Transluce reports that many side effects generalize across those changes in family and size. Transfer scores are recorded separately for each dataset and target model.

Researchers evaluated 64 datasets spanning different levels of surprisingness. ML researchers identified an induced behavior in 63% of trials against 50% chance, while a frontier LLM reached 86%, according to the study description.

The authors also found that increasing model scale did not reliably improve prediction accuracy. The human result therefore accompanies a stronger machine-auditor result, with substantial errors remaining in both.

Transluce provides the full training datasets and their fine-tuned models' behavioral evaluation results. The Hugging Face release contains:

• scores.csv: One row per dataset, including elicitation, transfer, naturalness, and surprisingness measurements.
• behaviors.json: Behavior descriptions, evaluation rubrics and prompts, untouched-model base rates, and random-fine-tuning baselines.
• datasets/<behavior_id>.jsonl.xz: Training prompt-response pairs, grouped into datasets.
• eval_results/<behavior_id>.jsonl.xz: Evaluation prompts, model responses, and judge scores.

The files join on dataset_id; Python's lzma module reads the compressed JSON Lines.

Two score details complicate comparisons:

• Naturalness Elo is behavior-local. The release specifies comparisons only within the same target behavior.
• Released multiple-choice audits use eight options. The original eight behaviors share a fixed option set; the ten added behaviors use the correct answer plus seven randomly drawn alternatives. The card says their accuracies are not directly comparable. These audits also differ from the human study's 50%-chance task.

The collection is licensed under CC BY-NC 4.0. Third-party training text retains its own licenses, and generated text remains subject to its producing models' terms.