4M-parameter BERT Tiny reportedly beats Opus and Kimi after training on 10,000 examples

• The reported 4M BERT-tiny win over both Opus and Kimi conflicts with MaximeRivest's chart, which plots BERT-tiny at 89.2% and its Opus reference line at 92%.
• A 17M Ettin scored 91.5% after 42 seconds on Banking77, while Opus is shown at 92% on only 200 test questions, per MaximeRivest's Banking77 chart.
• Rivest's 10-task experiment priced Kimi K3 synthetic labels at $0.77 to $30 and measured label generation from eight seconds to about seven minutes, according to MaximeRivest's ten-task post.
• One attached device table puts training the same 17M model at 11 minutes on a laptop CPU and 21 minutes in a Galaxy S21 browser GPU, according to MaximeRivest's device chart.

The Ettin 17M model card labels the base model as a mobile and edge configuration, with fine-tuning examples. Banking77 contains 13,083 customer-service queries across 77 fine-grained intents, while an independent agentic fine-tuning log describes using an agent to generate data, train, evaluate, and diagnose a local intent classifier.

Rivest's visual calls the result a 4M-parameter BERT-tiny model trained on 9,493 human-labeled messages. Its final plotted accuracy is 89.2%, 2.8 points below the chart's 92% Opus line; the chart also says none of the training curves had flattened.

The tweet's “beats Opus” wording therefore goes beyond the plotted result. The visual supports a stronger comparison with Kimi only through a separate 200-question measurement, which the Banking77 chart qualifies with a plus or minus two to three point range.

This is a domain-specific intent-routing comparison, not a measure of general reasoning. The current Banking77 dataset card lists 10,003 training examples and 3,080 test examples; Rivest's chart reports 9,493 training examples and 3,076 test questions.

The chart's five rows are:

• ModernBERT, 150M parameters: 93.2% accuracy after 77 seconds.
• Ettin, 17M parameters: 91.5% after 42 seconds.
• Claude Opus 5.5: 92% on a 200-question sample, with 2.4-second latency and a listed $8,120 per million messages.
• Kimi K3: 82% on that sample, 0.5-second latency, and $1,540 per million.
• Jev: 79.4%, 0.12-second latency, and $4 per million.

The chart footnote says the fine-tuned models were tested on 3,076 questions, while the two frontier-model figures came from 200. It also assigns an uncertainty of plus or minus two to three points to those samples.

Rivest's ten-task run used Kimi K3 to produce labels, then fine-tuned the 17M Ettin. The reported pipeline breaks into three measurable parts:

• Synthetic-data generation: eight seconds to about seven minutes, costing $0.77 to $30.
• RTX 3090 training: 45 seconds to three minutes.
• Comparison with Jev: the post says the distilled model exceeded Jev on two of ten tasks and landed in its ballpark on five more.

The attached chart titles the two favorable tasks as ties rather than wins, and gives test-set uncertainty of plus or minus 1.5 to 2 points. It also says the student remained two to six points behind Kimi K3, and flags TREC, tweet hate speech, and SST-5 as cases where Kimi itself trailed Jev. In those cases, the chart says its break-even calculation is not like-for-like.

Rivest framed the work as optimizing AI programs through application-specific fine-tuning MaximeRivest's clarification. He said he ran the GPU experiment himself on an RTX 3090 while making API calls MaximeRivest's hardware note, and said a longer write-up is in progress MaximeRivest's follow-up.