Artificial Analysis
Independent AI model and API provider benchmarking
AI-model intelligence platform providing benchmarks, leaderboards, and analysis for evaluating models and AI API providers across quality, speed, latency, and cost.
Recent stories
Xiaomi released Pro and Flash MiMo-V2.6 mixture-of-experts models with open weights and a 1M-token context window. The release includes an RL dashboard and day-one vLLM support, while training artifacts are planned.
Two analysts argue that DeepSeek V4.1 Flash's results across benchmark vintages are consistent with public-benchmark contamination. The model also leads Artificial Analysis's new private evaluation, complicating the assessment.
OpenAI released GPT-Live-Transcribe and GPT-Transcribe in the API for streaming and offline speech recognition. The launch adds context prompts, keywords, language hints, and WER gains, with Artificial Analysis reporting GPT-Transcribe at 3.31% AA-WER and $4.50 per 1,000 audio minutes.
Artificial Analysis launched AA-Briefcase, a benchmark for multi-week knowledge-work projects with thousands of source files, and Claude Fable 5 leads at 1587 Elo. The first results show a wide cost spread, so teams should compare both quality and task cost before choosing a model.
Artificial Analysis launched AA-WER Streaming to benchmark streaming speech-to-text models on accuracy and latency for voice agents. The first leaderboard puts Cartesia Ink-2 and ElevenLabs Scribe v2 on the price-latency frontier, so teams should compare cost against latency before choosing a model.