Skip to content

explore all stories all tools all skills

All Tools›Categories›Evals & Observability

📊

Evals & Observability

159 tools

Evaluation harnesses, LLM tracing, monitoring, and prompt/agent observability. Dashboards, judges, replay tools, and regression suites for LLM apps.

Arena Intelligence, Inc.

Introducing Agent Mode: Agentic AI is now measured in the Arena.

Test LLMs on real tasks. Compare models side-by-side.

Build on the Claude Platform

Artificial Analysis

Agentic Knowledge Work Benchmark

Applied Theory LLC

AI Agent Compliance, Governance & Control

Arena Intelligence, Inc.

Benchmark AI models on real agent work

AI agents for your enterprise

Agent Development Kit (ADK)

Build powerful AI agents with flexibility and control.

Kubernetes SIG Apps

A Kubernetes-native solution for running AI agents in isolated environments.

Live Per-Session Quota Burn for Codex and Claude on macOS

Property viewings reimagined.

Open-source agent tracing

The missing feedback layer for AI coding agents.

Every MCP server & agent tool, ranked

Kenn Software LLC

A local-first desktop and web app for browsing, searching, and analyzing your past AI coding sessions.

Outshift by Cisco

The Internet of Agents

Listen to users at scale with AI

One API for all AI models.

Optimization framework for real-world AI agents

A long-context eval

Amida Technology Platform

Amida Technology Solutions, Inc.

A Low-Code AI Test Automation Platform for Enterprise Applications

Platform for building, running, and evaluating any Safe Autonomous Organization.

Antithesis Operations LLC

Autonomous Testing Platform

ARC Prize Foundation

A benchmark for measuring intelligence in interactive environments

Open-source AI observability and evaluation.

Artificial Analysis

Artificial Analysis, Inc.

Independent AI model and API provider benchmarking

Generative Artificial Intelligence Research Lab

Let AI Do the Research, You Keep the Insight

Adaptive Spec-driven Scoring for Evaluation and Regression Testing

Attention Head Visualiser

Mapping What Each Head in GPT-2 Actually Does

Open-source framework for agentic quantitative finance research.

AutomationBench

AI benchmark leaderboard

AutomationBench-AA

Artificial Analysis

Independent leaderboard for Zapier's AutomationBench

Baidu AI Cloud Qianfan

AI-native application development platform

Compare User Productivity to Real Peers

Build reliable, testable, production-grade AI agents with Better Agents CLI - the reliability layer for agent development

Block-Sparse Featurizers

A family of methods for decomposing activations into multidimensional concept blocks.

Braintrust Data, Inc.

The AI observability platform.

The World's #1 Vibe Coding Benchmark

BrowseComp-Plus

A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent

A CLI tool for analyzing Claude Code usage from local JSONL files.

Chrome DevTools MCP

Chrome DevTools for coding agents

A minimal browser extension that shows token count, cache timer, and usage bars on claude.ai.

Hermann Haraldsson

ESP32 desk dashboard that shows Claude Code usage

Arena Intelligence, Inc.

Build & Test with AI Coding Models

Unverified software product

Explore Interfaces Inc.

The analytics platform for LLM products.

The simulation and evaluation platform for AI agents.

A benchmark for computer-use agents on professional software

De-risk AI answers without leaving ChatGPT, Claude, Gemini, and other AI tools.

Benchmarking AI models on real-world cybersecurity vulnerabilities.

Frontier AI for cyber defenders.

DeepAgents Deploy

An open alternative to Claude Managed Agents

Diffusion-based speculative decoding for faster LLM inference

LangGenius, Inc.

The platform for agentic workflow development.

Dogfood your product, the efficient way

Stanford NLP Group

Programming—not prompting—language models.

Measuring real-world environment learning and discovering a new scaling law

Enterprise Worlds

Executable environments for training and evaluating AI agents on realistic enterprise workflows

EnterpriseRAG-Bench

A RAG benchmark for the real world.

Developer platform for humans and agents

A benchmark for end-of-turn voice detection

FrontierMath: Open Problems

A collection of unsolved mathematics problems that have resisted serious attempts by professional mathematicians.

Fullstack Code Arena

Build, Deploy, and Evaluate with Fullstack Code Arena

AI Agents hallucinate, fix it faster.

Gemini Enterprise

The AI platform for work.

A research-level benchmark for a harder kind of AI progress.

Sky Computing Lab

Reflective prompt evolution for AI systems.

Make prompt optimization visible and cheaper to inspect.

GitHub Repository Signals

Extract and score leads from GitHub repositories.

Google AI Edge Gallery

Run AI models locally on your Android device.

Benchmark LLM inference performance

Carnegie Mellon University

Verified desktop-app agent environments

Hermes Agent Control Room

Control Room-first template for managing Hermes agents from one VPS agent to specialist teams and orchestrated workflows

How to Eval AI Agents

AGI Context, Inc.

The Graph AI Runs On.

Automated multi-turn agent for alignment auditing of language models.

Interfere, Inc.

Ship software that never breaks

Enterprise-agent evaluation benchmark for IT service management

Scaling Intelligence Lab

Can LLMs Write Efficient GPU Kernels?

Because your benchmark score doesn't pay the API bills.

Insure your agent. Get covered, fast.

The platform for agent engineering.

LangSmith Engine

Turns agent traces into issues, evals, fixes, and memory updates.

LangSmith Fleet

Collaborative agent workspace for sharing and running capable agents where teams work.

LangSmith Sandboxes

Give agents a secure computer

Latitude Data S.L.

Open-source agent monitoring.

Intelligent Ollama Model Selector

Parallel models, rankings, and synthesis for transparent deep research.

Elastic Context Orchestration for Long-Horizon Search Agents

Lucent AI, Inc.

Turn audience intelligence into landing pages that convert.

You ask. It works.

Kepler Software Inc.

The TypeScript AI agent framework.

MCPMark Verified

MCPMark: Stress-Testing Comprehensive MCP Benchmark

Open medical LLM evaluation suite

Stanford IRIS Lab

Searches for better scaffolding around a fixed model

micro1 Intelligence Platform

Human intelligence platform for AI training, evaluation, and data workflows at scale.

Microsoft Agent 365

The control plane for AI agents

Mini GEPA implementation

A long-horizon SWE benchmark for autonomous coding runs.

Mistral AI Studio

Build your frontier with Studio.

Probe LLM knowledge cutoffs with dated software release versions.

Parallel Web Systems, Inc.

Automated web monitoring for news, products, research, filings, events, corporate development

A Big-World Benchmark for Continual Reinforcement Learning

Cornell University

Material-Point-Method Simulations for Inferring and Extrapolating Physical Dynamics

Neo Research Inc.

Fully autonomous AI engineering agent

Visibility into AI agent activity on endpoints, with on-device detection, optional pre-action blocking, and forensic reconstruction.

Stanford University

An Agentic Framework with Extensible Tools for Complex Reasoning

Allen Institute for AI

A fully open-source framework for instruction tuning and preference tuning of LMs.

OpenAI Agents SDK

A lightweight, powerful framework for multi-agent workflows.

Evals is a framework for evaluating OpenAI models and an open-source registry of benchmarks.

Open-source background agent system

Open-source LLM evaluation and observability platform

Overmind Technology Inc.

Understand your infrastructure.

Paper Assistant

AI-assisted scientific paper review

AI-powered operations for your business

Score your tweet with the open-source X algorithm.

Vibe-training for real-time, tailored agent evals and guardrails.

Prime Intellect

Autonomous AI research agent

Promptfoo, Inc.

AI security testing and evaluation

Decoding Intelligence, Unleashing Potential

Raindrop Workshop

Local trace inspection for AI-agent runs

The #1 AI spreadsheet editor, built for financial modeling and analysis

An interactive visualization of Cloudflare Agents' chat recovery system.

Recursive Superintelligence, Inc.

Recursive self-improving superintelligence to automate knowledge discovery.

Rerun Technologies AB

The multimodal data stack for physical AI.

Amazon Web Services

Robust Panel of LLM Judges

Sakana AI Recursive Self-Improvement (RSI) Lab

A dedicated research group within Sakana AI, tasked with redesigning the AI development process itself with AI.

AI Reliability Platform

AI debugging agent

The sensor that helps AI agents close the feedback loop.

Sentry MCP Server

Give your AI assistant access to Sentry

Automated interpretability and RL experimentation platform

Optimize agent skills against measured task batches

The MCP Registry

Open Medical Superintelligence

A skill for removing AI tells from prose.

A more challenging benchmark for coding agents

10x Faster Bug Detection

Tessl AI Limited

AI-native development platform

Open Source Agile Planning Poker app

Thinking Machines Lab

A research API for fine-tuning language models

Count tokens before sending a request

Lean output compaction for terminal-heavy agent workflows.

Claude Code token cost, in your menu bar

LightSeek Foundation

A speed-of-light LLM inference engine.

Traversal, Inc.

The AI SRE for complex systems

Track what matters. Skip the infrastructure.

Insurance for your cloud commitments.

Benchmark Generative AI for Enterprise Applications.

The AI product development platform.

Vercel Observability

Monitor, debug, and understand your applications on Vercel.

An open-source Large Multimodal Model (LMM) powered agent for computer vision.

Evaluation tooling for Vitest

Weights & Biases

Explore and compare local W&B runs from the terminal with the LEET (Lightweight Experiment Exploration Tool) TUI.

Benchmark for deep and wide research

Zero Trust, run as a service.

Weights & Biases

Weights & Biases

The AI developer platform.

a decision-based fidelity benchmark for world models

AI Force Singapore Pte. Ltd.

One API for all AI models.

AI Primer

Your daily guide to AI tools, workflows, and creative inspiration.

© 2026 AI Primer. All rights reserved.