Skip to content

explore all stories all tools all skills

All Tools›Categories›Evals & Observability

📊

Evals & Observability

149 tools

Evaluation harnesses, LLM tracing, monitoring, and prompt/agent observability. Dashboards, judges, replay tools, and regression suites for LLM apps.

Benchmark Generative AI for Enterprise Applications.

Artificial Analysis

Agentic Knowledge Work Benchmark

Artificial Analysis

Artificial Analysis, Inc.

Independent analysis of AI

Block Diffusion for Flash Speculative Decoding

The Agent Engineering Platform

Weights & Biases

Weights and Biases, LLC

The AI developer platform

The AI Gateway for developers

IT service management environment for testing whether agents can carry out policy-governed work inside a system of record.

Kepler Software Inc.

TypeScript AI Framework for Agents and Apps

OpenAI Agents SDK

Build sandbox, text, and voice agents with a small set of primitives.

Build AI models the way you write software

Text-space optimization for frozen agents

Arena Intelligence, Inc.

AI Agent Performance Leaderboard

Arena Intelligence, Inc.

Crowdsourced AI Model Evaluation Platform

AutomationBench-AA

Artificial Analysis

Agentic SaaS Workflow Benchmark

An enterprise-grade, all-in-one large model platform centered on Agent development

Block-Sparse Featurizers

Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds

Braintrust Data, Inc.

The active observability platform for agents

Chrome DevTools for agents

Help your agent build, debug, and verify your code correctly.

Develop with Claude

Arena Intelligence, Inc.

Build & Test with AI Coding Models

Explore Interfaces Inc.

Build, deploy, and improve AI agents.

Make your agents better at computers

Frontier AI for defenders.

Enterprise Worlds

Executable enterprise environments for measuring operational agents

A research-level benchmark measuring how AI agents navigate ambiguity and make consequential judgments in computational biology.

Carnegie Mellon University

Turn Any Software into an Agent Environment

LangSmith Engine

Your proactive agent engineer

LangSmith Sandboxes

Give agents a secure computer

Latitude Data S.L.

Make your AI agents self-healing

Open medical AI benchmarks

What's the largest software project AI can complete on its own?

A Big-World Benchmark for Continual Reinforcement Learning

Visibility into AI agent activity on endpoints, with on-device detection, optional pre-action blocking, and forensic reconstruction.

Stanford University

An Agentic Framework with Extensible Tools for Complex Reasoning

The real world trust platform for AI agents

Raindrop Workshop

The local debugger your agent is missing.

The #1 AI spreadsheet editor, built for financial modeling and analysis

Sentry MCP plugs Sentry's API directly into your LLM.

Evaluating challenging long-horizon software engineering tasks in public open source repositories

Thinking Machines Lab

Tinker is a training API for researchers

Lean output compaction for terminal-heavy agent workflows.

LightSeek Foundation

Speed-of-light LLM inference

A benchmark for high-volume, evidence-heavy knowledge work

Applied Theory LLC

AI Agent Compliance, Governance & Control

Production AI agents

Agent Development Kit (ADK)

Build production agents, not prototypes.

Kubernetes SIG Apps

Enables easy management of isolated, stateful, singleton workloads, ideal for use cases like AI agent runtimes.

Live Per-Session Quota Burn for Codex and Claude on macOS

Property viewings reimagined.

Open-source agent tracing

Visual feedback for AI coding agents

Google PageRank for AI agents.

Kenn Software LLC

A local-first desktop and web app for browsing, searching, and analyzing your past AI coding sessions.

Outshift by Cisco

Building the Internet of Agents (IoA)

Optimization framework for real-world AI agents

A framework for evaluating multi-turn LLM conversations with support for text, realtime audio, and speech-to-speech models.

Platform for building, running, and evaluating Safe Autonomous Organizations

Bug-free systems, unlimited velocity

ARC Prize Foundation

The first interactive reasoning benchmark designed to measure human-like intelligence in AI agents.

Generative Artificial Intelligence Research Lab (GAIR)

Let AI Do the Research, You Keep the Insight

Adaptive Spec-driven Scoring for Evaluation and Regression Testing

Attention Head Visualiser

Mapping what each head in GPT-2 actually does

an open-source agentic framework for quantitative finance research

Test LLMs on real tasks. Compare models side-by-side.

Directory of AI Benchmarks

Build reliable, testable, production-grade AI agents with Better Agents CLI - the reliability layer for agent development

The World's #1 Vibe Coding Benchmark

BrowseComp-Plus

A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent

Coding (Agent) CLI Usage Analysis

Shows ~token count, cache timer, and native session/weekly usage bars on claude.ai.

Claude Token Count API

Count the number of tokens in a Message.

ESP32 desk dashboard that shows Claude Code usage

Unverified software product

Voice AI Testing & Evaluation Platform

Can AI agents find security bugs and patch them?

Deep Agents Deploy

Deploy a model-agnostic, open source agent to production with a single command.

LangGenius, Inc.

The Platform for Production-Ready Agentic Workflows.

Dogfood software product

Stanford NLP Group

Program, don’t prompt, your LLMs.

An ultra-long-horizon benchmark built to measure learning from environments

EnterpriseRAG-Bench

A RAG benchmark for the real world.

Developer platform for humans and agents

The open benchmark for end-of-turn detection

Fullstack Code Arena

Build, Deploy, and Evaluate with Fullstack Code Arena

AI Agents hallucinate, fix it faster.

Gemini Enterprise Agent Platform

Innovate, build, and deploy enterprise ready agents

Optimize prompts, code, and more with AI-powered Reflective Optimization

Live visualization for GEPA prompt-optimization runs.

GitHub Repo Signals

Extract and score leads from GitHub repositories using stars, forks, issues, PRs, comments, and contributions.

Google AI Edge Gallery

Explore, Experience, and Evaluate the Future of On-Device Generative AI with Google AI Edge.

SLO-aware Benchmarking and Evaluation Platform for Optimizing Real-World LLM Inference

Hermes Agent Control Room

Control Room-first template for managing Hermes agents from one VPS agent to specialist teams and orchestrated workflows

Unverified vendor

Unverified software product

AGI Context, Inc.

The Graph AI Runs On.

Interfere, Inc.

Production visibility for teams that ship fast

Scaling Intelligence Lab at Stanford University

Can LLMs Write Efficient GPU Kernels?

AI Coding Model Benchmark Results

AI Agent Insurance, Certification & Guarantee

LangSmith Fleet

Agents for the whole company

Intelligent Ollama Model Selector

When the Stakes Are Too High for One AI Answer

Elastic Context Orchestration for Long-Horizon Search Agents

Lucent AI, Inc.

See what's really happening in your product

You ask. It works.

MCPMark Verified

Stabilized, version-pinned subset of MCPMark's standard tasks for reliable and reproducible MCP evaluation.

Stanford IRIS Lab

End-to-End Optimization of Model Harnesses

Data lab to train frontier models & evaluate agents

Microsoft Agent 365

The Control Plane for Agents

Mini GEPA implementation

Your AI production platform.

Probe LLM knowledge cutoffs with dated software release versions.

Cornell University

Material-Point-Method Simulations for Inferring and Extrapolating Physical Dynamics

Neo Research Inc.

Autonomous AI Agent to build AI Agents, evaluate AI Agents, Optimize LLM Prompts, Train AI Models and Finetune LLMs

Observability for TrueFoundry applications and infrastructure.

AllenAI's post-training codebase

Evaluations test model outputs to ensure they meet style and content criteria that you specify.

Self-hostable background coding agents for software teams

AI Observability & Evals For the Agentic Era

Overmind Technology Inc.

Prevent your next outage

Paper Assistant Tool

Gemini-backed automated feedback for scientific manuscripts

Parallel Monitor API

Parallel Web Systems Inc.

Automated web monitoring for news, products, research, filings, events, corporate development

Expert data for AGI

The open-source platform for agent development and evaluation

Score your tweet with the open-source X algorithm.

Build Secure AI Applications

Recursive self-improving superintelligence to automate knowledge discovery.

The Data Layer for Physical AI

Amazon Web Services

Robust Panel of LLM-as-Judge

Sakana AI Recursive Self-Improvement (RSI) Lab

A dedicated research group within Sakana AI tasked with redesigning the AI development process itself with AI.

Stop guessing on retrieval quality

Seer knows everything. Ask it anything.

The sensor that helps AI agents close the feedback loop.

Connect agents to services in minutes

Open Medical Superintelligence

A skill for removing AI tells from prose.

10x Faster Bug Detection

Tessl AI Limited

Agent Enablement Platform

Open Source Agile Planning Poker app

Claude Code token cost, in your menu bar

Traversal, Inc.

The AI SRE for complex systems

Time-Series Metrics Made Simple

Your own, personal intelligence

Harness-backed AI evaluation tests on top of Vitest.

Weights & Biases

Explore and compare local W&B runs from the terminal with the LEET TUI.

Zero Trust, run as a service.

A decision-based fidelity benchmark for world models

AI Force Singapore Pte. Ltd.

Unified API for 100+ AI Models

AI Primer

Your daily guide to AI tools, workflows, and creative inspiration.

© 2026 AI Primer. All rights reserved.