Skip to content
AI Primer
workflow

LongTracer opens local STS+NLI claim checks for RAG validation

LongTracer open-sourced local STS+NLI claim checks, while qi published a private search engine with a Claude Code plugin and LM Studio users shared MCP search configs for Qwen. Use these stacks to ground retrieval and verify answers without a second judge model.

3 min read
LongTracer opens local STS+NLI claim checks for RAG validation
LongTracer opens local STS+NLI claim checks for RAG validation

TL;DR

  • LongTracer's launch post open-sourced a local claim checker for RAG outputs, using claim splitting, semantic matching, and NLI classification instead of a second judge model.
  • According to the LongTracer repo, the stack uses all-MiniLM-L6-v2 for sentence matching and nli-deberta-v3-xsmall for entailment, contradiction, or neutral labels, with adapters for LangChain, LlamaIndex, Haystack, and LangGraph.
  • qi's launch post packaged a separate local-first retrieval layer, combining BM25, embeddings, and cited Q&A over private docs, codebases, logs, and notes.
  • The qi README says it now ships as a Claude Code plugin, while an LM Studio thread shows users wiring similar web search tools into local Qwen setups through MCP servers and mcp.json.

You can read the LongTracer repo for the exact verifier pipeline, browse qi's command set for its index, query, and ask flow, and the LM Studio MCP docs confirm that local and remote MCP servers can be installed either through mcp.json or an "Add to LM Studio" button.

LongTracer

LongTracer is the more interesting release here because it targets the ugly part of production RAG, checking whether a generated answer actually stays inside retrieved evidence. The Reddit launch post describes a three-stage local pipeline, and the official repo fills in the model choices and adapters.

  • Claim splitting breaks a response into atomic statements.
  • STS matching maps each claim to the most relevant source sentence.
  • NLI verification labels that pair as entailment, contradiction, or neutral.
  • A final verdict turns those labels into a trust score and hallucination count.

The repository also says the verifier stays vector-store and model agnostic, takes plain strings as input, and can persist traces to SQLite, PostgreSQL, MongoDB, or Redis.

r/LLMDevs

We open-sourced LongTracer (MIT): A local STS + NLI pipeline to detect RAG hallucinations without LLM-as-a-judge

0 comments

qi

Where LongTracer verifies outputs, qi handles retrieval. Its launch post pitches it as a local search engine for private knowledge bases, and the README is unusually concrete about the mechanics.

  • BM25 full-text search runs on SQLite FTS5.
  • Vector search can use Ollama, LM Studio, llama.cpp, MLX, or OpenAI models.
  • Hybrid mode fuses lexical and vector rankings with Reciprocal Rank Fusion.
  • qi ask returns grounded answers with citations.
  • The plugin can be installed in Claude Code with /plugin marketplace add itsmostafa/qi and /plugin install qi.

That makes this a neat companion piece to LongTracer, one tool for grounding the answer, one tool for checking whether the answer stayed grounded.

r/LLMDevs

A local knowledge search engine for AI Agents

0 comments

LM Studio MCP

The third thread is less a product launch than a glimpse of how local-agent users are assembling the same stack by hand. In the LocalLLM thread, commenters point to SearXNG, Bing, and Exa-backed MCP servers, while one reply shares a minimal mcp.json snippet that connects LM Studio to a local SearXNG instance.

The official LM Studio docs back up the configuration path: since version 0.3.17, LM Studio can host both local and remote MCP servers, added either by editing mcp.json or through install buttons when a server exposes one. That turns the Reddit thread from hacky folklore into a documented feature.

r/localLLM

How to get qwen 3.5 using LM studio to search the internet?

23 comments

Share on X