LFM adapters reportedly cut Pi tool calls by up to 40%

• The Pi adapter offers a tidy efficiency win: yb2698 reports up to 40% fewer tool calls with new LFM2.5-24B LoRAs.
• The 100-step Pi checkpoint gave the best reported tradeoff; the Claude Code variant showed no meaningful boost, according to the author's comparison.
• Training took roughly 24 hours on one B300, with inference and training sharing the GPU, yb2698 said.
• GGUF adapters are available for llama.cpp, as announced in the release thread.

Answer correctness supplies the reward in the Pi model card, which lists no explicit tool-call penalty. Claude Code's results table leaves its step-100 native-harness scores marked TODO, while the Pi GGUF release packages the adapter as a roughly 37 MB addition to a separate base model.

The release adds Pi and Claude Code LoRAs to LiquidAI's LFM2.5-24B-A2B; yb2698 confirmed these are two separate adapters. The author picked Pi's 100-step checkpoint for its balance of performance retention and tool-call efficiency.

The Pi model card reports:

• Tool calls: 39% fewer on tasks that both the adapter and base model solved.
• Accuracy: pass@1 fell 4.2 percentage points, within the card's estimated ±5-point noise.
• Evaluation: the first 100 held-out SmolDataEnvs data-analysis tasks, one attempt per task at temperature 0.8.

The efficiency result uses a paired subset of successful tasks. Training and evaluation cover data analysis, so the card leaves transfer to other coding work open.

The Claude Code run produced no meaningful boost, according to yb2698. Its model card reports these step-100 results when evaluated under Pi:

• pass@1: 1 percentage point below the base model.
• Tool calls: 2% fewer on tasks both models solved.

The serving paths also differed: Claude Code's evaluation used checkpoints merged into the base model on vLLM 0.25.1, while Pi's evaluation used unmerged adapters. The card notes that small numerical differences between those setups are expected.

Training ran Pi inside task sandboxes; yb2698 said it used no synthetically generated Pi traces.

The released configuration specifies:

• Training data: the first 1,000 tasks from the 5,000-task SmolDataEnvs Harbor training suite, whose answers are checked deterministically.
• Trainer: experimental TRL AsyncGRPO, with Harbor tasks served through OpenEnv.
• LoRA: rank 32, alpha 64, all linear layers, learning rate 1e-5.
• Rollouts: eight per task, capped at 17 model calls and 600 seconds, with up to 16,384 tokens per model call.

The model cards document different client contracts:

• Pi: the vLLM configuration sets "api": "openai-completions" in ~/.pi/agent/models.json. The Responses API path was not used for training or evaluation.
• Claude Code: the serving configuration uses vLLM's native /v1/messages endpoint without a proxy. ANTHROPIC_BASE_URL omits /v1, ANTHROPIC_AUTH_TOKEN must be set, and all three model-tier aliases point to the adapter to keep background calls on the same server.
• Parsing: both configurations specify --tool-call-parser lfm2 and --reasoning-parser qwen3; the Pi card explicitly requires the former for parsing tool calls.

The LoRAs load directly in llama.cpp, as yb2698 confirmed.

Published evaluation scores came from vLLM at BF16. The GGUF card says the conversion has not been re-evaluated through llama.cpp, and quantizing the base model can change the results.