Skip to content
AI Primer
workflow

Developers use Markdown files as long-term memory for agents

Developers described agents writing learned context into linked Markdown pages instead of repeatedly rebuilding it from RAG chunks. Related posts argued Markdown docs and llms.txt can improve agent navigation.

7 min read
Developers use Markdown files as long-term memory for agents
Developers use Markdown files as long-term memory for agents

TL;DR

  • Agents are moving long-term memory into linked Markdown: pauliusztin_'s architecture thread describes a wiki cache built from unified memory and served through MCP, while hwchase17 said OpenWiki is adopting OKF as an open memory format.
  • The hard part is keeping those files fresh: pauliusztin_'s tradeoff list says persistent wiki synchronization is still unsolved, and olvrgln's onboarding post argues agents re-onboard every session against whatever context is written down.
  • Docs are getting the same treatment: skeptrune said Mintlify's markdown plus llms.txt benchmark proved agents want Markdown, after stripped docs made Claude Code hallucinate Browserbase URLs.
  • The pattern is spreading beyond memory into software process: bcherny framed CLAUDE.md, REVIEW.md, skills, docs, and memories as domain knowledge encoded as infrastructure, and trq212 compressed the prompting style to thin prompts, thick artifacts and context, thin skills.

Google's Open Knowledge Format post calls OKF a portable LLM-wiki format: Markdown files with YAML frontmatter. The OKF spec is deliberately boring, a directory of files that humans can read, agents can parse, and Git can diff. LangChain's OpenWiki 0.2 announcement says its CLI generates repo wikis, connects them to coding agents, and keeps them updated as code changes. Mintlify's 2,400-run benchmark found that a single llms.txt link fixed most of the blind navigation problem created by serving stripped Markdown pages.

Wiki memory

Wiki memory treats the wiki as a working knowledge layer, not the underlying store. In pauliusztin_'s design, unified memory remains the source of truth, while the wiki becomes a local, task-specific layer the agent can reason over.

The architecture in that thread has four moving parts:

  1. Unified memory, optionally backed by a knowledge graph.
  2. An MCP server with tools for search, wiki build, source sync, and persistence.
  3. A Markdown wiki cache, with raw files and linked wiki pages.
  4. A harness skill that points the agent at the wiki instead of raw chunks.

The useful shift is blunt: the agent stops rediscovering the same facts from retrieved chunks and starts editing a durable map of what it already learned.

Paulius later added that wikis should stay a thin layer for finding the right slice of data, with full truth still resolved back into the original docs or chunks in a reply. Google's OKF formalizes the same shape as a directory of Markdown files with YAML frontmatter, and OpenWiki 0.2 adds that structure to generated codebase wikis.

Synchronization

Generating the wiki is now the easy part. Persistence and freshness are the messy bits.

pauliusztin_'s persistence note says his Tree system can build an interlinked Markdown wiki from GraphRAG and serve it through MCP, while Scrabble persists Markdown and YAML across Obsidian, Readwise, Notion, and Google Drive. The missing piece, in his telling, is caching the wiki by topic, letting it evolve, and serving it through MCP.

The tradeoffs from pauliusztin_'s list are clean enough to keep as a checklist:

Pros

  • No GraphRAG over the entire knowledge base.
  • Existing data can stay in Notion, Google Drive, CRMs, and similar systems.
  • Repeat queries get faster because the wiki compounds.
  • Markdown stays inspectable.

Cons

  • Persistent wiki synchronization is not solved.
  • The pattern fits recurring research better than one-off questions.

olvrgln put the same problem in codebase terms: a new engineer onboards once, while an agent onboards every session. Stale written context makes the agent fail confidently or gradually dirty the codebase.

Automation does not remove curation. Agents can be poor judges of what should be updated and why, olvrgln said in a reply, while cwolferesearch described a session-level Markdown scratchpad followed by a final pass that updates persistent Markdown and removes slop.

llms.txt maps

Mintlify's bug was wonderfully specific: serving Markdown made docs easier for agents to read, but stripping navigation made agents guess URLs. The fix was adding an llms.txt link at the top of the Markdown page, according to skeptrune.

The llms.txt spec proposes a Markdown file at /llms.txt with background, guidance, and links to detailed Markdown files. Mintlify's benchmark post says it tested four serving formats across 2,400 Claude Code and Codex runs, and found a single linked llms.txt did almost all of the work.

The docs pack idea is already turning into workflow muscle memory. nummanali asked for a programmatic indexer that grabs an llms.txt page and all linked Markdown files, while his follow-up said a local indexed docs pack is easier to grep, read, and share with subagents than repeated curl calls.

Mintlify's own traffic hints at the demand shift. skeptrune said its MCP page was outdrawing every docs page, with the attached table showing /docs/mcp far ahead of the main docs and llms.txt pages.

CLAUDE.md, skills, and memory directories

Claude Code's memory docs now describe two cross-session mechanisms: CLAUDE.md files that humans write, and auto memory notes Claude writes from corrections and preferences. Both load at the start of each conversation, and the docs explicitly say Claude treats them as context, not enforced configuration.

Nick Dobos mapped the evolution as four file-shaped stages in his diagram:

  1. CLAUDE.md, a single-file memory.
  2. memory_read(), memory_write(), and memory_edit(), memory as a tool.
  3. SKILL.md, procedural memory.
  4. memory/, files read and written by agents.

The design center is shifting from giant instruction prompts to maintained artifacts. trq212 summarized the target as thin prompts, thick artifacts plus context, and thin skills; his follow-up said skills are dynamically loaded, so the model sees only the name and firing condition by default.

bcherny argued that code comments, skills, CLAUDE.md rules, and memories now let teams encode domain knowledge that used to live in heads. GeoffreyHuntley's lookup-table example showed an AGENTS.md table mapping Lean libraries to matching skills before edits.

Agent teams

Memory files become more interesting when multiple agents have to work without sharing the same blind spots. omarsar0 described a Raft research team with a scout, critic, and synthesizer, each with a role and its own memory.

The mechanics were explicit:

  • The scout finds papers.
  • The critic challenges the claims.
  • The synthesizer writes the brief.
  • The critic runs on a different model from the scout, according to omarsar0's critic note.
  • Agents can write Markdown work reports, then revise after inline human comments, as shown in the report loop.
  • A scheduled daily run can kick off Scout, Critic, and Synthesizer, then post the brief back into the same channel per the reminder screenshot.

The same structure is showing up in coding loops. pauliusztin_'s Squid post described six specialized agents, Product Architect, Software Engineer, Tester, Pull Request Reviewer, On-call Engineer, and Self-improvement Agent. ADRs and a project glossary carry architectural decisions and shared terminology forward into future tasks.

A Reddit builder hit the opposite failure mode: the agent built the wrong thing cleanly because product and design thinking had not been decomposed before coding. the r/cursor post described a three-step planning handoff, product, design, engineering, where each implementation task carries only its own relevant requirements.

Markdown-first docs tooling

The tooling layer is catching up. haydenbleasel shipped Blume migration helpers that install with npx skills add haydenbleasel/blume --skill blume-migrate and target Mintlify, Fumadocs, Nextra, Starlight, and Docusaurus.

The linked [Blume page]Blume describes a Markdown-first docs framework with generated navigation, search, SEO, machine-readable agent output, Ask AI, and MCP access. haydenbleasel's follow-up said the repo hit 500 stars in 24 hours.

Agent-facing instructions are moving onto product surfaces too. PhotonHQ said it added agent instructions to its landing page, with a visible “Copy Agent Instruction” button in the attached screenshot.

Further reading

Discussion across the web

Where this story is being discussed, in original context.

On X· 7 threads
TL;DR4 posts
Wiki memory2 posts
Synchronization4 posts
llms.txt maps3 posts
CLAUDE.md, skills, and memory directories4 posts
Agent teams4 posts
Markdown-first docs tooling3 posts
Share on X