Skip to content
AI Primer
workflow

Perplexity Projects commits agent work to S3-backed Git repositories

Perplexity Projects agents commit work from ephemeral sandboxes to S3-backed Git repositories. Short-lived locks and head checks serialize updates, while real checkouts support grep and standard file tools.

5 min read
Perplexity Projects commits agent work to S3-backed Git repositories
Perplexity Projects commits agent work to S3-backed Git repositories

TL;DR

Perplexity's FUSE prototype ran roughly 400 times slower than local files on filesystem-heavy workloads, a brutal result for a fancier abstraction. Attaching to a warm checkout takes about 2.9 seconds, versus roughly 8 seconds for a cold start.

Five-component forge

Projects is the shared workspace for long-running tasks in Perplexity Computer. Perplexity's architecture has five components:

  1. CLI: Maintains each sandbox's local checkout, synchronizes updates, and submits changes.
  2. Control plane: Locates repositories and issues short-lived credentials. It never reads or writes repository data.
  3. Git service: Transfers repository data and decides which submission becomes the accepted head.
  4. S3-backed storage: Keeps repository data durable independently of compute.
  5. Downstream processing: Extracts text and metadata from saved documents for subsequent checkout and sync.

An agent proxy checks sandbox access before requests reach the Git service.

S3Files and ephemeral sandboxes

Perplexity stores bare Git repositories on AWS S3Files, with objects and references rather than working checkouts. Stock Git runs against that filesystem without modification; AWS documents S3 Files as an EFS-built shared filesystem exposing S3 data through file and directory operations.

Storage is partitioned into isolated cells, each pairing a shared filesystem with Git service replicas. Any replica in a cell can serve its repositories, although requests normally return to the same replica for cache locality.

Replacing or adding replicas requires no repository migration between servers. A sandbox crash puts only unsubmitted work at risk.

Submit locks and head checks

Concurrent writers follow a short acceptance sequence:

  1. The CLI holds a short-lived lock to serialize submits.
  2. The Git service accepts a submit only if it builds on the current head.
  3. A stale submit receives a structured response; the agent replays its change on the new head and resubmits.

Production workflows reject fewer than 2% of submits as stale. Perplexity reports median submit latency below 1.7 seconds and p99 below 8 seconds.

Repository contracts and compaction

The Git service checks repository invariants instead of requiring a human to review each change.

Mechanical checks limit repository growth, including keeping large and binary files in Git LFS rather than raw Git blobs.

After a successful submit, the CLI asks whether compaction is needed. When retained commits exceed a threshold, the service:

  • Replaces older history with a single starting snapshot.
  • Retains a smaller set of recent commits and replays them.
  • Rewrites those commit identifiers while preserving the files at the new head.

Local checkouts and lazy downloads

Agents run in SPACE, Perplexity's in-house sandbox platform, and explore real files with grep and standard file tools, as Perplexity explains. The CLI materializes files lazily:

  • Large and binary files initially arrive as LFS pointers carrying the source object's size.
  • A manifest identifies documents with extracted text, which agents read as ordinary files.
  • Original documents are downloaded when text is insufficient, such as for figures, layout, or embedded data.

Perplexity's FUSE implementation required a network round trip for every filesystem request and privileged capabilities from the host, which would have meant privileged agent sandboxes.

Parsed documents outside Git

Each accepted write generates an outbox entry, which downstream parser workers consume:

  1. Workers derive changed files from commit metadata and fetch raw bytes using the content hash.
  2. They extract text and metadata into a separate store keyed by the source document's hash.
  3. Checkout and sync materialize those artifacts in a reserved directory managed by the CLI.

Parsing runs asynchronously and does not block checkouts. Parsed artifacts never become repository commits, avoiding extra concurrent writes and document-derived noise in history.

File exploration benchmark

Perplexity's internal comparison tested an exploration agent against semantic search over an index, using 150 single-document financial questions over the same corpus:

  • Correct-document retrieval: About 39% → 86.7%, approximately +47.7 percentage points.
  • Correct answers: About 37% → 62.7%, approximately +25.7 percentage points.
  • Latency: About 25% lower with exploration.

Search delegates project exploration to a separate agent with a limited budget, which returns file paths and supporting excerpts. The answering model then reads the selected files; the exploration agent mostly uses extracted text and rarely downloads original PDFs.

Brain memory repositories

Perplexity Brain, Computer's knowledge wiki, stores memory in separate knowledge repositories with their own history and policies. Background memory updates therefore do not compete with project-file edits; a session needing both loads both repositories.

Background agents incorporate new information into wiki pages and save updates as commits. People can edit those pages too, with commit history recording how the knowledge evolved.

Share on X