On this page

LLM context compression on your machine

Document type
Field manual
Updated
Reading time
9 min

Headroom compresses LLM context locally by 60–95%. Set up its library, proxy or MCP server, review five compressors, output shaping, benchmarks and limits.

SourceOpen original report
Document details
headroomlabs-ai/headroom

Headroom compresses tool output, logs, files and RAG excerpts locally before sending them to an LLM. Tests on code search, SRE debugging and other tasks reduced tokens by 60–95% without changing answers. It supports Python / TypeScript libraries, an HTTP proxy requiring no code changes and an MCP server; this manual covers setup, five compressors, integration interfaces, a worked example and limits.

GitHub Stars
46.7k
Token reduction
60–95%
Integration methods
3
Open-source license
Apache-2.0

01Tool overview

Local compression of LLM input

Headroom addresses a specific cost issue: AI agents process large, verbose contexts containing command output, API responses, logs and retrieved documents. Every token costs money, and output tokens on pricier models cost roughly five times as much as input tokens. Headroom compresses this content on your machine before it reaches the LLM.

It is a Python project under Apache-2.0, published as headroom-ai, with three integration methods: libraries for Python and TypeScript, a proxy providing an OpenAI-compatible HTTP interface without code changes, and an MCP server exposing compression as agent tools. All three share the same compression engines; choose according to the cost of changing your project.

Specialized compressors divide the work: SmartCrusher compresses JSON; CodeCompressor provides AST-aware code compression; Kompress-base is a text compression model trained on agentic traces; CacheAligner stabilizes prefixes to improve KV-cache hits; CCR provides reversible compression with originals cached locally for retrieval on demand. The official description states that compressing tool output, logs, files and RAG excerpts reduces tokens by 60–95% without changing answers.

  1. Tool output / Logs / RAG

  2. Local compressors

  3. compress()

  4. LLM

  5. Retrieve originals when needed

Compress tool output, logs, files and RAG excerpts before they reach the LLM. Use 60–95% fewer tokens with unchanged answers.

— Headroom README,Project purpose · Translated from Chinese

02Installation and integration

Library, proxy and MCP integration

Requires Python 3.10+. Choose features with installation extras; [all] is common. The most direct approach uses the library: import compress and compress messages before sending them to the model.

bash
# Python library(install all extras)
pip install "headroom-ai[all]"

# Compress messages in code before sending them to the model
from headroom import compress
compressed = compress(messages, model="...")

# TypeScript / Node also has a library
npm install headroom-ai
await compress(messages, { model })

Proxy without code changes

Start the proxy if you prefer not to change application code. It is an OpenAI-compatible drop-in HTTP proxy; route traffic through it. The MCP server installs with one command and exposes three tools: headroom_compress, headroom_retrieve and headroom_stats.

bash
# HTTP proxy requiring no code changes(OpenAI compatible)
headroom proxy --port 8787

# Install MCP server,expose three compression tools
headroom mcp install

03Five compressors + Three integration methods

Compressors and integration interfaces

Specialized compressors handle structured JSON, code, natural-language text and KV-cache prefixes. The library, proxy and MCP server connect the same engines to different layers of your system. Capabilities include five compression mechanisms and three integration points.

Compressor · 01

SmartCrusher

Compress JSON

Compress structured JSON, including large API responses and tool outputs.

Compressor · 02

CodeCompressor

AST-aware code compression

AST-based compression for six languages, Python, JavaScript, Go, Rust, Java and C++.

Compressor · 03

Kompress-base

Text compression model

A HuggingFace model trained on agentic traces to compress natural-language text.

Compressor · 04

CacheAligner

Stabilize KV-cache

Stabilize prefixes to improve KV-cache hit rates. More hits mean less recomputation, reducing both latency and cost.

Compressor · 05

CCR

Reversible compression

Reversible compression with originals cached locally, retrieval on demand and configurable TTL. Restore full content when needed.

Entry · 06

Library

Library integration

Python compress(messages) or TypeScript await compress(messages, { model }) gives explicit control of compression points in code.

Entry · 07

Proxy

No-code-change HTTP proxy

headroom proxy --port 8787 starts an OpenAI-compatible drop-in proxy. Route traffic through it without changing application code.

Entry · 08

MCP server

Agent tools

headroom mcp install exposes headroom_compress, headroom_retrieve and headroom_stats to agents.

Comparison of three integration methods

MethodCommand / InterfaceCode changesSuitable use
Librarycompress(messages, model=…)Yes (explicit calls)Precise control of compression points
Proxyheadroom proxy --port 8787NoKeep existing code
MCP serverheadroom mcp installNo (tool calls)Agent-controlled compression / retrieval

04Usage · Official documentation

Eight usage points

These points come from the official Headroom README and documentation site. Choose an integration method based on implementation cost, retrieve originals when needed and measure reductions on your own workloads; do not rely only on the headline 60–95% range.

Use the proxy to keep existing code

The library requires calling compress() in code. To keep existing code, start an OpenAI-compatible proxy with headroom proxy --port 8787 and route traffic through it.

Source · Official README

Retrieve originals with CCR

CCR provides reversible compression with local caching, retrieval on demand and configurable TTL. Retrieve full content for replay or audits when needed; otherwise retain the compressed form.

Source · Official README · CCR

Output shaper reduces model output

Set HEADROOM_OUTPUT_SHAPER=1 before starting the proxy to reduce model output through verbosity steering and effort routing as well as compressing input. Output tokens on pricier models cost roughly five times as much as input tokens.

Source · Official README · output shaper

headroom learn derives settings from failed sessions

Preview with headroom learn --verbosity; add --apply to save. It examines failed sessions and writes corrections into each agent's own markdown file.

Source · Official README · learn

Install only required extras

Instead of [all] every time, choose [proxy], [mcp], [code], [ml], [langchain] and other extras as needed for a smaller installation with fewer dependencies.

Source · Official README · extras

CodeCompressor covers six languages

AST-aware code compression supports Python, JavaScript, Go, Rust, Java and C++, reducing code context sent to the LLM.

Source · Official README · CodeCompressor

Add --memory --code-graph when wrapping claude

Headroom wraps multiple coding agents. Claude Code has dedicated options, headroom wrap claude --memory --code-graph, adding cross-session memory and a code graph.

Source · Official README · agent wrapping

Benchmarks require workload verification

Official benchmarks show accuracy is broadly retained, GSM8K math 0.870, TruthfulQA 0.560 and SQuAD v2 at 97% with 19% compression. Measure on your own workloads as well.

Source · Official README · benchmarks

05Worked example

Proxy and token reduction verification

Start the proxy, compress code-search results once through an MCP tool, inspect actual reductions in stats and then reduce model output with output shaper. The official benchmark reports 100 code-search results compressed from 17,765 to 1,408 tokens, saving 92%.

~/work · headroom proxy · MCP


# 1 · Install(all extras)
$ pip install "headroom-ai[all]"


# 2 · Start an OpenAI compatible proxy requiring no code changes
$ headroom proxy --port 8787
  ok: Headroom proxy listening on http://127.0.0.1:8787
  # compressors: SmartCrusher · CodeCompressor · Kompress-base · CacheAligner · CCR


# 3 · Install MCP server,expose three tools to the agent
$ headroom mcp install
  # tools: headroom_compress · headroom_retrieve · headroom_stats


# 4 · The agent compresses one set of code search results
  # → headroom_compress { input: code_search(100 results) }
    hl: 17,765 → 1,408 tokens  ·  saved 92%


# 5 · Check actual reduction statistics
  # → headroom_stats {}
    # SRE incident debugging   65,694 → 5,118   (92%)
    # GitHub issue triage      54,174 → 14,761  (73%)


# 6 · Retrieve the original when needed(CCR can restore it)
  # → headroom_retrieve { id: ... }
    ok: restored original from local cache


# 7 · Reduce model output too:enable output shaper
$ export HEADROOM_OUTPUT_SHAPER=1
$ headroom proxy --port 8787


# 8 · Measure estimated output reduction(including confidence intervals)
$ headroom output-savings
    ok: estimated output reduction reported with confidence intervals

Code search with 100 results: 17,765 → 1,408 tokens, saving 92%. Accuracy is broadly retained.

— Headroom official benchmark(real workloads) · Translated from Chinese

Type-aware compression and retrieval

Token reductions come from type-aware local compression, without directly discarding information: JSON uses SmartCrusher, code uses CodeCompressor, text uses Kompress-base and prefixes use CacheAligner. CCR retrieves full originals from local cache when needed. Both compression and retrieval happen on your machine, keeping data flow controlled.

Output shaper changes prompt and effort. It reduces output tokens through verbosity steering and effort routing, modifying the system prompt and amount of thinking. Before production use, measure quality and savings on your own workloads to decide whether to keep it enabled.

06Boundaries and limitations

Compression limits

07Next steps

Headroom advanced integration

After basic compression works, extend integration with output reduction, settings learned from failures, framework interfaces and wrappers for your usual coding agents.

Advanced settings

1. Enable output shaper by starting the proxy after export HEADROOM_OUTPUT_SHAPER=1. Verbosity steering adds concise instructions to the system prompt; effort routing reduces thinking on routine steps such as processing tool results. Use headroom output-savings to measure estimated savings.

2. Adjust settings with the learn loop. Preview with headroom learn --verbosity and add --apply to save the best settings. It examines failed sessions and writes corrections into agent-specific markdown files.

3. Connect your framework. Interfaces include Anthropic SDK withHeadroom(new Anthropic()), OpenAI SDK withHeadroom(new OpenAI()), LangChain HeadroomChatModel(your_llm), Vercel AI SDK wrapLanguageModel() middleware and LiteLLM HeadroomCallback().

4. Wrap coding agents. Headroom can wrap Claude Code, Codex, Cursor, Aider, Copilot CLI, OpenClaw and Cortex Code. Claude Code has dedicated options, headroom wrap claude --memory --code-graph, for cross-session memory and a code graph.

5. Control installation with the extras matrix. Choose [proxy], [mcp], [ml], [code], [memory], [relevance], [langchain] and others as needed. On Apple Silicon, set HEADROOM_EMBEDDER_RUNTIME=pytorch_mps to run embeddings on the GPU.

Official resources

① Official documentation: Full installation, compressors, integration interfaces and benchmark documentation. ② github.com/headroomlabs-ai/headroom: Source, README, issues and change history.

Compress tool output, logs, files and RAG excerpts before they reach the LLM. Choose Library, proxy or MCP server.

— Headroom · headroomlabs-ai,Project README · Translated from Chinese