intro handled in meta - body only

In a development agent, a large share of the prompt repeats turn after turn: system instructions, tool descriptions, project rules, shared memory. Without prompt caching, that prefix is resent and rebilled on every request. With caching, the provider mostly bills the delta: new message, new tool outputs, growing history tip.

Navin enables prompt caching for providers that support it (managed routes, Anthropic, and compatible gateways). It pairs naturally with shell output compaction.

Definition

The provider remembers a stable prefix of the request. Later turns that start with the same prefix cost less (and are often faster) on the cached portion.

This is not a cache of ready-made answers. It is a cache of input context.

Prompt caching vs response caching

Navin relies on prompt caching. HTTP response caching is not used for the agent: after a file read or a test, the next request is almost never byte-identical.

What gets cached

Depending on the provider, Navin marks eligible blocks (cache_control or the declared equivalent):

• system instructions and policies;
• tool descriptions;
• stable project context injected every turn.

Usually outside cache or at the prompt tip:

• the latest user message;
• the latest tool results (why compacting them matters);
• growing history.

Why it matters in an agent IDE

A light chatbot has a short system prompt. An IDE like Navin regularly sends:

• a large tool catalog (Code, git, board, Vision 360…);
• modes and workflows (Agent, Plan, Review…);
• project memory (Project Brain).

Without caching, every "continue" republishes that prefix. With caching, the bill mostly tracks new work in the turn.

When the gain peaks

• Long sessions, many turns, same model and tool set.
• Stable system prompt and tools during the session.
• Weaker if you change preset or model on every message.

Caching handles the prefix; compaction handles tool outputs. Together they fit shell-heavy sessions.

Checking that it is active

1. Pick a model / provider that advertises prompt caching.
2. Run several turns without changing model or tool set.
3. Check provider usage (cached tokens / cache hit per dashboard).

On Free, caching does not cancel rate limits; it mainly reduces cost when token billing applies. No "enable cache" toggle is required in Navin for supported providers: it is the normal call path.

Common pitfalls

• Confusing response caching with prompt caching.
• Expecting a large Free cut when the limit is request-based.
• Switching models too often and invalidating the prefix.
• Letting huge shell logs through: even with a cached prefix, the tool suffix stays expensive.

FAQ

Is prompt caching already available in Navin?

Yes, for providers that declare it.

Do I need a special header?

Not for normal use. Navin applies cache marks per provider.

Does it work with free models?

When the Free provider exposes caching, yes. Otherwise the benefit is mainly on paid models and managed budgets.

Does it replace command compaction?

No. Caching = stable prefix. Compaction = variable outputs.

Can I force an HTTP response cache?

That is not the mechanism Navin uses for the agent.

Next steps

1. Download Navin and work in a continuous session on one model.
2. Let prompt caching amortize the prefix.
3. Rely on exec compaction for shell outputs.

Download Navin · Features · Pricing