The economics

Every query is the cheap part.

RAG workloads spend your fleet's GPU time re-reading the same documents on every question. Engram Smart CAG onboards documents once and serves every answer from the saved memory. Same model, same answers, measured head-to-head; the difference is the waste removed from your GPU spend.

See the work you'd save ↓

Measured head-to-head vs standard RAG

1,000 docs · 500 questions · same model & retrieval

Answer qualityIdentical — Measured by an independent judge

Reading work per answer91× less

Time to first token10× faster

Total time per answer1.3× faster

One deliberately hard benchmark, identical on both sides: expert-written questions about scientific papers, the right paper hidden among 1,000, graded strictly against expert answers. Same answers, a fraction of the work.

Fleet scale

Capacity opens up as load climbs.

A real concurrency ramp on one 4-GPU serving tier — same model, same retrieval on both sides. Resident memories skip the re-read, so throughput keeps climbing while RAG flattens under its own re-prefill.

Engram Smart CAG Standard RAG (churned cache) freed capacity Hover any load point for throughput, TTFT and GPU $/1k.

Measured on one 4-GPU serving tier (Qwen3-30B FP8, TP=4), server-timed time-to-first-token and latency, GPU priced at $4.60/hr, 1–128 concurrent queries. RAG runs in the multi-tenant regime — a churned prefix cache, the realistic fleet operating point — so it pays its re-prefill on every request while cartridge memories stay resident.

Payoff calculator

Your workload, your volume.

Model your Smart CAG vs standard RAG spend for any model, instance and volume.

Your workload

Model

Instance

Concurrent queries / GPU node

Documents

Queries / month

Tokens / document

word count × 0.75

RAG tokens / query

context re-processed per question

Smart CAG vs standard RAG

Per selected model, instance and concurrency. Δ is green in the better direction.

Metric	Smart CAG	RAG	Δ

One-time onboarding (spot)

—

read + compress every document, once

Monthly memory storage

—

compressed KV + S3 Standard

Bottom line

—

Monthly GPU savings vs RAG

—

Savings %

—

Payback on onboarding

How to use

Model: Dropdown — the model you are currently running.
Instance: Dropdown — the GPU hardware you are currently running on.
Number of documents: Free input — documents in the document base.
Queries per month: Free input — queries per month currently invoking RAG.
Concurrent queries / GPU node: Dropdown — concurrent queries on the GPU; this depends how you are balancing load. If you do not know, leave it as is.
Tokens per document: Free input — average tokens per document (drives onboarding + cartridge storage). Token count = word count × 0.75.
RAG tokens re-processed per query: Free input — average context tokens RAG re-reads per query (drives RAG cost, latency and savings). If you do not know, leave it as is.

Fixed cost, scaling savings

Storage is flat. The savings scale with every question.

Saved memories are a flat monthly line item — our 4× memory compression plus automatic cloud tiering keeps storage costs low and constant while the GPU savings grow with volume: every question asked is savings increase. The busier the library, the further the savings pull ahead of the flat storage line — the calculator above nets the two for your numbers.

How it stacks up

Engram Smart CAG vs. standard RAG.

Same model, same grounding, same answers — the difference is doing the reading once instead of on every query.

	Engram Smart CAG	Standard RAG
GPU work per query	Dozens of tokens, not thousands	Re-reads context every query
Latency (time-to-first-token)	Low & stable — 28 ms, ~90 ms even at 24 concurrent	Grows with context
Grounded & sourced	Yes — quality tie with RAG	Yes
Setup / onboarding	One forward pass per doc — no training	Index once
Fresh on updates	Re-onboard the doc once (seconds)	Instant re-index
Storage footprint	Large — the trade-off: 4× compressed and tiered, measured	Small index
Private on your cloud	Yes — your VPC, your GPUs	Depends

Get in touch