The True AI Infrastructure Cost

Developers over-index on the API token sticker and underestimate the hidden multipliers: output tokens billed at 3–5× input, agent loops multiplying consumption 10–100×, and the vector-database and storage layers underneath. Stack all four layers and see the bill finance actually receives. Model defaults are editable — verify against the live ledger.

Layer 1 — LLM Inference

Conversations / month

Input tokens / conversation

Output tokens / conversation

Model preset (fills editable prices)

Input $ / 1M tokens

Output $ / 1M tokens

Agentic loop multiplier — tool calls, retries, chains re-consume the context every step

12×

Prompt-cache hit rate (cache-read $ /1M)

40%

Layers 2–4 — the stack underneath

Layer 2 · Dedicated GPU compute Layer 3 · Vector database (RAG)

Vectors (millions)

Dimensions

Vector storage $ / GB / mo

Layer 4 · Object storage & logs

Storage GB

$ / GB / mo

The actual monthly bill

L1 Inference $0 L2 GPU $0 L3 Vector DB $0 L4 Storage $0

Ready to compare vendors on these exact primitives? Open the AI directory → or test the self-hosting tipping point →