Calculator
edit any field, results updateThe throughput default is not a spec-sheet number. It is our measured aggregate decode rate for an 8B dense model served with FP8 weights at concurrency 32 on one RTX PRO 6000 Blackwell, from the published sample report. Your model and hardware will differ; that is the point of measuring.
Method
every formula, in the openAPI monthly cost is volume times price: input millions times the input rate plus output millions times the output rate.
Self-hosted monthly cost is provisioned hours times the hourly rate: GPUs times USD per GPU-hour times 730 hours. It does not change with volume until you saturate the box.
Capacity is what your measured decode throughput can produce in a month at your utilization: tokens per second times 3600 times 730 times utilization. If your output volume exceeds capacity, the calculator scales the GPU count up for you and says so.
Break-even is the output volume at which the API bill equals the fixed self-hosted bill, holding your input volume constant. Above it, the GPUs win; below it, the API wins. Everything here ignores engineering time, which is real and belongs in your decision.
Measured, not assumed
the variable that moves the answerEvery input above is public knowledge except one: the throughput your model actually sustains on your hardware. Spec sheets do not answer it. Quantization format, batch shape, KV cache settings and scheduler behavior move that number by 2x or more, which moves every cost in this calculator by the same factor.
We measure it as a fixed-scope service: your model, your target GPU, throughput and latency percentiles across concurrency levels, plus a per-prompt quality regression check when quantization is involved. The deliverable format is public: sample report.
FAQ
short and plainWhat throughput should I assume for my model?
If you cannot measure yet, scale roughly with parameter count: a 70B dense model produces on the order of one tenth the tokens per second of an 8B on the same card. Mixture models, long prompts and strict latency targets all break that rule of thumb, in both directions.
Why is utilization in the formula?
A provisioned GPU bills for 730 hours a month whether traffic arrives or not. Real services see peaks and idle troughs, so the cost per useful token is the hourly bill divided by the tokens you actually served. 50 to 70 percent is a common planning band for steady products; spiky workloads run lower.
Does this include engineering time?
No. Self-hosting carries setup and operations work that per token math does not capture. If the dollar gap between the two columns is small, the API is usually the right answer on that ground alone.