Local LLM vs API vs Rented GPU: Cost and Break-Even

Let $N$ be monthly input plus visible output tokens, and $f$ the input share. Generated tokens include the separately entered reasoning ratio. With measured prefill and decode rates $q_p,q_d$, active hours are $[Nf/q_p+N(1-f)(1+r)/q_d]/3600/u$. Use $u=1$ for end-to-end effective rates. Prefill and decode are treated as sequential contributions to aggregate GPU work.

Whole-system energy is $E=P_a t_a+P_i(t_{on}-t_a)$, with watts converted to kW. The idle interval is clamped at zero only to keep diagnostics readable above capacity; that route is unavailable there. The energy method follows Virginia Tech BSE-301NP, page 1. Its PC example rounds 394.2 kWh to 394 kWh.

Own monthly cost is $(purchase-resale)/months+E\,tariff+upkeep\,rate+other$. Inside capacity it can also be written $F+vN/10^6$: $F$ includes the all-idle baseline, and $v$ uses active minus idle power. Rental bills sequential prefill and decode GPU hours divided by productive utilization, plus fixed fees and upkeep. Pan et al., arXiv:2509.18101v3, sections V–VI, provides the capital/operating-cost framework; this tool derives its own workload-specific equations and does not import that paper's quality rankings or payback estimates.

API input cost uses mutually exclusive ordinary, cache-write and cache-read fractions. The selected write rate replaces ordinary input billing. Multiply the cache-weighted input price and billed output price by the request-length tier, exchange rate and $1-b(1-d)$, where $b$ is batch share and $d$ its price multiplier. Hidden reasoning is charged at output rates. A typical request over a tier threshold prices its entire input and output at that tier.

Monthly curves hold token mix, speeds, cache fractions and context tier fixed. Pairwise crossings solve $F_a+v_aN=F_b+v_bN$ and are retained only at positive demand within both routes' capacities. They are equality points: inspect the curves to see which route wins on each side. No positive feasible crossing also covers parallel lines or a crossing outside capacity. Markers outside the chart require a larger chart maximum.

Cash payback is $T=purchase/(C_{alternative,cash}-O_{own,cash})$, using actual cash outlays. Upkeep cash equals upkeep hours times hourly rate times paid share. The remaining unpaid time stays in all-in costs but is excluded from cash spending. The displayed payback is unavailable without positive cash savings and valid capacity/request limits. All-in monthly ownership amortizes purchase minus resale. Cash curves use the full purchase upfront and no resale; the horizon totals credit resale at the end. Input/output GPU prices allocate every fixed cost in proportion to active prefill/decode hours; their workload-weighted sum equals the blended cost. These allocations are not two independent invoices. API input/output rates exclude non-token monthly fees; the blended API cost includes those fees.

API request validation infers visible output per request as $n_p(1-f)/f$ for a positive input share, then multiplies by $1+r_{API}$ for billed generation. This generated count must not exceed the model's standard output cap, and input plus generated tokens must fit its context window. An output-only workload cannot define finite requests with the entered positive input length, so API is excluded. Nonzero effective cache fractions require input length at least the row's cache minimum. This is a necessary length check; actual prefix eligibility and hits still come from billing logs. Anthropic cache limitations, model specifications and context-window rules are linked below.

Prices are editable snapshots, checked 2026-10-27. API rows use first-party global list pricing; OpenAI GPT-4.1 is a dated 2025-04-14 schedule checked against its official model documentation. Runpod's page is dated 2026-10-08. Presets may become unavailable or change.

The default rig purchase (2400 CU), resale (600 CU), 450 W active, 65 W idle, tariff and upkeep are illustrative planning assumptions, not measured hardware specifications. Default prefill/decode rates are the RTX 3090 Llama 2 7B Q4_0 CUDA no-Flash-Attention measurement posted 2025-08-01 by m18coppola, commit c76b420, in llama.cpp discussion 15013. No power measurement is paired with those rates here.

Apostolov's June 2026 metered RTX 3090 article motivates measuring watts and delivered throughput. Its GPU-only energy boundary excludes the rest of the computer; use wall measurements here, or explicitly treat your inputs as a GPU-only scenario.

One owned rig and one rented GPU instance are considered. Usage is averaged across calendar days; input/output token counts are assumed comparable for your chosen tasks. Throughput and productive fractions are fixed. API and local cache behavior are independent: API cache reads still contribute to local prefill unless your measured local rates already reflect reuse. Local and rental use the same reasoning ratio.

The same cache and output mix is assumed for batched and standard calls. Retention is a policy descriptor, not a predicted hit probability. OpenAI short retention is indicative (typically 5–10 minutes idle); its extended option is up to 24 hours where supported. Claude short/extended retention uses 5/60-minute pricing. Use billing logs for cache fractions and output reasoning. CU means the selected currency. No live exchange or price feed is used.

Both paid upkeep shares default to zero: the entered time is valued for the all-in comparison but is unpaid. Set a paid share to one when all that upkeep is paid at the entered hourly rate, or enter an intermediate fraction. The same hourly rate values and pays the selected hours. Other fees and API charges are treated as cash expenses.

API presets use standard output caps, including reasoning. Optional Anthropic 300K-output batch beta is excluded unless you deliberately edit the row to match an enabled service. Claude cache minima are 512 input tokens for the included 5.5 models. GPT-4.1 uses a 1024-token baseline from OpenAI's legacy cache guide; current documentation notes that earlier-model minima can depend on request settings. Verify your request's effective minimum and increase the editable row value when needed.

Use for steady inference cost planning after checking that the chosen model and KV cache fit the hardware. Separate prefill/decode benchmark rates are an approximation; replace them when long context, concurrent batching, CPU offload or an end-to-end workload changes speed. Capacity is average time capacity, not a peak-latency or memory guarantee.

API eligibility checks the inferred request's output cap, combined context including reasoning, and minimum prompt length for nonzero cache fractions. These are deterministic checks for the one typical request mix entered here, not a guarantee of API rate limits, cache hits or peak performance. Verify exact reusable prefixes, any higher request-specific cache minimum, service availability and provider-specific discounts. All API requests are assumed to have the entered typical input length; mixed context tiers should be costed separately. Regional premiums, tax, financing, repairs and replacement purchases are included only if you put them in your inputs. The horizon assumes no replacement; payback beyond it is an extrapolation, not a promise. Quality, privacy value, training, multimodal/tool workloads and subscription token caps are not evaluated.

Charts and cost cards exclude an unavailable route from the cheapest-route choice. Numeric allocation and horizon tables remain hypothetical for rows whose status is unavailable. Cash curves count paid upkeep only and exclude resale. All-in horizon totals include paid and unpaid upkeep and credit resale at the end. There is no discounted cash-flow or embodied-energy model.