The Token Math Nobody Runs Before Choosing Self-Hosted AI

A Quiet Warning From Tor's Exit Nodes

Security researchers tracking Tor's network recently flagged 333 malicious exit nodes that were actively intercepting SSH traffic and pulling plaintext email credentials from users who assumed their connection was private. Tor routes your traffic through volunteer-run relays to anonymize it, and the exit node is the last stop before your data reaches the open internet. If that final relay is compromised, anything leaving it unencrypted is exposed to whoever is running it. Email, by default, often travels that way.

Why does this matter if you're not a privacy researcher or a journalist dodging surveillance? Because the same basic math applies to any business routing sensitive data through infrastructure you don't control and can't audit. Every time you send a customer list, a draft contract, or internal strategy notes through a third-party API, you're trusting that provider's security practices, their logging policies, and their incentive structure to stay aligned with yours.

That trust isn't free, either. It comes with a literal line item on your invoice every month, and most businesses never sit down and calculate whether that line item still makes sense once volume climbs. The Tor story is a security lesson. The cost side of that same decision gets ignored just as often.

What An API Bill Actually Measures

Most API pricing runs on tokens, not words, and that distinction trips up almost everyone running the math for the first time. A token is roughly four characters of English text, so a thousand-word prompt is closer to 1,300 tokens once you account for punctuation, spacing, and whatever formatting you pasted in. Providers charge separately for input tokens (what you send) and output tokens (what the model generates back), and output almost always costs more per token than input. That asymmetry matters because the tasks businesses actually want AI for—drafting, summarizing, expanding—are output-heavy by nature.

Context length compounds this fast. Every message in a conversation gets re-sent to the model along with the new one, because these systems don't have persistent memory between calls. A ten-turn conversation with a long system prompt and reference documents attached doesn't cost ten times a single message. It costs something closer to the sum of every prior turn, resent again and again, because the model has to re-read the whole conversation each time to generate the next reply.

Teams notice the invoice before they understand the mechanism. By the time someone asks why the bill tripled, the usage pattern that caused it has already been running in production for months.

Where The Breakeven Point Sits

Run the numbers before you run the model. A business generating heavy daily volume—say, a few million tokens a month across drafting, summarizing, and customer replies—can watch that monthly API bill climb well past the cost of a single consumer GPU within a year, sometimes faster. A 7B or 13B parameter model running locally through Ollama handles drafting and summarization tasks reasonably well on hardware you already might own, or on a rented cloud GPU instance billed hourly instead of per token. The breakeven isn't a fixed number because your usage pattern sets it. Low-volume, occasional use almost never justifies the upfront cost or the setup time. High-volume, repetitive tasks—support ticket triage, internal documentation, first-draft content—tip the math toward self-hosting fast, because you're no longer paying per token for work that's structurally similar every time.

BYOK setups complicate this further. When a platform lets you bring your own API key instead of marking up usage through their infrastructure, you're back to paying the provider's raw token rate, which changes whether self-hosting even clears the bar. A business already on BYOK pricing needs a higher volume threshold to justify the switch than one paying a platform's bundled rate, because the comparison point itself just moved closer to the provider's actual cost.

Deciding If Self-Hosting Fits Your Workload

Before you touch Ollama or price out a GPU, pull your actual usage numbers. Log into whatever dashboard your current provider gives you and look at monthly token volume for the last three months, not one. One busy month tells you nothing. A consistent pattern across a quarter tells you whether you're a candidate at all.

Then look at what's actually in those tokens. Are you sending customer names, contract terms, health information, anything that creates liability if it sits on a server you don't control? Data sensitivity can justify self-hosting even at volume that wouldn't otherwise clear the cost bar. A law firm drafting from client files has a different calculation than a solo creator drafting social captions, even if their token counts look similar on paper.

Technical capacity is the third filter, and it's where most small operators stall out. Running a local model through Ollama isn't plug-and-play. Someone needs to manage the hardware or the rented GPU instance, handle model updates, and troubleshoot when output quality drifts. If that's not a skill already in-house, factor in the hiring or contracting cost alongside the hardware cost.

If your volume is low, your data isn't sensitive, and nobody on staff wants to own infrastructure, staying on an API is the boring, correct answer this year.