If you’re building an AI product as a solo developer, one of the first questions you’ll face is: where do I run this thing?
Serverless sounds great on paper — pay per request, no server management, auto-scales to zero. But when you actually look at the bill at the end of the month, the picture gets murky. I’ve been running several AI services on different architectures over the past year, and here’s the breakdown I wish someone had given me before I started.
## The Three Tiers of AI App Hosting
For indie developers building AI products, there are really three tiers of hosting to consider:
**1. Serverless (Cloudflare Workers / AWS Lambda / Vercel)**
**2. VPS / Dedicated (DigitalOcean / Hetzner / AWS EC2)**
**3. GPU instances (RunPod / Lambda Labs / AWS SageMaker)**
Most AI apps need a combination of compute for inference (which needs GPU) and compute for the API backend (which usually doesn’t). The mistake is treating them as the same problem.
## The Serverless Trap: It’s Cheap Until It’s Not
Cloudflare Workers is $5/month for 1 million requests. AWS Lambda Free Tier gives you 1 million requests and 400,000 GB-seconds per month. For a prototype or low-traffic app, this is basically free.
But here’s what happens when you grow. Say you’re running an AI API proxy — your function calls an LLM provider, does some processing, returns a response. Each request might take 2-3 seconds on Lambda (including the network call to the LLM). At $0.0000166667 per GB-second, a single request costs about $0.00005. That’s $50 for a million requests. Not bad, but add API Gateway ($3.50/million) and data transfer, and you’re looking at $80-100/month for a million requests.
Compare this to a $12/month Hetzner VPS running a simple FastAPI server. The VPS handles unlimited requests (within its capacity — maybe 50-100 concurrent). The break-even point is around 100,000-200,000 requests per month. Above that, VPS is significantly cheaper.
Data from Cloudflare’s 2024 developer survey showed that 47% of Workers users reported costs under $5/month, but for those handling over 10 million requests monthly, the median cost jumped to $200+. The long tail of serverless costs is real (source: Cloudflare Blog, Serverless Cost Analysis, 2024).
## Where Serverless Actually Wins
Serverless shines in three scenarios:
**1. Spiky traffic.** If your app gets 1 request per hour for 23 hours, then 10,000 in one hour, serverless handles that gracefully. A VPS needs to be provisioned for peak load.
**2. Global distribution.** Cloudflare Workers run in 300+ locations. If your users are spread across continents, serverless can deliver sub-50ms cold-start responses (Workers specifically — Lambda cold starts can be 1-3 seconds).
**3. Prototyping and zero-traffic apps.** When you don’t know if anyone will use your app, $0/month for idle is unbeatable. I’ve launched side projects that got zero users — serverless cost me nothing.
## The VPS Sweet Spot
Once you have consistent traffic, VPS becomes the obvious choice. A Hetzner CX22 (2 vCPU, 4GB RAM) at €3.99/month can handle a surprising amount of traffic if you optimize well.
A typical setup: Hetzner CX32 (€7.99/month) running Docker with 6 containers — Nginx reverse proxy, FastAPI backend, Redis, PostgreSQL, Prometheus, and a cron service. Total CPU usage rarely exceeds 30%. For an AI API proxy service handling about 50,000 requests/month, this costs less than Cloudflare Workers would.
The key insight: the VPS is cheaper not because it’s better technology, but because you’re pre-paying for capacity rather than paying a premium for elasticity. If your traffic is predictable (which it usually is for B2B SaaS), you’re overpaying for serverless.
## GPU Inference: The Real Cost Center
This is where most indie developers get the math wrong. Hosting the backend is cheap. The real cost is model inference.
OpenAI API costs $2.50/1M input tokens for GPT-4o mini and $10/1M output tokens. A single chat completion with a 2K input and 500 output tokens costs about $0.0075. If your app does 10,000 of those per day, that’s $75/day in API costs alone.
Running your own model on RunPod or similar can cut this by 5-10x if you have consistent traffic. An A100 instance on RunPod costs about $0.79/hour. Running Llama 3 70B on it, you can serve roughly 100-200 requests/minute. At 10,000 requests/day, that’s about 1-2 hours of compute = $0.79-1.58/day, compared to $75/day via API.
The math flips again when traffic drops below a certain threshold. If you only need 100 requests/day, the API is cheaper because you’re not paying for idle GPU time.
## A Decision Framework
After iterating through multiple architectures, here’s the framework I use now:
– **Phase 1 (0-100 users):** Serverless backend + API-based inference. Cost: ~$20-50/month. Don’t overthink infrastructure.
– **Phase 2 (100-1,000 users):** VPS backend + API inference. Cost: ~$50-200/month. The backend cost drops, inference stays.
– **Phase 3 (1,000+ users):** VPS backend + self-hosted inference (if you have predictable load). Cost: ~$200-500/month. This is where you start saving significantly.
Most indie developers try to optimize too early. They spend a week setting up Kubernetes on a cluster of VPSes when they have 10 users. Don’t be that person. Use serverless for speed, migrate to VPS when the bill hurts, and only self-host models when you have consistent daily traffic.
## Quick Wins
If you’re already on serverless and wondering if you should switch:
– Check your last 3 months of Cloudflare/Lambda bills. If you’re paying more than $30/month for backend compute, a VPS will likely be cheaper.
– Start with Hetzner CX22 (€3.99/month). It’s enough for most indie AI apps.
– Keep serverless for webhooks, scheduled tasks, and edge functions. The hybrid approach often works best.
The hosting decision for an AI app is rarely the bottleneck to success. Pick something simple, ship fast, and only optimize when the data tells you to.
