5 Things Nobody Tells You About Deploying AI Apps on Serverless (Cost Data Included)

Every AI SaaS founder starts the same way. You prototype with an API key, build a quick frontend, deploy on Vercel or Cloudflare Workers, and it costs almost nothing. The demo works. You feel like a genius.

Then you launch. A few hundred users show up. Your serverless bill goes from $5 to $500 in a week. And you’re sitting there wondering what went wrong.

I’ve been through this cycle three times now with different AI products. Each time I learned something the first post didn’t tell me. Here are the five things I wish someone had laid out clearly before I started.

1. The Per-Request Cost Math Is Deceptive

The standard pitch for serverless is “pay per request.” Sounds great. But the cost structure changes dramatically when you’re dealing with AI workloads.

Take Cloudflare Workers. The free tier gives you 100,000 requests per day. Sounds generous until you realize each AI inference request might be 5-10 seconds of compute time, and Workers charge by CPU time after the first 10ms of “wall time.” A single OpenAI API call proxied through a Worker can cost you in the 5-10 millicents range — not much, but it adds up fast.

Here’s a real cost breakdown from one of my projects: a simple AI text analysis tool that calls GPT-4o-mini per request.

At 100 requests/day: essentially $0 (within free tiers)
At 1,000 requests/day: ~$0.30/day in compute + $0.50/day in API costs
At 10,000 requests/day: ~$3/day in compute + $5/day in API costs
At 100,000 requests/day: ~$30/day in compute + $50/day in API costs

The compute costs from the serverless platform ended up being 30-40% of the total cost. I’d only budgeted for the AI API costs. That was mistake number one.

According to Cloudflare’s own pricing documentation (2026), Workers Unbound pricing is $0.015 per million CPU milliseconds and $0.015 per million requests. If each request averages 50ms CPU time (typical for a proxy + small computation), 100,000 requests costs about $0.075 in CPU + $0.0015 in requests = ~$0.077. That doesn’t sound bad until your request grows 10x.

The solution: run your AI API calls from a VPS with persistent connections instead of proxying through serverless functions. I switched to a $10/month VPS running a simple Node.js proxy with keepalive, and my compute costs dropped by 90%.

2. Cold Starts Are Worse With AI Than Regular APIs

Everyone knows serverless has cold starts. But with AI apps, the problem compounds in ways that aren’t obvious.

A typical web API cold start is 100-500ms. Annoying but manageable. With AI apps, you often need to load configuration, initialize the AI client, load prompt templates, and sometimes initialize embeddings. A cold start for an AI endpoint can easily hit 2-3 seconds.

Now imagine you’re proxying streaming responses. A cold start on a streaming endpoint means the user stares at a blank screen for 3 seconds before the first token appears. You’ve already lost them.

I ran a test comparing warm vs cold start latency on Cloudflare Workers for an AI streaming endpoint. The cold start made the experience feel broken. Even with aggressive caching, about 15% of requests hit cold starts.

Fix options:

Use a reserved concurrency or provisioned concurrency (available on AWS Lambda, not on Workers)
Run a small Node.js proxy on a VPS that forwards to Workers — keeps the connection warm
Accept cold starts but show a better loading state

I went with option two. A $5/month VPS running HAProxy forwarding to Workers eliminated 90% of cold starts because the proxy keeps the connection alive.

3. API Gateway Costs Can Exceed Compute Costs

This one blindsided me. You deploy your AI app, it uses Workers or Lambda, and you think the cost is just compute + AI API calls. But if you put it behind an API gateway for rate limiting, auth, and monitoring, the gateway charges can add up fast.

AWS API Gateway charges $3.50 per million requests plus data transfer. Cloudflare API Gateway charges $0.05 per 10,000 requests for some features.

For a small app with 50,000 requests/month, the gateway costs might be $5-10. For a growing app at 500,000 requests/month, you’re looking at $20-50/month just for the gateway.

The hidden cost is on the data transfer. If your AI responses are large (50-100KB per response), data transfer through the gateway can double your effective cost.

The fix that worked for me: skip the dedicated API gateway until you really need it. Use the built-in rate limiting on Cloudflare Workers (free) or Vercel Edge Functions (included in Pro plan).

4. Streaming Responses Change the Cost Equation

If you’re building an AI app that streams responses, the cost structure shifts again.

Streaming keeps the serverless function alive longer. A non-streaming request might complete in 2 seconds of wall time. A streaming request for the same response takes 5-10 seconds because you’re waiting for the AI model to generate tokens.

Cloudflare Workers charge by wall clock time after the first 10ms. AWS Lambda charges by duration in 1ms increments. A streaming endpoint that runs for 8 seconds costs 80x more per request than a fast API call that completes in 100ms.

I tracked this on my project. Streaming endpoints accounted for 40% of my compute costs despite being only 15% of total requests.

Solutions:

Use Server-Sent Events instead of WebSocket for streaming (SSE is cheaper on most platforms)
Consider switching to a VPS for streaming-heavy endpoints
Implement response caching for common queries

5. Database Connection Management in Serverless Is a Pain

This is the most technically annoying problem. Serverless functions are ephemeral — they spin up, execute, and die. Databases expect persistent connections.

If you’re using PostgreSQL with Prisma or similar, every cold start opens a new database connection. With 100 concurrent users, you might have 50 connections open simultaneously. Most managed Postgres plans max out at 20-50 connections.

The AI-specific twist: many AI apps store embeddings in vector databases like pgvector. Vector queries are computationally expensive and connection pooling matters even more.

What actually worked:

Use Prisma Accelerate or a dedicated connection pooler like PgBouncer
Reduce connection timeout aggressively (10s max)
Use Neon’s serverless Postgres — it’s designed for this pattern
For vector search, consider Pinecone or Qdrant cloud

What I’m Doing Now

After going through this three times, my current AI app stack is:

Compute: Cloudflare Workers for API proxying, VPS for streaming endpoints
Database: Neon Postgres (serverless-friendly) + Prisma Accelerate
Vector DB: Qdrant Cloud (they have a generous free tier and handle connections)
Cold start fix: $5 VPS with HAProxy as a warm-up proxy
Cost monitoring: OpenCost dashboard tracking per-endpoint costs

The biggest lesson across all three projects: serverless for AI is not “set and forget.” You need to monitor cost per endpoint from day one, or you’ll get a surprise bill that kills your margins.

Start with a simple stack, add one user, track the exact cost, and scale from there. The math changes at every order of magnitude, and the only way to know your numbers is to measure them.