You have an AI idea. Now comes the hard part: where do you host it? How do you keep costs under control? How do you handle the boring infrastructure stuff so you can focus on the actual product?
I have been through this loop a few times now. Every time I start a new AI side project, I end up writing the same boilerplate: a Workers entry point, an AI binding, some environment config, a deployment script. So I finally packaged it into a reusable template. Here it is, step by step, so you can skip the setup and go straight to building.
Why Cloudflare Workers for AI Apps?
Cloudflare Workers AI is serverless GPU inference running on Cloudflare global network. As of August 2026, it offers 50+ open-source models including Llama 3.1, DeepSeek, and various embedding models (source: Cloudflare Workers AI Models).
The pricing model uses Neurons: you get 10,000 Neurons per day free on the Workers Free plan. Beyond that, it costs $0.011 per 1,000 Neurons on the Workers Paid plan (source: Workers AI Pricing). To put that in perspective: running a single Llama 3.1 8B inference costs about 180 Neurons, so the free tier covers roughly 55 requests per day. For a prototype or personal tool, that is plenty.
What makes Workers compelling is not just the AI part. It is the whole platform: you get a global edge network, KV storage for caching, D1 for SQL, Vectorize for embeddings, and R2 for object storage. All in one account, one bill. No separate AWS accounts, no VPC config, no cluster management.
The Template Structure
Here is what a minimal but production-ready AI Worker project looks like:
my-ai-worker/
├── src/
│ ├── index.ts # Main entry point
│ ├── ai.ts # AI inference wrapper
│ └── utils.ts # Helper functions
├── wrangler.jsonc # Worker configuration
├── package.json
├── tsconfig.json
└── .env.example
Step 1: Project Setup
Start with create-cloudflare:
npm create cloudflare@latest -- my-ai-worker
# Select: Hello World example → Worker only → TypeScript → No to deploy
cd my-ai-worker
This scaffolds a basic Worker with TypeScript support and Wrangler CLI.
Step 2: Configure the AI Binding
Edit wrangler.jsonc to add the AI binding:
{
"name": "my-ai-worker",
"main": "src/index.ts",
"compatibility_date": "2026-08-01",
"ai": {
"binding": "AI"
}
}
If you prefer TOML (wrangler.toml):
[ai]
binding = "AI"
This binding makes env.AI available in your Worker code, giving you direct access to all Workers AI models without any API key management.
Step 3: The AI Inference Wrapper
Create src/ai.ts:
import { Ai } from "./cloudflare"; // Type is auto-generated by Wrangler
interface AIResponse {
response: string;
usage?: { neurons: number };
}
export async function runLLM(
env: { AI: Ai },
model: string,
prompt: string,
system?: string
): Promise<AIResponse> {
const messages = [];
if (system) messages.push({ role: "system", content: system });
messages.push({ role: "user", content: prompt });
const result = await env.AI.run(model, {
messages,
stream: false,
});
return {
response: (result as any).response || "",
usage: (result as any).usage,
};
}
export async function generateEmbeddings(
env: { AI: Ai },
model: string,
text: string | string[]
): Promise {
const result = await env.AI.run(model, {
text: Array.isArray(text) ? text : [text],
});
return (result as any).data.map((d: any) => d.embedding);
}
This wrapper handles two common patterns: LLM chat completion and text embeddings. The run method auto-resolves the model endpoint, so you just pass the model slug like "@cf/meta/llama-3.1-8b-instruct".
Step 4: Main Handler with Caching
Now src/index.ts:
import { runLLM } from "./ai";
export default {
async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
if (request.method !== "POST") {
return new Response("Send a POST request with JSON body", { status: 405 });
}
try {
const { prompt, model } = await request.json() as {
prompt?: string;
model?: string;
};
if (!prompt) {
return new Response(JSON.stringify({ error: "prompt is required" }), {
status: 400,
headers: { "Content-Type": "application/json" },
});
}
const selectedModel = model || "@cf/meta/llama-3.1-8b-instruct";
const cacheKey = `ai:${selectedModel}:${prompt}`;
// Check KV cache first
const cached = await env.KV?.get(cacheKey);
if (cached) {
return new Response(cached, {
headers: { "Content-Type": "application/json", "X-Cache": "HIT" },
});
}
const result = await runLLM(env, selectedModel, prompt);
const responseBody = JSON.stringify({ result: result.response });
// Cache for 1 hour (non-critical data)
ctx.waitUntil(
env.KV?.put(cacheKey, responseBody, { expirationTtl: 3600 })
);
return new Response(responseBody, {
headers: { "Content-Type": "application/json" },
});
} catch (error) {
return new Response(JSON.stringify({ error: String(error) }), {
status: 500,
headers: { "Content-Type": "application/json" },
});
}
},
} satisfies ExportedHandler<Env>;
This adds KV caching out of the box. For a personal tool or prototype, caching identical prompts saves both latency and Neuron usage. The ctx.waitUntil pattern ensures the cache write happens after the response is sent, so the user does not wait for it.
Step 5: Deploy and Test
npx wrangler deploy
Once deployed, you get a URL like https://my-ai-worker.your-subdomain.workers.dev. Test it:
curl -X POST https://my-ai-worker.your-subdomain.workers.dev \
-H "Content-Type: application/json" \
-d '{"prompt": "Explain serverless AI in one sentence"}'
Cost Breakdown for a Real Project
Let us say you build a simple AI chat tool that gets 1,000 requests per day. Each request uses Llama 3.1 8B (~180 Neurons). That is 180,000 Neurons/day. With the free 10,000 Neurons, you pay for 170,000 Neurons. At $0.011/1,000 Neurons, that is $1.87/day, or about $56/month. Plus Workers Paid plan ($10/month). Total: roughly $66/month for a serverless AI app handling 1K requests/day. Compare that to renting a GPU instance, and the math is clear.
Where To Go From Here
This template is intentionally minimal. Here are a few ways to extend it:
- Add Vectorize for RAG: store embeddings and do semantic search over your documents
- Add D1 for persisting chat history
- Add AI Gateway for rate limiting, analytics, and model fallback
- Add authentication with Cloudflare Access or a simple API key check
If you want the complete version, check the Cloudflare Workers AI getting started guide (source: Workers AI Get Started).
The point is: do not spend a weekend on boilerplate. Spend it on what makes your AI app different.
