Fastify with AI SDK

Learn how to integrate the AI SDK into Fastify applications for high-performance AI endpoints.

Why Fastify?

Fastify is an excellent choice for AI applications:

Fast: Low overhead framework optimized for speed
Schema-based: Built-in JSON schema validation
Plugin System: Extensible architecture
TypeScript: First-class TypeScript support
Logging: Built-in structured logging

Prerequisites

Node.js 18+
Basic knowledge of Fastify
Vercel AI Gateway API key

Quick Start

Create a new project:

mkdir my-ai-server
cd my-ai-server
pnpm init

Install dependencies:

pnpm add fastify ai
pnpm add -D tsx typescript

Configure TypeScript:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "Node16",
    "moduleResolution": "Node16",
    "outDir": "./dist",
    "esModuleInterop": true,
    "strict": true
  }
}

Set environment variables:

echo "AI_GATEWAY_API_KEY=your-api-key" > .env

Basic Streaming

Stream AI responses with Fastify:

import { streamText } from 'ai';
import Fastify from 'fastify';
import 'dotenv/config';

const fastify = Fastify({ logger: true });

fastify.post('/api/chat', async function (request, reply) {
  const { prompt } = request.body as { prompt: string };

  const result = streamText({
    model: 'openai/gpt-4o',
    prompt: prompt || 'Tell me a fun fact',
  });

  reply.header('Content-Type', 'text/plain; charset=utf-8');
  return reply.send(result.toUIMessageStream());
});

fastify.listen({ port: 8080 }, (err) => {
  if (err) throw err;
  console.log('Server running on http://localhost:8080');
});

Run the server:

pnpx tsx src/index.ts

Test with curl:

curl -X POST http://localhost:8080/api/chat \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Why is the sky blue?"}'

UI Message Stream

Stream messages compatible with useChat:

import { streamText, convertToModelMessages, UIMessage } from 'ai';
import Fastify from 'fastify';
import cors from '@fastify/cors';

const fastify = Fastify({ logger: true });

await fastify.register(cors, {
  origin: true,
});

fastify.post('/api/chat', async function (request, reply) {
  const { messages } = request.body as { messages: UIMessage[] };

  const result = streamText({
    model: 'openai/gpt-4o',
    messages: await convertToModelMessages(messages),
  });

  return reply.send(result.toUIMessageStreamResponse());
});

fastify.listen({ port: 8080 });

Text-Only Stream

Stream plain text responses:

import { streamText } from 'ai';
import Fastify from 'fastify';

const fastify = Fastify({ logger: true });

fastify.post('/api/generate', async function (request, reply) {
  const { prompt } = request.body as { prompt: string };

  const result = streamText({
    model: 'openai/gpt-4o',
    prompt,
  });

  reply.header('Content-Type', 'text/plain; charset=utf-8');
  return reply.send(result.textStream);
});

fastify.listen({ port: 8080 });

Custom Data Streaming

Send custom data with AI responses:

import { createUIMessageStream, streamText } from 'ai';
import Fastify from 'fastify';

const fastify = Fastify({ logger: true });

fastify.post('/api/chat', async function (request, reply) {
  const { prompt } = request.body as { prompt: string };

  const stream = createUIMessageStream({
    execute: async ({ writer }) => {
      writer.write({ type: 'start' });

      writer.write({
        type: 'data-custom',
        data: {
          timestamp: new Date().toISOString(),
          requestId: Math.random().toString(36),
        },
      });

      const result = streamText({
        model: 'openai/gpt-4o',
        prompt,
      });

      writer.merge(result.toUIMessageStream({ sendStart: false }));
    },
    onError: error => {
      return error instanceof Error ? error.message : String(error);
    },
  });

  reply.header('Content-Type', 'text/plain; charset=utf-8');
  return reply.send(stream);
});

fastify.listen({ port: 8080 });

Tool Calling

Implement AI tools with Fastify:

import {
  streamText,
  convertToModelMessages,
  tool,
  UIMessage,
} from 'ai';
import Fastify from 'fastify';
import { z } from 'zod';

const fastify = Fastify({ logger: true });

fastify.post('/api/chat', async function (request, reply) {
  const { messages } = request.body as { messages: UIMessage[] };

  const result = streamText({
    model: 'openai/gpt-4o',
    messages: await convertToModelMessages(messages),
    tools: {
      getWeather: tool({
        description: 'Get current weather for a city',
        inputSchema: z.object({
          city: z.string().describe('City name'),
        }),
        execute: async ({ city }) => {
          const weatherData = {
            city,
            temperature: 72,
            condition: 'Sunny',
            humidity: 65,
          };
          return weatherData;
        },
      }),
      searchWeb: tool({
        description: 'Search the web for information',
        inputSchema: z.object({
          query: z.string(),
        }),
        execute: async ({ query }) => {
          // Implement web search
          return {
            query,
            results: [
              { title: 'Result 1', url: 'https://example.com/1' },
              { title: 'Result 2', url: 'https://example.com/2' },
            ],
          };
        },
      }),
    },
  });

  return reply.send(result.toUIMessageStreamResponse());
});

fastify.listen({ port: 8080 });

Structured Output

Generate structured JSON with schema validation:

import { generateObject, Output } from 'ai';
import Fastify from 'fastify';
import { z } from 'zod';

const fastify = Fastify({ logger: true });

const recipeSchema = z.object({
  name: z.string(),
  ingredients: z.array(z.string()),
  steps: z.array(z.string()),
  prepTime: z.number(),
  cookTime: z.number(),
});

fastify.post('/api/recipe', async function (request, reply) {
  const { dish } = request.body as { dish: string };

  const { object: recipe } = await generateObject({
    model: 'openai/gpt-4o',
    prompt: `Generate a recipe for ${dish}`,
    output: Output.object({
      schema: recipeSchema,
    }),
  });

  return reply.send(recipe);
});

fastify.listen({ port: 8080 });

Schema Validation

Use Fastify’s built-in schema validation:

import { streamText } from 'ai';
import Fastify from 'fastify';

const fastify = Fastify({ logger: true });

fastify.post(
  '/api/chat',
  {
    schema: {
      body: {
        type: 'object',
        required: ['prompt'],
        properties: {
          prompt: { type: 'string', minLength: 1, maxLength: 1000 },
          temperature: { type: 'number', minimum: 0, maximum: 2 },
        },
      },
      response: {
        200: {
          type: 'object',
          properties: {
            text: { type: 'string' },
          },
        },
      },
    },
  },
  async function (request, reply) {
    const { prompt, temperature } = request.body as {
      prompt: string;
      temperature?: number;
    };

    const result = streamText({
      model: 'openai/gpt-4o',
      prompt,
      temperature,
    });

    return reply.send(result.toUIMessageStreamResponse());
  },
);

fastify.listen({ port: 8080 });

Plugins

Rate Limiting Plugin

import fp from 'fastify-plugin';
import rateLimit from '@fastify/rate-limit';

export default fp(async (fastify) => {
  await fastify.register(rateLimit, {
    max: 100,
    timeWindow: '15 minutes',
  });
});

Authentication Plugin

import fp from 'fastify-plugin';

export default fp(async (fastify) => {
  fastify.decorate('authenticate', async (request, reply) => {
    const apiKey = request.headers['x-api-key'];

    if (!apiKey || apiKey !== process.env.API_KEY) {
      reply.code(401).send({ error: 'Unauthorized' });
    }
  });
});

Use plugins:

import Fastify from 'fastify';
import ratelimitPlugin from './plugins/ratelimit';
import authPlugin from './plugins/auth';

const fastify = Fastify({ logger: true });

await fastify.register(ratelimitPlugin);
await fastify.register(authPlugin);

fastify.post(
  '/api/chat',
  { onRequest: [fastify.authenticate] },
  async (request, reply) => {
    // Protected route
  },
);

fastify.listen({ port: 8080 });

Error Handling

Global error handler:

import Fastify from 'fastify';

const fastify = Fastify({ logger: true });

fastify.setErrorHandler((error, request, reply) => {
  request.log.error(error);

  reply.status(error.statusCode || 500).send({
    error: error.message,
    statusCode: error.statusCode || 500,
  });
});

fastify.post('/api/chat', async (request, reply) => {
  try {
    // AI logic
  } catch (error) {
    throw error;
  }
});

fastify.listen({ port: 8080 });

Production Setup

Project Structure

my-ai-server/
├── src/
│   ├── index.ts
│   ├── routes/
│   │   ├── chat.ts
│   │   └── generate.ts
│   ├── plugins/
│   │   ├── auth.ts
│   │   └── ratelimit.ts
│   └── schemas/
│       └── chat.ts
├── .env
├── package.json
└── tsconfig.json

Routes as Plugins

import { FastifyPluginAsync } from 'fastify';
import { streamText } from 'ai';

const chatRoutes: FastifyPluginAsync = async (fastify) => {
  fastify.post('/chat', async (request, reply) => {
    const { prompt } = request.body as { prompt: string };

    const result = streamText({
      model: 'openai/gpt-4o',
      prompt,
    });

    return reply.send(result.toUIMessageStreamResponse());
  });
};

export default chatRoutes;

import Fastify from 'fastify';
import chatRoutes from './routes/chat';
import generateRoutes from './routes/generate';

const fastify = Fastify({ logger: true });

await fastify.register(chatRoutes, { prefix: '/api' });
await fastify.register(generateRoutes, { prefix: '/api' });

fastify.listen({ port: 8080 });

Build Scripts

{
  "scripts": {
    "dev": "tsx watch src/index.ts",
    "build": "tsc",
    "start": "node dist/index.js",
    "test": "vitest"
  }
}

Deployment

Docker

FROM node:20-alpine

WORKDIR /app

COPY package*.json ./
RUN npm install --production

COPY dist ./dist

EXPOSE 8080

CMD ["npm", "start"]

Environment Variables

AI_GATEWAY_API_KEY=your-key
API_KEY=your-auth-key
PORT=8080
NODE_ENV=production

Performance Tips

Enable HTTP/2: For better streaming performance
Use Async/Await: Properly handle promises
Connection Pooling: Reuse database connections
Caching: Cache frequent responses
Compression: Use for non-streaming routes

Best Practices

Use Plugins: Modular architecture
Schema Validation: Validate all inputs
Logging: Use Fastify’s built-in logger
Error Handling: Centralized error handling
TypeScript: Full type safety
Testing: Write tests for routes

Example Repository

View the complete example: github.com/vercel/ai/examples/fastify

Documentation Index

​Fastify with AI SDK

​Why Fastify?

​Prerequisites

​Quick Start

​Basic Streaming

​UI Message Stream

​Text-Only Stream

​Custom Data Streaming

​Tool Calling

​Structured Output

​Schema Validation

​Plugins

​Rate Limiting Plugin

​Authentication Plugin

​Error Handling

​Production Setup

​Project Structure

​Routes as Plugins

​Build Scripts

​Deployment

​Docker

​Environment Variables

​Performance Tips

​Best Practices

​Example Repository

​Resources