Streaming AI responses in Next.js

Chat completions can stream server-sent events. If your route waits for the full body, the UI sits idle until the model finishes. The AI SaaS Starter returns the upstream body as a ReadableStream and sets Content-Type: text/event-stream.

Upstream request

streamChat in src/lib/ai/provider.ts:

const response = await fetch(`${process.env.AI_BASE_URL || "https://api.openai.com/v1"}/chat/completions`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.AI_API_KEY}`,
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    model: process.env.AI_MODEL || "gpt-4o-mini",
    stream: true,
    messages
  })
});
if (!response.ok || !response.body) throw new Error("AI provider request failed");
return response.body;

stream: true is the request flag. The function returns response.body, a web ReadableStream, not a parsed string. The default model string is gpt-4o-mini only when AI_MODEL is unset. The kit does not send stream_options, so it does not ask the provider for a usage object on the stream.

Re-encoding for the browser

The route reads that stream, copies each chunk to the client, and keeps a decoded copy for the database:

const reader = upstream.getReader();
const decoder = new TextDecoder();
let full = "";
const stream = new ReadableStream({
  async pull(controller) {
    const { value, done } = await reader.read();
    if (done) {
      await prisma.generation.create({
        data: { userId: user.id, prompt: String(prompt), output: full, credits: 1 }
      });
      controller.close();
      return;
    }
    full += decoder.decode(value);
    controller.enqueue(value);
  }
});
return new Response(stream, { headers: { "Content-Type": "text/event-stream" } });

enqueue(value) forwards the original bytes. The client still has to parse SSE (data: {...} lines) if it wants token text. The kit’s stored output is the concatenated SSE payload, not a stripped assistant string. If you display history, parse the data: lines before you render them, or stop storing the raw frames.

TextDecoder is used without { stream: true }. A multibyte character split across chunks can decode incorrectly at the boundary. For a log that is good enough to debug, that is acceptable. For user-visible history, pass { stream: true } and finish the decoder on done.

What this route does not do

It sends a single user message. It does not load earlier Generation rows into messages, so the model does not see the previous turn unless you add that. It does not set a timeout. A hung provider holds the response open until the platform closes it.

Credit checks happen before streamChat. The design of that check is in implementing AI usage limits and credits.

Want to skip the setup? The AI SaaS Starter already streams from an OpenAI-compatible endpoint and stores the generation when the stream ends.