All writing
Next.jsOpenAIReactPerformance

Streaming Without Jank: Making an AI Chat Feel Instant in Next.js

6 min read

An AI chat product is judged in the first two seconds. The model can take thirty; the interface still has to feel instant. Building NextChat taught me that most of that feeling is frontend architecture, not model speed.

Perceived latency is the real metric

Users forgive a slow answer. They do not forgive a blank screen. The moment the first token arrives, the UI must show it — but naively re-rendering the whole thread per token melts under long conversations.

// Buffer chunks; paint on animation frames, not per token
const buffer = useRef('');
function onToken(chunk: string) {
  buffer.current += chunk;
  if (!rafId.current) {
    rafId.current = requestAnimationFrame(() => {
      setStreamedText((t) => t + buffer.current);
      buffer.current = '';
      rafId.current = 0;
    });
  }
}

One repaint per frame is visually identical to one repaint per token and costs an order of magnitude less. Pair it with an abort path — a stopped generation should keep what it produced, not destroy it.

Sessions, not one big conversation

Multi-chat support forced a clean model: each chat is an independent session with its own message list and in-flight state. Switching chats never loses a streaming response, because the stream belongs to the session, not the currently-visible component.

Streaming is a UX feature wearing an infrastructure costume. The budget is frames, not tokens.

Need this kind of thinking on your product?

I take on select freelance and contract work — Next.js, Node.js, and AI-powered products.

Keep reading

Online now

Usually replies within an hour

Chat on WhatsApp1