Streaming Without Jank: Making an AI Chat Feel Instant in Next.js
An AI chat product is judged in the first two seconds. The model can take thirty; the interface still has to feel instant. Building NextChat taught me that most of that feeling is frontend architecture, not model speed.
Perceived latency is the real metric
Users forgive a slow answer. They do not forgive a blank screen. The moment the first token arrives, the UI must show it — but naively re-rendering the whole thread per token melts under long conversations.
// Buffer chunks; paint on animation frames, not per token
const buffer = useRef('');
function onToken(chunk: string) {
buffer.current += chunk;
if (!rafId.current) {
rafId.current = requestAnimationFrame(() => {
setStreamedText((t) => t + buffer.current);
buffer.current = '';
rafId.current = 0;
});
}
}One repaint per frame is visually identical to one repaint per token and costs an order of magnitude less. Pair it with an abort path — a stopped generation should keep what it produced, not destroy it.
Sessions, not one big conversation
Multi-chat support forced a clean model: each chat is an independent session with its own message list and in-flight state. Switching chats never loses a streaming response, because the stream belongs to the session, not the currently-visible component.
- Persist session metadata and history server-side; hydrate lazily so a reload lands you exactly where you were.
- Optimistic user messages: render before the round trip, reconcile after — the single biggest 'is it fast?' win.
- Keep the component system honest: export, edit, and regenerate features dropped into NextChat without rewrites because the chat surface was built as reusable pieces.
Streaming is a UX feature wearing an infrastructure costume. The budget is frames, not tokens.
Need this kind of thinking on your product?
I take on select freelance and contract work — Next.js, Node.js, and AI-powered products.