33.2 Streaming UX patterns that feel instant

The Illusion of Speed

You can't make the model think faster. But you can make users feel faster by showing progress immediately.

Without streaming: User waits 5s, sees everything at once
With streaming: User sees first word at 0.5s, rest streams in

Implementation

// Gemini streaming
import { GoogleGenerativeAI } from '@google/generative-ai';

async function streamResponse(prompt: string) {
  const genAI = new GoogleGenerativeAI(process.env.GOOGLE_API_KEY);
  const model = genAI.getGenerativeModel({ model: 'gemini-1.5-flash' });
  
  const result = await model.generateContentStream(prompt);
  
  for await (const chunk of result.stream) {
    const text = chunk.text();
    process.stdout.write(text);  // Display as it arrives
  }
}

// Server-Sent Events (SSE) for web
app.get('/api/generate', async (req, res) => {
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');
  
  const stream = await model.generateContentStream(req.query.prompt);
  
  for await (const chunk of stream.stream) {
    res.write(`data: ${JSON.stringify({ text: chunk.text() })}\n\n`);
  }
  
  res.write('data: [DONE]\n\n');
  res.end();
});

UX Patterns

• Typewriter: Show text as it arrives (chat interfaces)
• Skeleton + stream: Show placeholder, fill with content
• Progress indicator: Show "Thinking..." then replace
• Speculative UI: Render container, stream data into it

// React streaming component
function StreamingMessage({ prompt }) {
  const [text, setText] = useState('');
  
  useEffect(() => {
    const source = new EventSource(`/api/generate?prompt=${prompt}`);
    source.onmessage = (e) => {
      if (e.data === '[DONE]') {
        source.close();
      } else {
        const { text } = JSON.parse(e.data);
        setText(prev => prev + text);
      }
    };
    return () => source.close();
  }, [prompt]);
  
  return 
{text || 'Thinking...'}
;
}

Where to go next

• 33.3 Parallelizing retrieval