How ChatGPT streams messages back to you

Recently I was thinking about how I'd design a system like ChatGPT, particularly its response-streaming effect. Curious about the magic behind this effect, I began to poke around Chrome’s network inspector to see exactly how the frontend and backend were communicating.

Checking the network traffic

_{DevTools Network tab}

Looking at the network tab, I first noticed that a POST request to the path /conversation was made whenever you send a message.

I imagined that this would lead to a websocket connection being established, but I could not find any such websocket being created.

After a fair bit of head-scratching and further digging, I noticed the content-type of the POST response: text/event-stream. I also noticed the body of the response was a funky format that definitely wasn't standard JSON.

_{DevTools Network tab}

And this was the moment I learned about Server-Sent Events (SSE). This simple, unidirectional communication method allows the server to push data to the client over a regular HTTP connection—no fancy bidirectional protocol required.

Understanding Server-Sent Events (SSE)

SSE is a standard built on top of HTTP that lets servers send automatic updates to the client. Unlike WebSockets, SSE is strictly one-way: the server sends data, and the client listens. This makes it a lightweight and effective solution for cases where the client doesn’t need to send continuous data back to the server.

And it turns out implementing SSE into your own application is actually pretty simple. Here’s a basic example:

// Front-end JavaScript
 
// Create an EventSource instance that connects to your SSE endpoint.
const eventSource = new EventSource("/sse");
 
// Listen for incoming messages.
eventSource.onmessage = (event) => {
  console.log(`Received event: ${event.data}`);
};
 
// Optionally handle errors.
eventSource.onerror = (error) => {
  console.error("EventSource failed:", error);
};

Building Your Own SSE Endpoint with Node.js

Inspired by ChatGPT’s approach, I decided to set up my own SSE endpoint using Node.js and Express. This example demonstrates a simple server that sends a series of messages to the client:

// server.js
const express = require('express');
const app = express();
const port = process.env.PORT || 3000;
 
app.get('/sse', (req, res) => {
  // Set headers to establish SSE connection.
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');
 
  // Send an initial message to confirm connection.
  res.write('data: Connected to SSE endpoint\n\n');
 
  // Function to send messages periodically.
  let messageCount = 0;
  const intervalId = setInterval(() => {
    messageCount++;
    res.write(`data: Message number ${messageCount}\n\n`);
 
    // For this demo, stop sending after 10 messages.
    if (messageCount === 10) {
      clearInterval(intervalId);
      res.write('data: Closing connection\n\n');
      res.end();
    }
  }, 1000);
 
  // If the client closes the connection, clean up the interval.
  req.on('close', () => {
    clearInterval(intervalId);
  });
});
 
app.listen(port, () => {
  console.log(`SSE server running on http://localhost:${port}`);
});

This Node.js example sets up an endpoint at /sse that:

• Sets the appropriate headers for SSE.
• Immediately notifies the client of the successful connection.
• Sends a new message every second.
• Ends the connection after sending 10 messages, with cleanup in case the client disconnects early.