TutorialsStreaming Responses

In this tutorial, we'll explore how to implement streaming with NextJS, as showcased by AgentForge. Streaming allows for real-time data transfer between the server and client, enhancing the responsiveness and interactivity of your web applications. We will walk through the server-side and client-side implementations.

Server-Side Streaming

Step 1: Create the Workflow

First, create the workflow and initiate streaming.

1import { createWorkflow } from '@/ai/workflow';
2import { HumanMessage } from '@langchain/core/messages';
3import { iteratorToStream } from './utils';
4
5export default async function handler(req, res) {
6    const body = await req.json();
7    const workflow = await createWorkflow();
8
9    const streamResults = await workflow.stream(
10        {
11            messages: [
12                new HumanMessage({
13                    content: body.question,
14                }),
15            ],
16        },
17        { recursionLimit: 20 }
18    );
19
20    const stream = iteratorToStream(streamResults);
21    return new Response(stream);
22}
23
24

Step 2: Convert Iterator to Stream

Use the iteratorToStream helper function to wrap the iterator into a stream.

1function iteratorToStream(iterator: any) {
2    return new ReadableStream({
3        async pull(controller) {
4            const { value, done } = await iterator.next();
5
6            if (done) {
7                controller.close();
8            } else {
9                const json = JSON.stringify(value);
10                controller.enqueue(Buffer.from(json));
11            }
12        },
13    });
14}
15
16export { iteratorToStream };
17
18

Step 3: Return the Stream as a Response

Return the stream as the response to the client.

1...
2return new Response(stream);
3...
4
5

Client-Side Streaming

Client-side streaming is implemented using the useAgents hook. To build your custom streaming logic, you can obtain a reader from the response body and read each message as follows:

1const reader = response.body.getReader();
2let done, value;
3while (!done) {
4	({ value, done } = await reader.read());
5	const str = new TextDecoder().decode(value);
6	...
7}

Conclusion

By following this tutorial, you've implemented streaming with NextJS, enabling real-time data transfer between the server and client. This setup enhances the interactivity and responsiveness of your application, providing a smoother user experience. Explore further customizations to optimize the streaming process for your specific needs.

Happy building with NextJS and AgentForge!