Note: multi-conversation memory
If you need memory that is shared across multiple conversations or users (cross-thread persistence), check out this how-to guide).
Note
In this how-to, we will create our agent from scratch to be transparent (but verbose). You can accomplish similar functionality using the createReactAgent(model, tools=tool, checkpointer=checkpointer) (API doc) constructor. This may be more appropriate if you are used to LangChain's AgentExecutor class.
## Setup
This guide will use OpenAI's GPT-4o model. We will optionally set our API key
for [LangSmith tracing](https://smith.langchain.com/), which will give us
best-in-class observability.
```typescript
// process.env.OPENAI_API_KEY = "sk_...";
// Optional, add tracing in LangSmith
// process.env.LANGCHAIN_API_KEY = "ls__...";
process.env.LANGCHAIN_CALLBACKS_BACKGROUND = "true";
process.env.LANGCHAIN_TRACING_V2 = "true";
process.env.LANGCHAIN_PROJECT = "Persistence: LangGraphJS";
```
```output
Persistence: LangGraphJS
```
## Define the state
The state is the interface for all of the nodes in our graph.
```typescript
import { Annotation } from "@langchain/langgraph";
import { BaseMessage } from "@langchain/core/messages";
const GraphState = Annotation.Root({
messages: Annotation({
reducer: (x, y) => x.concat(y),
}),
});
```
## Set up the tools
We will first define the tools we want to use. For this simple example, we will
use create a placeholder search engine. However, it is really easy to create
your own tools - see documentation
[here](https://js.langchain.com/docs/how_to/custom_tools) on how to do
that.
```typescript
import { tool } from "@langchain/core/tools";
import { z } from "zod";
const searchTool = tool(async ({}: { query: string }) => {
// This is a placeholder for the actual implementation
return "Cold, with a low of 13 ℃";
}, {
name: "search",
description:
"Use to surf the web, fetch current information, check the weather, and retrieve other information.",
schema: z.object({
query: z.string().describe("The query to use in your search."),
}),
});
await searchTool.invoke({ query: "What's the weather like?" });
const tools = [searchTool];
```
We can now wrap these tools in a simple
ToolNode.
This object will actually run the tools (functions) whenever they are invoked by
our LLM.
```typescript
import { ToolNode } from "@langchain/langgraph/prebuilt";
const toolNode = new ToolNode(tools);
```
## Set up the model
Now we will load the
[chat model](https://js.langchain.com/docs/concepts/#chat-models).
1. It should work with messages. We will represent all agent state in the form
of messages, so it needs to be able to work well with them.
2. It should work with
[tool calling](https://js.langchain.com/docs/how_to/tool_calling/#passing-tools-to-llms),
meaning it can return function arguments in its response.
Note
These model requirements are not general requirements for using LangGraph - they are just requirements for this one example.
```typescript
import { ChatOpenAI } from "@langchain/openai";
const model = new ChatOpenAI({ model: "gpt-4o" });
```
After we've done this, we should make sure the model knows that it has these
tools available to call. We can do this by calling
[bindTools](https://v01.api.js.langchain.com/classes/langchain_core_language_models_chat_models.BaseChatModel.html#bindTools).
```typescript
const boundModel = model.bindTools(tools);
```
## Define the graph
We can now put it all together. We will run it first without a checkpointer:
```typescript
import { END, START, StateGraph } from "@langchain/langgraph";
import { AIMessage } from "@langchain/core/messages";
import { RunnableConfig } from "@langchain/core/runnables";
const routeMessage = (state: typeof GraphState.State) => {
const { messages } = state;
const lastMessage = messages[messages.length - 1] as AIMessage;
// If no tools are called, we can finish (respond to the user)
if (!lastMessage.tool_calls?.length) {
return END;
}
// Otherwise if there is, we continue and call the tools
return "tools";
};
const callModel = async (
state: typeof GraphState.State,
config?: RunnableConfig,
) => {
const { messages } = state;
const response = await boundModel.invoke(messages, config);
return { messages: [response] };
};
const workflow = new StateGraph(GraphState)
.addNode("agent", callModel)
.addNode("tools", toolNode)
.addEdge(START, "agent")
.addConditionalEdges("agent", routeMessage)
.addEdge("tools", "agent");
const graph = workflow.compile();
```
```typescript
let inputs = { messages: [{ role: "user", content: "Hi I'm Yu, nice to meet you." }] };
for await (
const { messages } of await graph.stream(inputs, {
streamMode: "values",
})
) {
let msg = messages[messages?.length - 1];
if (msg?.content) {
console.log(msg.content);
} else if (msg?.tool_calls?.length > 0) {
console.log(msg.tool_calls);
} else {
console.log(msg);
}
console.log("-----\n");
}
```
```output
Hi I'm Yu, nice to meet you.
-----
Hi Yu! Nice to meet you too. How can I assist you today?
-----
```
```typescript
inputs = { messages: [{ role: "user", content: "Remember my name?" }] };
for await (
const { messages } of await graph.stream(inputs, {
streamMode: "values",
})
) {
let msg = messages[messages?.length - 1];
if (msg?.content) {
console.log(msg.content);
} else if (msg?.tool_calls?.length > 0) {
console.log(msg.tool_calls);
} else {
console.log(msg);
}
console.log("-----\n");
}
```
```output
Remember my name?
-----
You haven't shared your name with me yet. What's your name?
-----
```
## Add Memory
Let's try it again with a checkpointer. We will use the
MemorySaver,
which will "save" checkpoints in-memory.
```typescript
import { MemorySaver } from "@langchain/langgraph";
// Here we only save in-memory
const memory = new MemorySaver();
const persistentGraph = workflow.compile({ checkpointer: memory });
```
```typescript
let config = { configurable: { thread_id: "conversation-num-1" } };
inputs = { messages: [{ role: "user", content: "Hi I'm Jo, nice to meet you." }] };
for await (
const { messages } of await persistentGraph.stream(inputs, {
...config,
streamMode: "values",
})
) {
let msg = messages[messages?.length - 1];
if (msg?.content) {
console.log(msg.content);
} else if (msg?.tool_calls?.length > 0) {
console.log(msg.tool_calls);
} else {
console.log(msg);
}
console.log("-----\n");
}
```
```output
Hi I'm Jo, nice to meet you.
-----
Hello Jo, nice to meet you too! How can I assist you today?
-----
```
```typescript
inputs = { messages: [{ role: "user", content: "Remember my name?"}] };
for await (
const { messages } of await persistentGraph.stream(inputs, {
...config,
streamMode: "values",
})
) {
let msg = messages[messages?.length - 1];
if (msg?.content) {
console.log(msg.content);
} else if (msg?.tool_calls?.length > 0) {
console.log(msg.tool_calls);
} else {
console.log(msg);
}
console.log("-----\n");
}
```
```output
Remember my name?
-----
Yes, I'll remember that your name is Jo. How can I assist you today?
-----
```
## New Conversational Thread
If we want to start a new conversation, we can pass in a different
**`thread_id`**. Poof! All the memories are gone (just kidding, they'll always
live in that other thread)!
```typescript
config = { configurable: { thread_id: "conversation-2" } };
```
```output
{ configurable: { thread_id: 'conversation-2' } }
```
```typescript
inputs = { messages: [{ role: "user", content: "you forgot?" }] };
for await (
const { messages } of await persistentGraph.stream(inputs, {
...config,
streamMode: "values",
})
) {
let msg = messages[messages?.length - 1];
if (msg?.content) {
console.log(msg.content);
} else if (msg?.tool_calls?.length > 0) {
console.log(msg.tool_calls);
} else {
console.log(msg);
}
console.log("-----\n");
}
```
```output
you forgot?
-----
``````output
Could you please provide more context or details about what you are referring to? This will help me assist you better.
-----
```
---
how-tos/pass-run-time-values-to-tools.ipynb
---
# How to pass runtime values to tools
This guide shows how to define tools that depend on dynamically defined variables. These values are provided by your program, not by the LLM.
Tools can access the [config.configurable](https://langchain-ai.github.io/langgraphjs/reference/interfaces/langgraph.LangGraphRunnableConfig.html) field for values like user IDs that are known when a graph is initially executed, as well as managed values from the [store](https://langchain-ai.github.io/langgraphjs/reference/classes/checkpoint.BaseStore.html) for persistence across threads.
However, it can be convenient to access intermediate runtime values which are not known ahead of time, but are progressively generated as a graph executes, such as the current graph state. This guide will cover two techniques for this: The `getCurrentTaskInput` utility function, and closures.
## Setup
Install the following to run this guide:
```bash
npm install @langchain/langgraph @langchain/openai @langchain/core
```
Next, configure your environment to connect to your model provider.
```bash
export OPENAI_API_KEY=your-api-key
```
Optionally, set your API key for [LangSmith tracing](https://smith.langchain.com/), which will give us best-in-class observability.
```bash
export LANGCHAIN_TRACING_V2="true"
export LANGCHAIN_CALLBACKS_BACKGROUND="true"
export LANGCHAIN_API_KEY=your-api-key
```
## The `getCurrentTaskInput` Utility Function
The `getCurrentTaskInput` utility function makes it easier to get the current state in areas of your application that might be called indirectly, like tool handlers.
Compatibility
This functionality was added in @langchain/langgraph>=0.2.53.
It also requires async_hooks support, which is supported in many popular JavaScript environments (such as Node.js, Deno, and Cloudflare Workers), but not all of them (mainly web browsers). If you are deploying to an environment where this is not supported, see the closures section below.
Let's start off by defining a tool that an LLM can use to update pet preferences for a user. The tool will retrieve the current state of the graph from the current context.
### Define the agent state
Since we're just tracking messages, we'll use the `MessagesAnnotation`:
```typescript
import { MessagesAnnotation } from "@langchain/langgraph";
```
Now, declare a tool as shown below. The tool receives values in three different ways:
1. It will receive a generated list of `pets` from the LLM in its `input`.
2. It will pull a `userId` populated from the initial graph invocation.
3. It will fetch the input that was passed to the currenty executing task (either a `StateGraph` node handler, or a Functional API `entrypoint` or `task`) via the `getCurrentTaskInput` function.
It will then use LangGraph's [cross-thread persistence](https://langchain-ai.github.io/langgraphjs/how-tos/cross-thread-persistence/) to save preferences:
```typescript
import { z } from "zod";
import { tool } from "@langchain/core/tools";
import {
getCurrentTaskInput,
LangGraphRunnableConfig,
} from "@langchain/langgraph";
const updateFavoritePets = tool(async (input, config: LangGraphRunnableConfig) => {
// Some arguments are populated by the LLM; these are included in the schema below
const { pets } = input;
// Fetch the current input to the task that called this tool.
// This will be identical to the input that was passed to the `ToolNode` that called this tool.
const currentState = getCurrentTaskInput() as typeof MessagesAnnotation.State;
// Other information (such as a UserID) are most easily provided via the config
// This is set when when invoking or streaming the graph
const userId = config.configurable?.userId;
// LangGraph's managed key-value store is also accessible from the config
const store = config.store;
await store.put([userId, "pets"], "names", pets);
// Store the initial input message from the user as a note.
// Using the same key will override previous values - you could
// use something different if you wanted to store many interactions.
await store.put([userId, "pets"], "context", { content: currentState.messages[0].content });
return "update_favorite_pets called.";
},
{
// The LLM "sees" the following schema:
name: "update_favorite_pets",
description: "add to the list of favorite pets.",
schema: z.object({
pets: z.array(z.string()),
}),
});
```
If we look at the tool call schema, which is what is passed to the model for tool-calling, we can see that only `pets` is being passed:
```typescript
import { zodToJsonSchema } from "zod-to-json-schema";
console.log(zodToJsonSchema(updateFavoritePets.schema));
```
```output
{
type: 'object',
properties: { pets: { type: 'array', items: [Object] } },
required: [ 'pets' ],
additionalProperties: false,
'$schema': 'http://json-schema.org/draft-07/schema#'
}
```
Let's also declare another tool so that our agent can retrieve previously set preferences:
```typescript
const getFavoritePets = tool(
async (_, config: LangGraphRunnableConfig) => {
const userId = config.configurable?.userId;
// LangGraph's managed key-value store is also accessible via the config
const store = config.store;
const petNames = await store.get([userId, "pets"], "names");
const context = await store.get([userId, "pets"], "context");
return JSON.stringify({
pets: petNames.value,
context: context.value.content,
});
},
{
// The LLM "sees" the following schema:
name: "get_favorite_pets",
description: "retrieve the list of favorite pets for the given user.",
schema: z.object({}),
}
);
```
## Define the nodes
From here there's really nothing special that needs to be done. This approach works with both `StateGraph` and functional agents, and it works just as well with prebuilt agents like `createReactAgent`! We'll demonstrate it by defining a custom ReAct agent using `StateGraph`. This is very similar to the agent that you'd get if you were to instead call `createReactAgent`
Let's start off by defining the nodes for our graph.
1. The agent: responsible for deciding what (if any) actions to take.
2. A function to invoke tools: if the agent decides to take an action, this node will then execute that action.
We will also need to define some edges.
1. After the agent is called, we should either invoke the tool node or finish.
2. After the tool node have been invoked, it should always go back to the agent to decide what to do next
```typescript
import {
END,
START,
StateGraph,
MemorySaver,
InMemoryStore,
} from "@langchain/langgraph";
import { AIMessage } from "@langchain/core/messages";
import { ToolNode } from "@langchain/langgraph/prebuilt";
import { ChatOpenAI } from "@langchain/openai";
const model = new ChatOpenAI({ model: "gpt-4o" });
const tools = [getFavoritePets, updateFavoritePets];
const routeMessage = (state: typeof MessagesAnnotation.State) => {
const { messages } = state;
const lastMessage = messages[messages.length - 1] as AIMessage;
// If no tools are called, we can finish (respond to the user)
if (!lastMessage?.tool_calls?.length) {
return END;
}
// Otherwise if there is, we continue and call the tools
return "tools";
};
const callModel = async (state: typeof MessagesAnnotation.State) => {
const { messages } = state;
const modelWithTools = model.bindTools(tools);
const responseMessage = await modelWithTools.invoke([
{
role: "system",
content: "You are a personal assistant. Store any preferences the user tells you about."
},
...messages
]);
return { messages: [responseMessage] };
};
const workflow = new StateGraph(MessagesAnnotation)
.addNode("agent", callModel)
.addNode("tools", new ToolNode(tools))
.addEdge(START, "agent")
.addConditionalEdges("agent", routeMessage)
.addEdge("tools", "agent");
const memory = new MemorySaver();
const store = new InMemoryStore();
const graph = workflow.compile({ checkpointer: memory, store: store });
```
```typescript
import * as tslab from "tslab";
const graphViz = graph.getGraph();
const image = await graphViz.drawMermaidPng();
const arrayBuffer = await image.arrayBuffer();
await tslab.display.png(new Uint8Array(arrayBuffer));
```
## Use it!
Let's use our graph now!
```typescript
import {
BaseMessage,
isAIMessage,
isHumanMessage,
isToolMessage,
HumanMessage,
ToolMessage,
} from "@langchain/core/messages";
let inputs = {
messages: [ new HumanMessage({ content: "My favorite pet is a terrier. I saw a cute one on Twitter." }) ],
};
let config = {
configurable: {
thread_id: "1",
userId: "a-user",
},
};
function printMessages(messages: BaseMessage[]) {
for (const message of messages) {
if (isHumanMessage(message)) {
console.log(`User: ${message.content}`);
} else if (isAIMessage(message)) {
const aiMessage = message as AIMessage;
if (aiMessage.content) {
console.log(`Assistant: ${aiMessage.content}`);
}
if (aiMessage.tool_calls) {
for (const toolCall of aiMessage.tool_calls) {
console.log(`Tool call: ${toolCall.name}(${JSON.stringify(toolCall.args)})`);
}
}
} else if (isToolMessage(message)) {
const toolMessage = message as ToolMessage;
console.log(`${toolMessage.name} tool output: ${toolMessage.content}`);
}
}
}
let { messages } = await graph.invoke(inputs, config);
printMessages(messages);
```
```output
User: My favorite pet is a terrier. I saw a cute one on Twitter.
Tool call: update_favorite_pets({"pets":["terrier"]})
update_favorite_pets tool output: update_favorite_pets called.
Assistant: I've added "terrier" to your list of favorite pets. If you have any more favorites, feel free to let me know!
```
Now verify it can properly fetch the stored preferences and cite where it got the information from:
```typescript
inputs = { messages: [new HumanMessage({ content: "What're my favorite pets and what did I say when I told you about them?" })] };
config = {
configurable: {
thread_id: "2", // New thread ID, so the conversation history isn't present.
userId: "a-user"
}
};
messages = (await graph.invoke(inputs, config)).messages;
printMessages(messages);
```
```output
User: What're my favorite pets and what did I say when I told you about them?
Tool call: get_favorite_pets({})
get_favorite_pets tool output: {"pets":["terrier"],"context":"My favorite pet is a terrier. I saw a cute one on Twitter."}
Assistant: Your favorite pet is a terrier. You mentioned, "My favorite pet is a terrier. I saw a cute one on Twitter."
```
As you can see the agent is able to properly cite that the information came from Twitter!
## Closures
If you cannot use context variables in your environment, you can use [closures](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Closures) to create tools with access to dynamic content. Here is a high-level example:
```typescript
function generateTools(state: typeof MessagesAnnotation.State) {
const updateFavoritePets = tool(
async (input, config: LangGraphRunnableConfig) => {
// Some arguments are populated by the LLM; these are included in the schema below
const { pets } = input;
// Others (such as a UserID) are best provided via the config
// This is set when when invoking or streaming the graph
const userId = config.configurable?.userId;
// LangGraph's managed key-value store is also accessible via the config
const store = config.store;
await store.put([userId, "pets"], "names", pets )
await store.put([userId, "pets"], "context", {content: state.messages[0].content})
return "update_favorite_pets called.";
},
{
// The LLM "sees" the following schema:
name: "update_favorite_pets",
description: "add to the list of favorite pets.",
schema: z.object({
pets: z.array(z.string()),
}),
}
);
return [updateFavoritePets];
};
```
Then, when laying out your graph, you will need to call the above method whenever you bind or invoke tools. For example:
```typescript
const toolNodeWithClosure = async (state: typeof MessagesAnnotation.State) => {
// We fetch the tools any time this node is reached to
// form a closure and let it access the latest messages
const tools = generateTools(state);
const toolNodeWithConfig = new ToolNode(tools);
return toolNodeWithConfig.invoke(state);
};
```
---
how-tos/subgraph-persistence.ipynb
---
# How to add thread-level persistence to subgraphs
Prerequisites
This guide assumes familiarity with the following:
This guide shows how you can add [thread-level](https://langchain-ai.github.io/langgraphjs/how-tos/persistence/) persistence to graphs that use [subgraphs](https://langchain-ai.github.io/langgraphjs/how-tos/subgraph/).
## Setup
First, let's install required packages:
```bash
$ npm install @langchain/langgraph @langchain/core
```
Set up LangSmith for LangGraph development
Sign up for LangSmith to quickly spot issues and improve the performance of your LangGraph projects. LangSmith lets you use trace data to debug, test, and monitor your LLM apps built with LangGraph — read more about how to get started here.
## Define the graph with persistence
To add persistence to a graph with subgraphs, all you need to do is pass a [checkpointer](https://langchain-ai.github.io/langgraphjs/reference/classes/checkpoint.BaseCheckpointSaver.html) when **compiling the parent graph**. LangGraph will automatically propagate the checkpointer to the child subgraphs.
Note
You shouldn't provide a checkpointer when compiling a subgraph. Instead, you must define a **single** checkpointer that you pass to parentGraph.compile(), and LangGraph will automatically propagate the checkpointer to the child subgraphs. If you pass the checkpointer to the subgraph.compile(), it will simply be ignored. This also applies when you add a node that invokes the subgraph explicitly.
Let's define a simple graph with a single subgraph node to show how to do this.
```typescript
import { StateGraph, Annotation } from "@langchain/langgraph";
// subgraph
const SubgraphStateAnnotation = Annotation.Root({
foo: Annotation,
bar: Annotation,
});
const subgraphNode1 = async (state: typeof SubgraphStateAnnotation.State) => {
return { bar: "bar" };
};
const subgraphNode2 = async (state: typeof SubgraphStateAnnotation.State) => {
// note that this node is using a state key ('bar') that is only available in the subgraph
// and is sending update on the shared state key ('foo')
return { foo: state.foo + state.bar };
};
const subgraph = new StateGraph(SubgraphStateAnnotation)
.addNode("subgraphNode1", subgraphNode1)
.addNode("subgraphNode2", subgraphNode2)
.addEdge("__start__", "subgraphNode1")
.addEdge("subgraphNode1", "subgraphNode2")
.compile();
// parent graph
const StateAnnotation = Annotation.Root({
foo: Annotation,
});
const node1 = async (state: typeof StateAnnotation.State) => {
return {
foo: "hi! " + state.foo,
};
};
const builder = new StateGraph(StateAnnotation)
.addNode("node1", node1)
// note that we're adding the compiled subgraph as a node to the parent graph
.addNode("node2", subgraph)
.addEdge("__start__", "node1")
.addEdge("node1", "node2");
```
We can now compile the graph with an in-memory checkpointer (`MemorySaver`).
```typescript
import { MemorySaver } from "@langchain/langgraph-checkpoint";
const checkpointer = new MemorySaver();
// You must only pass checkpointer when compiling the parent graph.
// LangGraph will automatically propagate the checkpointer to the child subgraphs.
const graph = builder.compile({
checkpointer: checkpointer
});
```
## Verify persistence works
Let's now run the graph and inspect the persisted state for both the parent graph and the subgraph to verify that persistence works. We should expect to see the final execution results for both the parent and subgraph in `state.values`.
```typescript
const config = { configurable: { thread_id: "1" } };
```
```typescript
const stream = await graph.stream({
foo: "foo"
}, {
...config,
subgraphs: true,
});
for await (const [_source, chunk] of stream) {
console.log(chunk);
}
```
```output
{ node1: { foo: 'hi! foo' } }
{ subgraphNode1: { bar: 'bar' } }
{ subgraphNode2: { foo: 'hi! foobar' } }
{ node2: { foo: 'hi! foobar' } }
```
We can now view the parent graph state by calling `graph.get_state()` with the same config that we used to invoke the graph.
```typescript
(await graph.getState(config)).values;
```
```output
{ foo: 'hi! foobar' }
```
To view the subgraph state, we need to do two things:
1. Find the most recent config value for the subgraph
2. Use `graph.getState()` to retrieve that value for the most recent subgraph config.
To find the correct config, we can examine the state history from the parent graph and find the state snapshot before we return results from `node2` (the node with subgraph):
```typescript
let stateWithSubgraph;
const graphHistories = await graph.getStateHistory(config);
for await (const state of graphHistories) {
if (state.next[0] === "node2") {
stateWithSubgraph = state;
break;
}
}
```
The state snapshot will include the list of `tasks` to be executed next. When using subgraphs, the `tasks` will contain the config that we can use to retrieve the subgraph state:
```typescript
const subgraphConfig = stateWithSubgraph.tasks[0].state;
console.log(subgraphConfig);
```
```output
{
configurable: {
thread_id: '1',
checkpoint_ns: 'node2:25814e09-45f0-5b70-a5b4-23b869d582c2'
}
}
```
```typescript
(await graph.getState(subgraphConfig)).values
```
```output
{ foo: 'hi! foobar', bar: 'bar' }
```
If you want to learn more about how to modify the subgraph state for human-in-the-loop workflows, check out this [how-to guide](https://langchain-ai.github.io/langgraph/how-tos/subgraphs-manage-state/).
---
how-tos/react-return-structured-output.ipynb
---
# How to return structured output from the prebuilt ReAct agent
!!! info "Prerequisites"
- Agent Architectures
- [Chat Models](https://js.langchain.com/docs/concepts/chat_models/)
- [Tools](https://js.langchain.com/docs/concepts/tools/)
- [Structured Output](https://js.langchain.com/docs/concepts/structured_outputs/)
To return structured output from the prebuilt ReAct agent you can provide a `responseFormat` parameter with the desired output schema to [`createReactAgent`](https://langchain-ai.github.io/langgraphjs/reference/functions/prebuilt.createReactAgent.html):
```typescript
import { z } from "zod";
import { createReactAgent } from "@langchain/langgraph/prebuilt";
const responseFormat = z.object({
// Respond to the user in this format
mySpecialOutput: z.string(),
})
const graph = createReactAgent({
llm: llm,
tools: tools,
// specify the schema for the structured output using `responseFormat` parameter
responseFormat: responseFormat
})
```
The agent will return the output in the format specified by the `responseFormat` schema by making an additional LLM call at the end of the conversation, once there are no more tool calls to be made. You can read this guide to learn about an alternate way - treating the structured output as another tool - to achieve structured output from the agent.
## Setup
First, we need to install the required packages.
```bash
yarn add @langchain/langgraph @langchain/openai @langchain/core zod
```
This guide will use OpenAI's GPT-4o model. We will optionally set our API key
for [LangSmith tracing](https://smith.langchain.com/), which will give us
best-in-class observability.
```typescript
// process.env.OPENAI_API_KEY = "sk_...";
// Optional, add tracing in LangSmith
// process.env.LANGSMITH_API_KEY = "ls__..."
process.env.LANGSMITH_TRACING = "true";
process.env.LANGSMITH_PROJECT = "ReAct Agent with system prompt: LangGraphJS";
```
## Code
```typescript
import { ChatOpenAI } from "@langchain/openai";
import { createReactAgent } from "@langchain/langgraph/prebuilt";
import { tool } from "@langchain/core/tools";
import { z } from "zod";
const weatherTool = tool(
async (input): Promise => {
if (input.city === "nyc") {
return "It might be cloudy in nyc";
} else if (input.city === "sf") {
return "It's always sunny in sf";
} else {
throw new Error("Unknown city");
}
},
{
name: "get_weather",
description: "Use this to get weather information.",
schema: z.object({
city: z.enum(["nyc", "sf"]).describe("The city to get weather for"),
}),
}
);
const WeatherResponseSchema = z.object({
conditions: z.string().describe("Weather conditions"),
});
const tools = [weatherTool];
const agent = createReactAgent({
llm: new ChatOpenAI({ model: "gpt-4o", temperature: 0 }),
tools: tools,
responseFormat: WeatherResponseSchema,
});
```
## Usage
Let's now test our agent:
```typescript
const response = await agent.invoke({
messages: [
{
role: "user",
content: "What's the weather in NYC?",
},
],
})
```
You can see that the agent output contains a `structuredResponse` key with the structured output conforming to the specified `WeatherResponse` schema, in addition to the message history under `messages` key
```typescript
response.structuredResponse
```
```output
{ conditions: 'cloudy' }
```
### Customizing system prompt
You might need to further customize the second LLM call for the structured output generation and provide a system prompt. To do so, you can pass an object with the keys `prompt`, `schema` to the `responseFormat` parameter:
```typescript
const agent = createReactAgent({
llm: new ChatOpenAI({ model: "gpt-4o", temperature: 0 }),
tools: tools,
responseFormat: {
prompt: "Always return capitalized weather conditions",
schema: WeatherResponseSchema,
}
});
const response = await agent.invoke({
messages: [
{
role: "user",
content: "What's the weather in NYC?",
},
],
})
```
You can verify that the structured response now contains a capitalized value:
```typescript
response.structuredResponse
```
```output
{ conditions: 'Cloudy' }
```
---
how-tos/pass_private_state.ipynb
---
# How to pass private state
Oftentimes, you may want nodes to be able to pass state to each other that should NOT be part of the main schema of the graph. This is often useful because there may be information that is not needed as input/output (and therefore doesn't really make sense to have in the main schema) but is needed as part of the intermediate working logic.
Let's take a look at an example below. In this example, we will create a RAG pipeline that:
1. Takes in a user question
2. Uses an LLM to generate a search query
3. Retrieves documents for that generated query
4. Generates a final answer based on those documents
We will have a separate node for each step. We will only have the `question` and `answer` on the overall state. However, we will need separate states for the `search_query` and the `documents` - we will pass these as private state keys by defining an `input` annotation on each relevant node.
Let's look at an example!
```typescript
import { Annotation, StateGraph } from "@langchain/langgraph";
// The overall state of the graph
const OverallStateAnnotation = Annotation.Root({
question: Annotation,
answer: Annotation,
});
// This is what the node that generates the query will return
const QueryOutputAnnotation = Annotation.Root({
query: Annotation,
});
// This is what the node that retrieves the documents will return
const DocumentOutputAnnotation = Annotation.Root({
docs: Annotation,
});
// This is what the node that retrieves the documents will return
const GenerateOutputAnnotation = Annotation.Root({
...OverallStateAnnotation.spec,
...DocumentOutputAnnotation.spec
});
// Node to generate query
const generateQuery = async (state: typeof OverallStateAnnotation.State) => {
// Replace this with real logic
return {
query: state.question + " rephrased as a query!",
};
};
// Node to retrieve documents
const retrieveDocuments = async (state: typeof QueryOutputAnnotation.State) => {
// Replace this with real logic
return {
docs: [state.query, "some random document"],
};
};
// Node to generate answer
const generate = async (state: typeof GenerateOutputAnnotation.State) => {
return {
answer: state.docs.concat([state.question]).join("\n\n"),
};
};
const graph = new StateGraph(OverallStateAnnotation)
.addNode("generate_query", generateQuery)
.addNode("retrieve_documents", retrieveDocuments, { input: QueryOutputAnnotation })
.addNode("generate", generate, { input: GenerateOutputAnnotation })
.addEdge("__start__", "generate_query")
.addEdge("generate_query", "retrieve_documents")
.addEdge("retrieve_documents", "generate")
.compile();
await graph.invoke({
question: "How are you?",
});
```
```output
{
question: 'How are you?',
answer: 'How are you? rephrased as a query!\n\nsome random document\n\nHow are you?'
}
```
Above, the original `question` value in the input has been preserved, but that the `generate_query` node rephrased it, the `retrieve_documents` node added `"some random document"`, and finally the `generate` node combined the `docs` in the state with the original question to create an `answer`. The intermediate steps populated by the `input` annotations passed to the individual nodes are not present in the final output.
---
how-tos/branching.ipynb
---
# How to create branches for parallel node execution
LangGraph natively supports fan-out and fan-in using either regular edges or
conditionalEdges.
This lets you run nodes in parallel to speed up your total graph execution.
Below are some examples showing how to add create branching dataflows that work
for you.
## Setup
First, install LangGraph.js
```bash
yarn add @langchain/langgraph @langchain/core
```
This guide will use OpenAI's GPT-4o model. We will optionally set our API key
for [LangSmith tracing](https://smith.langchain.com/), which will give us
best-in-class observability.
```typescript
// process.env.OPENAI_API_KEY = "sk_...";
// Optional, add tracing in LangSmith
// process.env.LANGCHAIN_API_KEY = "ls__..."
// process.env.LANGCHAIN_CALLBACKS_BACKGROUND = "true";
process.env.LANGCHAIN_CALLBACKS_BACKGROUND = "true";
process.env.LANGCHAIN_TRACING_V2 = "true";
process.env.LANGCHAIN_PROJECT = "Branching: LangGraphJS";
```
```output
Branching: LangGraphJS
```
## Fan out, fan in
First, we will make a simple graph that branches out and back in. When merging
back in, the state updates from all branches are applied by your **reducer**
(the `aggregate` method below).
```typescript
import { END, START, StateGraph, Annotation } from "@langchain/langgraph";
const StateAnnotation = Annotation.Root({
aggregate: Annotation({
reducer: (x, y) => x.concat(y),
})
});
// Create the graph
const nodeA = (state: typeof StateAnnotation.State) => {
console.log(`Adding I'm A to ${state.aggregate}`);
return { aggregate: [`I'm A`] };
};
const nodeB = (state: typeof StateAnnotation.State) => {
console.log(`Adding I'm B to ${state.aggregate}`);
return { aggregate: [`I'm B`] };
};
const nodeC = (state: typeof StateAnnotation.State) => {
console.log(`Adding I'm C to ${state.aggregate}`);
return { aggregate: [`I'm C`] };
};
const nodeD = (state: typeof StateAnnotation.State) => {
console.log(`Adding I'm D to ${state.aggregate}`);
return { aggregate: [`I'm D`] };
};
const builder = new StateGraph(StateAnnotation)
.addNode("a", nodeA)
.addEdge(START, "a")
.addNode("b", nodeB)
.addNode("c", nodeC)
.addNode("d", nodeD)
.addEdge("a", "b")
.addEdge("a", "c")
.addEdge("b", "d")
.addEdge("c", "d")
.addEdge("d", END);
const graph = builder.compile();
```
```typescript
import * as tslab from "tslab";
const representation = graph.getGraph();
const image = await representation.drawMermaidPng();
const arrayBuffer = await image.arrayBuffer();
await tslab.display.png(new Uint8Array(arrayBuffer));
```
```typescript
// Invoke the graph
const baseResult = await graph.invoke({ aggregate: [] });
console.log("Base Result: ", baseResult);
```
```output
Adding I'm A to
Adding I'm B to I'm A
Adding I'm C to I'm A
Adding I'm D to I'm A,I'm B,I'm C
Base Result: { aggregate: [ "I'm A", "I'm B", "I'm C", "I'm D" ] }
```
## Conditional Branching
If your fan-out is not deterministic, you can use
addConditionalEdges
directly like this:
```typescript
const ConditionalBranchingAnnotation = Annotation.Root({
aggregate: Annotation({
reducer: (x, y) => x.concat(y),
}),
which: Annotation({
reducer: (x: string, y: string) => (y ?? x),
})
})
// Create the graph
const nodeA2 = (state: typeof ConditionalBranchingAnnotation.State) => {
console.log(`Adding I'm A to ${state.aggregate}`);
return { aggregate: [`I'm A`] };
};
const nodeB2 = (state: typeof ConditionalBranchingAnnotation.State) => {
console.log(`Adding I'm B to ${state.aggregate}`);
return { aggregate: [`I'm B`] };
};
const nodeC2 = (state: typeof ConditionalBranchingAnnotation.State) => {
console.log(`Adding I'm C to ${state.aggregate}`);
return { aggregate: [`I'm C`] };
};
const nodeD2 = (state: typeof ConditionalBranchingAnnotation.State) => {
console.log(`Adding I'm D to ${state.aggregate}`);
return { aggregate: [`I'm D`] };
};
const nodeE2 = (state: typeof ConditionalBranchingAnnotation.State) => {
console.log(`Adding I'm E to ${state.aggregate}`);
return { aggregate: [`I'm E`] };
};
// Define the route function
function routeCDorBC(state: typeof ConditionalBranchingAnnotation.State): string[] {
if (state.which === "cd") {
return ["c", "d"];
}
return ["b", "c"];
}
const builder2 = new StateGraph(ConditionalBranchingAnnotation)
.addNode("a", nodeA2)
.addEdge(START, "a")
.addNode("b", nodeB2)
.addNode("c", nodeC2)
.addNode("d", nodeD2)
.addNode("e", nodeE2)
// Add conditional edges
// Third parameter is to support visualizing the graph
.addConditionalEdges("a", routeCDorBC, ["b", "c", "d"])
.addEdge("b", "e")
.addEdge("c", "e")
.addEdge("d", "e")
.addEdge("e", END);
const graph2 = builder2.compile();
```
```typescript
import * as tslab from "tslab";
const representation2 = graph2.getGraph();
const image2 = await representation2.drawMermaidPng();
const arrayBuffer2 = await image2.arrayBuffer();
await tslab.display.png(new Uint8Array(arrayBuffer2));
```
```typescript
// Invoke the graph
let g2result = await graph2.invoke({ aggregate: [], which: "bc" });
console.log("Result 1: ", g2result);
```
```output
Adding I'm A to
Adding I'm B to I'm A
Adding I'm C to I'm A
Adding I'm E to I'm A,I'm B,I'm C
Result 1: { aggregate: [ "I'm A", "I'm B", "I'm C", "I'm E" ], which: 'bc' }
```
```typescript
g2result = await graph2.invoke({ aggregate: [], which: "cd" });
console.log("Result 2: ", g2result);
```
```output
Adding I'm A to
Adding I'm C to I'm A
Adding I'm D to I'm A
Adding I'm E to I'm A,I'm C,I'm D
``````output
Result 2: { aggregate: [ "I'm A", "I'm C", "I'm D", "I'm E" ], which: 'cd' }
```
## Stable Sorting
When fanned out, nodes are run in parallel as a single "superstep". The updates
from each superstep are all applied to the state in sequence once the superstep
has completed.
If you need consistent, predetermined ordering of updates from a parallel
superstep, you should write the outputs (along with an identifying key) to a
separate field in your state, then combine them in the "sink" node by adding
regular `edge`s from each of the fanout nodes to the rendezvous point.
For instance, suppose I want to order the outputs of the parallel step by
"reliability".
```typescript
type ScoredValue = {
value: string;
score: number;
};
const reduceFanouts = (left?: ScoredValue[], right?: ScoredValue[]) => {
if (!left) {
left = [];
}
if (!right || right?.length === 0) {
// Overwrite. Similar to redux.
return [];
}
return left.concat(right);
};
const StableSortingAnnotation = Annotation.Root({
aggregate: Annotation({
reducer: (x, y) => x.concat(y),
}),
which: Annotation({
reducer: (x: string, y: string) => (y ?? x),
}),
fanoutValues: Annotation({
reducer: reduceFanouts,
}),
})
class ParallelReturnNodeValue {
private _value: string;
private _score: number;
constructor(nodeSecret: string, score: number) {
this._value = nodeSecret;
this._score = score;
}
public call(state: typeof StableSortingAnnotation.State) {
console.log(`Adding ${this._value} to ${state.aggregate}`);
return { fanoutValues: [{ value: this._value, score: this._score }] };
}
}
// Create the graph
const nodeA3 = (state: typeof StableSortingAnnotation.State) => {
console.log(`Adding I'm A to ${state.aggregate}`);
return { aggregate: ["I'm A"] };
};
const nodeB3 = new ParallelReturnNodeValue("I'm B", 0.1);
const nodeC3 = new ParallelReturnNodeValue("I'm C", 0.9);
const nodeD3 = new ParallelReturnNodeValue("I'm D", 0.3);
const aggregateFanouts = (state: typeof StableSortingAnnotation.State) => {
// Sort by score (reversed)
state.fanoutValues.sort((a, b) => b.score - a.score);
return {
aggregate: state.fanoutValues.map((v) => v.value).concat(["I'm E"]),
fanoutValues: [],
};
};
// Define the route function
function routeBCOrCD(state: typeof StableSortingAnnotation.State): string[] {
if (state.which === "cd") {
return ["c", "d"];
}
return ["b", "c"];
}
const builder3 = new StateGraph(StableSortingAnnotation)
.addNode("a", nodeA3)
.addEdge(START, "a")
.addNode("b", nodeB3.call.bind(nodeB3))
.addNode("c", nodeC3.call.bind(nodeC3))
.addNode("d", nodeD3.call.bind(nodeD3))
.addNode("e", aggregateFanouts)
.addConditionalEdges("a", routeBCOrCD, ["b", "c", "d"])
.addEdge("b", "e")
.addEdge("c", "e")
.addEdge("d", "e")
.addEdge("e", END);
const graph3 = builder3.compile();
// Invoke the graph
let g3result = await graph3.invoke({ aggregate: [], which: "bc" });
console.log("Result 1: ", g3result);
```
```output
Adding I'm A to
Adding I'm B to I'm A
Adding I'm C to I'm A
Result 1: {
aggregate: [ "I'm A", "I'm C", "I'm B", "I'm E" ],
which: 'bc',
fanoutValues: []
}
```
Our aggregateFanouts "sink" node in this case took the mapped values and then
sorted them in a consistent way. Notice that, because it returns an empty array
for `fanoutValues`, our `reduceFanouts` reducer function decided to overwrite
the previous values in the state.
```typescript
let g3result2 = await graph3.invoke({ aggregate: [], which: "cd" });
console.log("Result 2: ", g3result2);
```
```output
Adding I'm A to
Adding I'm C to I'm A
Adding I'm D to I'm A
Result 2: {
aggregate: [ "I'm A", "I'm C", "I'm D", "I'm E" ],
which: 'cd',
fanoutValues: []
}
```
---
how-tos/streaming-tokens-without-langchain.ipynb
---
# How to stream LLM tokens (without LangChain models)
In this guide, we will stream tokens from the language model powering an agent without using LangChain chat models. We'll be using the OpenAI client library directly in a ReAct agent as an example.
## Setup
To get started, install the `openai` and `langgraph` packages separately:
```bash
$ npm install openai @langchain/langgraph @langchain/core
```
Compatibility
This guide requires @langchain/core>=0.2.19, and if you are using LangSmith, langsmith>=0.1.39. For help upgrading, see this guide.
You'll also need to make sure you have your OpenAI key set as `process.env.OPENAI_API_KEY`.
## Defining a model and a tool schema
First, initialize the OpenAI SDK and define a tool schema for the model to populate using [OpenAI's format](https://platform.openai.com/docs/api-reference/chat/create#chat-create-tools):
```typescript
import OpenAI from "openai";
const openaiClient = new OpenAI({});
const toolSchema: OpenAI.ChatCompletionTool = {
type: "function",
function: {
name: "get_items",
description: "Use this tool to look up which items are in the given place.",
parameters: {
type: "object",
properties: {
place: {
type: "string",
},
},
required: ["place"],
}
}
};
```
## Calling the model
Now, define a method for a LangGraph node that will call the model. It will handle formatting tool calls to and from the model, as well as streaming via [custom callback events](https://js.langchain.com/docs/how_to/callbacks_custom_events).
If you are using [LangSmith](https://docs.smith.langchain.com/), you can also wrap the OpenAI client for the same nice tracing you'd get with a LangChain chat model.
Here's what that looks like:
```typescript
import { dispatchCustomEvent } from "@langchain/core/callbacks/dispatch";
import { wrapOpenAI } from "langsmith/wrappers/openai";
import { Annotation } from "@langchain/langgraph";
const StateAnnotation = Annotation.Root({
messages: Annotation({
reducer: (x, y) => x.concat(y),
}),
});
// If using LangSmith, use "wrapOpenAI" on the whole client or
// "traceable" to wrap a single method for nicer tracing:
// https://docs.smith.langchain.com/how_to_guides/tracing/annotate_code
const wrappedClient = wrapOpenAI(openaiClient);
const callModel = async (state: typeof StateAnnotation.State) => {
const { messages } = state;
const stream = await wrappedClient.chat.completions.create({
messages,
model: "gpt-4o-mini",
tools: [toolSchema],
stream: true,
});
let responseContent = "";
let role: string = "assistant";
let toolCallId: string | undefined;
let toolCallName: string | undefined;
let toolCallArgs = "";
for await (const chunk of stream) {
const delta = chunk.choices[0].delta;
if (delta.role !== undefined) {
role = delta.role;
}
if (delta.content) {
responseContent += delta.content;
await dispatchCustomEvent("streamed_token", {
content: delta.content,
});
}
if (delta.tool_calls !== undefined && delta.tool_calls.length > 0) {
// note: for simplicity we're only handling a single tool call here
const toolCall = delta.tool_calls[0];
if (toolCall.function?.name !== undefined) {
toolCallName = toolCall.function.name;
}
if (toolCall.id !== undefined) {
toolCallId = toolCall.id;
}
await dispatchCustomEvent("streamed_tool_call_chunk", toolCall);
toolCallArgs += toolCall.function?.arguments ?? "";
}
}
let finalToolCalls;
if (toolCallName !== undefined && toolCallId !== undefined) {
finalToolCalls = [{
id: toolCallId,
function: {
name: toolCallName,
arguments: toolCallArgs
},
type: "function" as const,
}];
}
const responseMessage = {
role: role as any,
content: responseContent,
tool_calls: finalToolCalls,
};
return { messages: [responseMessage] };
}
```
Note that you can't call this method outside of a LangGraph node since `dispatchCustomEvent` will fail if it is called outside the proper context.
## Define tools and a tool-calling node
Next, set up the actual tool function and the node that will call it when the model populates a tool call:
```typescript
const getItems = async ({ place }: { place: string }) => {
if (place.toLowerCase().includes("bed")) { // For under the bed
return "socks, shoes and dust bunnies";
} else if (place.toLowerCase().includes("shelf")) { // For 'shelf'
return "books, pencils and pictures";
} else { // if the agent decides to ask about a different place
return "cat snacks";
}
};
const callTools = async (state: typeof StateAnnotation.State) => {
const { messages } = state;
const mostRecentMessage = messages[messages.length - 1];
const toolCalls = (mostRecentMessage as OpenAI.ChatCompletionAssistantMessageParam).tool_calls;
if (toolCalls === undefined || toolCalls.length === 0) {
throw new Error("No tool calls passed to node.");
}
const toolNameMap = {
get_items: getItems,
};
const functionName = toolCalls[0].function.name;
const functionArguments = JSON.parse(toolCalls[0].function.arguments);
const response = await toolNameMap[functionName](functionArguments);
const toolMessage = {
tool_call_id: toolCalls[0].id,
role: "tool" as const,
name: functionName,
content: response,
}
return { messages: [toolMessage] };
}
```
## Build the graph
Finally, it's time to build your graph:
```typescript
import { StateGraph } from "@langchain/langgraph";
import OpenAI from "openai";
// We can reuse the same `GraphState` from above as it has not changed.
const shouldContinue = (state: typeof StateAnnotation.State) => {
const { messages } = state;
const lastMessage =
messages[messages.length - 1] as OpenAI.ChatCompletionAssistantMessageParam;
if (lastMessage?.tool_calls !== undefined && lastMessage?.tool_calls.length > 0) {
return "tools";
}
return "__end__";
}
const graph = new StateGraph(StateAnnotation)
.addNode("model", callModel)
.addNode("tools", callTools)
.addEdge("__start__", "model")
.addConditionalEdges("model", shouldContinue, {
tools: "tools",
__end__: "__end__",
})
.addEdge("tools", "model")
.compile();
```
```typescript
import * as tslab from "tslab";
const representation = graph.getGraph();
const image = await representation.drawMermaidPng();
const arrayBuffer = await image.arrayBuffer();
await tslab.display.png(new Uint8Array(arrayBuffer));
```
## Streaming tokens
And now we can use the [`.streamEvents`](https://js.langchain.com/docs/how_to/streaming#using-stream-events) method to get the streamed tokens and tool calls from the OpenAI model:
```typescript
const eventStream = await graph.streamEvents(
{ messages: [{ role: "user", content: "what's in the bedroom?" }] },
{ version: "v2" },
);
for await (const { event, name, data } of eventStream) {
if (event === "on_custom_event") {
console.log(name, data);
}
}
```
```output
streamed_tool_call_chunk {
index: 0,
id: 'call_v99reml4gZvvUypPgOpLgxM2',
type: 'function',
function: { name: 'get_items', arguments: '' }
}
streamed_tool_call_chunk { index: 0, function: { arguments: '{"' } }
streamed_tool_call_chunk { index: 0, function: { arguments: 'place' } }
streamed_tool_call_chunk { index: 0, function: { arguments: '":"' } }
streamed_tool_call_chunk { index: 0, function: { arguments: 'bed' } }
streamed_tool_call_chunk { index: 0, function: { arguments: 'room' } }
streamed_tool_call_chunk { index: 0, function: { arguments: '"}' } }
streamed_token { content: 'In' }
streamed_token { content: ' the' }
streamed_token { content: ' bedroom' }
streamed_token { content: ',' }
streamed_token { content: ' you' }
streamed_token { content: ' can' }
streamed_token { content: ' find' }
streamed_token { content: ' socks' }
streamed_token { content: ',' }
streamed_token { content: ' shoes' }
streamed_token { content: ',' }
streamed_token { content: ' and' }
streamed_token { content: ' dust' }
streamed_token { content: ' b' }
streamed_token { content: 'unnies' }
streamed_token { content: '.' }
```
And if you've set up LangSmith tracing, you'll also see [a trace like this one](https://smith.langchain.com/public/ddb1af36-ebe5-4ba6-9a57-87a296dc801f/r).
---
how-tos/use-in-web-environments.ipynb
---
# How to use LangGraph.js in web environments
LangGraph.js uses the [`async_hooks`](https://nodejs.org/api/async_hooks.html)
API to more conveniently allow for tracing and callback propagation within
nodes. This API is supported in many environments, such as
[Node.js](https://nodejs.org/api/async_hooks.html),
[Deno](https://deno.land/std@0.177.0/node/internal/async_hooks.ts),
[Cloudflare Workers](https://developers.cloudflare.com/workers/runtime-apis/nodejs/asynclocalstorage/),
and the
[Edge runtime](https://vercel.com/docs/functions/runtimes/edge-runtime#compatible-node.js-modules),
but not all, such as within web browsers.
To allow usage of LangGraph.js in environments that do not have the
`async_hooks` API available, we've added a separate `@langchain/langgraph/web`
entrypoint. This entrypoint exports everything that the primary
`@langchain/langgraph` exports, but will not initialize or even import
`async_hooks`. Here's a simple example:
```typescript
// Import from "@langchain/langgraph/web"
import {
END,
START,
StateGraph,
Annotation,
} from "@langchain/langgraph/web";
import { BaseMessage, HumanMessage } from "@langchain/core/messages";
const GraphState = Annotation.Root({
messages: Annotation({
reducer: (x, y) => x.concat(y),
}),
});
const nodeFn = async (_state: typeof GraphState.State) => {
return { messages: [new HumanMessage("Hello from the browser!")] };
};
// Define a new graph
const workflow = new StateGraph(GraphState)
.addNode("node", nodeFn)
.addEdge(START, "node")
.addEdge("node", END);
const app = workflow.compile({});
// Use the Runnable
const finalState = await app.invoke(
{ messages: [] },
);
console.log(finalState.messages[finalState.messages.length - 1].content);
```
```output
Hello from the browser!
```
Other entrypoints, such as `@langchain/langgraph/prebuilt`, can be used in
either environment.
Caution
If you are using LangGraph.js on the frontend, make sure you are not exposing any private keys!
For chat models, this means you need to use something like WebLLM
that can run client-side without authentication.
## Passing config
The lack of `async_hooks` support in web browsers means that if you are calling
a [`Runnable`](https://js.langchain.com/docs/concepts/runnables/) within a
node (for example, when calling a chat model), you need to manually pass a
`config` object through to properly support tracing,
[`.streamEvents()`](https://js.langchain.com/docs/how_to/streaming#using-stream-events)
to stream intermediate steps, and other callback related functionality. This
config object will passed in as the second argument of each node, and should be
used as the second parameter of any `Runnable` method.
To illustrate this, let's set up our graph again as before, but with a
`Runnable` within our node. First, we'll avoid passing `config` through into the
nested function, then try to use `.streamEvents()` to see the intermediate
results of the nested function:
```typescript
// Import from "@langchain/langgraph/web"
import {
END,
START,
StateGraph,
Annotation,
} from "@langchain/langgraph/web";
import { BaseMessage } from "@langchain/core/messages";
import { RunnableLambda } from "@langchain/core/runnables";
import { type StreamEvent } from "@langchain/core/tracers/log_stream";
const GraphState2 = Annotation.Root({
messages: Annotation({
reducer: (x, y) => x.concat(y),
}),
});
const nodeFn2 = async (_state: typeof GraphState2.State) => {
// Note that we do not pass any `config` through here
const nestedFn = RunnableLambda.from(async (input: string) => {
return new HumanMessage(`Hello from ${input}!`);
}).withConfig({ runName: "nested" });
const responseMessage = await nestedFn.invoke("a nested function");
return { messages: [responseMessage] };
};
// Define a new graph
const workflow2 = new StateGraph(GraphState2)
.addNode("node", nodeFn2)
.addEdge(START, "node")
.addEdge("node", END);
const app2 = workflow2.compile({});
// Stream intermediate steps from the graph
const eventStream2 = app2.streamEvents(
{ messages: [] },
{ version: "v2" },
{ includeNames: ["nested"] },
);
const events2: StreamEvent[] = [];
for await (const event of eventStream2) {
console.log(event);
events2.push(event);
}
console.log(`Received ${events2.length} events from the nested function`);
```
```output
Received 0 events from the nested function
```
We can see that we get no events.
Next, let's try redeclaring the graph with a node that passes config through
correctly:
```typescript
// Import from "@langchain/langgraph/web"
import {
END,
START,
StateGraph,
Annotation,
} from "@langchain/langgraph/web";
import { BaseMessage } from "@langchain/core/messages";
import { type RunnableConfig, RunnableLambda } from "@langchain/core/runnables";
import { type StreamEvent } from "@langchain/core/tracers/log_stream";
const GraphState3 = Annotation.Root({
messages: Annotation({
reducer: (x, y) => x.concat(y),
}),
});
// Note the second argument here.
const nodeFn3 = async (_state: typeof GraphState3.State, config?: RunnableConfig) => {
// If you need to nest deeper, remember to pass `_config` when invoking
const nestedFn = RunnableLambda.from(
async (input: string, _config?: RunnableConfig) => {
return new HumanMessage(`Hello from ${input}!`);
},
).withConfig({ runName: "nested" });
const responseMessage = await nestedFn.invoke("a nested function", config);
return { messages: [responseMessage] };
};
// Define a new graph
const workflow3 = new StateGraph(GraphState3)
.addNode("node", nodeFn3)
.addEdge(START, "node")
.addEdge("node", END);
const app3 = workflow3.compile({});
// Stream intermediate steps from the graph
const eventStream3 = app3.streamEvents(
{ messages: [] },
{ version: "v2" },
{ includeNames: ["nested"] },
);
const events3: StreamEvent[] = [];
for await (const event of eventStream3) {
console.log(event);
events3.push(event);
}
console.log(`Received ${events3.length} events from the nested function`);
```
```output
{
event: "on_chain_start",
data: { input: { messages: [] } },
name: "nested",
tags: [],
run_id: "22747451-a2fa-447b-b62f-9da19a539b2f",
metadata: {
langgraph_step: 1,
langgraph_node: "node",
langgraph_triggers: [ "start:node" ],
langgraph_task_idx: 0,
__pregel_resuming: false,
checkpoint_id: "1ef62793-f065-6840-fffe-cdfb4cbb1248",
checkpoint_ns: "node"
}
}
{
event: "on_chain_end",
data: {
output: HumanMessage {
"content": "Hello from a nested function!",
"additional_kwargs": {},
"response_metadata": {}
}
},
run_id: "22747451-a2fa-447b-b62f-9da19a539b2f",
name: "nested",
tags: [],
metadata: {
langgraph_step: 1,
langgraph_node: "node",
langgraph_triggers: [ "start:node" ],
langgraph_task_idx: 0,
__pregel_resuming: false,
checkpoint_id: "1ef62793-f065-6840-fffe-cdfb4cbb1248",
checkpoint_ns: "node"
}
}
Received 2 events from the nested function
```
You can see that we get events from the nested function as expected.
## Next steps
You've now learned about some special considerations around using LangGraph.js
in web environments.
Next, check out
some how-to guides on core functionality.
---
how-tos/map-reduce.ipynb
---
# How to create map-reduce branches for parallel execution
[Map-reduce](https://en.wikipedia.org/wiki/MapReduce) operations are essential for efficient task decomposition and parallel processing. This approach involves breaking a task into smaller sub-tasks, processing each sub-task in parallel, and aggregating the results across all of the completed sub-tasks.
Consider this example: given a general topic from the user, generate a list of related subjects, generate a joke for each subject, and select the best joke from the resulting list. In this design pattern, a first node may generate a list of objects (e.g., related subjects) and we want to apply some other node (e.g., generate a joke) to all those objects (e.g., subjects). However, two main challenges arise.
(1) the number of objects (e.g., subjects) may be unknown ahead of time (meaning the number of edges may not be known) when we lay out the graph and (2) the input State to the downstream Node should be different (one for each generated object).
LangGraph addresses these challenges through its `Send` API. By utilizing conditional edges, `Send` can distribute different states (e.g., subjects) to multiple instances of a node (e.g., joke generation). Importantly, the sent state can differ from the core graph's state, allowing for flexible and dynamic workflow management.
## Setup
This example will require a few dependencies. First, install the LangGraph library, along with the `@langchain/anthropic` package as we'll be using Anthropic LLMs in this example:
```bash
npm install @langchain/langgraph @langchain/anthropic @langchain/core
```
Next, set your Anthropic API key:
```typescript
process.env.ANTHROPIC_API_KEY = 'YOUR_API_KEY'
```
```typescript
import { z } from "zod";
import { ChatAnthropic } from "@langchain/anthropic";
import { StateGraph, END, START, Annotation, Send } from "@langchain/langgraph";
/* Model and prompts */
// Define model and prompts we will use
const subjectsPrompt = "Generate a comma separated list of between 2 and 5 examples related to: {topic}."
const jokePrompt = "Generate a joke about {subject}"
const bestJokePrompt = `Below are a bunch of jokes about {topic}. Select the best one! Return the ID (index) of the best one.
{jokes}`
// Zod schemas for getting structured output from the LLM
const Subjects = z.object({
subjects: z.array(z.string()),
});
const Joke = z.object({
joke: z.string(),
});
const BestJoke = z.object({
id: z.number(),
});
const model = new ChatAnthropic({
model: "claude-3-5-sonnet-20240620",
});
/* Graph components: define the components that will make up the graph */
// This will be the overall state of the main graph.
// It will contain a topic (which we expect the user to provide)
// and then will generate a list of subjects, and then a joke for
// each subject
const OverallState = Annotation.Root({
topic: Annotation,
subjects: Annotation,
// Notice here we pass a reducer function.
// This is because we want combine all the jokes we generate
// from individual nodes back into one list.
jokes: Annotation({
reducer: (state, update) => state.concat(update),
}),
bestSelectedJoke: Annotation,
});
// This will be the state of the node that we will "map" all
// subjects to in order to generate a joke
interface JokeState {
subject: string;
}
// This is the function we will use to generate the subjects of the jokes
const generateTopics = async (
state: typeof OverallState.State
): Promise> => {
const prompt = subjectsPrompt.replace("topic", state.topic);
const response = await model
.withStructuredOutput(Subjects, { name: "subjects" })
.invoke(prompt);
return { subjects: response.subjects };
};
// Function to generate a joke
const generateJoke = async (state: JokeState): Promise<{ jokes: string[] }> => {
const prompt = jokePrompt.replace("subject", state.subject);
const response = await model
.withStructuredOutput(Joke, { name: "joke" })
.invoke(prompt);
return { jokes: [response.joke] };
};
// Here we define the logic to map out over the generated subjects
// We will use this an edge in the graph
const continueToJokes = (state: typeof OverallState.State) => {
// We will return a list of `Send` objects
// Each `Send` object consists of the name of a node in the graph
// as well as the state to send to that node
return state.subjects.map((subject) => new Send("generateJoke", { subject }));
};
// Here we will judge the best joke
const bestJoke = async (
state: typeof OverallState.State
): Promise> => {
const jokes = state.jokes.join("\n\n");
const prompt = bestJokePrompt
.replace("jokes", jokes)
.replace("topic", state.topic);
const response = await model
.withStructuredOutput(BestJoke, { name: "best_joke" })
.invoke(prompt);
return { bestSelectedJoke: state.jokes[response.id] };
};
// Construct the graph: here we put everything together to construct our graph
const graph = new StateGraph(OverallState)
.addNode("generateTopics", generateTopics)
.addNode("generateJoke", generateJoke)
.addNode("bestJoke", bestJoke)
.addEdge(START, "generateTopics")
.addConditionalEdges("generateTopics", continueToJokes)
.addEdge("generateJoke", "bestJoke")
.addEdge("bestJoke", END);
const app = graph.compile();
```
```typescript
import * as tslab from "tslab";
const representation = app.getGraph();
const image = await representation.drawMermaidPng();
const arrayBuffer = await image.arrayBuffer();
tslab.display.png(new Uint8Array(arrayBuffer));
```
```typescript
// Call the graph: here we call it to generate a list of jokes
for await (const s of await app.stream({ topic: "animals" })) {
console.log(s);
}
```
```output
{
generateTopics: { subjects: [ 'lion', 'elephant', 'penguin', 'dolphin' ] }
}
{
generateJoke: {
jokes: [ "Why don't lions like fast food? Because they can't catch it!" ]
}
}
{
generateJoke: {
jokes: [
"Why don't elephants use computers? Because they're afraid of the mouse!"
]
}
}
{
generateJoke: {
jokes: [
"Why don't dolphins use smartphones? They're afraid of phishing!"
]
}
}
{
generateJoke: {
jokes: [
"Why don't you see penguins in Britain? Because they're afraid of Wales!"
]
}
}
{
bestJoke: {
bestSelectedJoke: "Why don't elephants use computers? Because they're afraid of the mouse!"
}
}
```