{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "51466c8d-8ce4-4b3d-be4e-18fdbeda5f53",
   "metadata": {},
   "source": [
    "# How to Review Tool Calls\n",
    "\n",
    "Human-in-the-loop (HIL) interactions are crucial for [agentic systems](https://langchain-ai.github.io/langgraph/concepts/agentic_concepts/#human-in-the-loop). A common pattern is to add some human in the loop step after certain tool calls. These tool calls often lead to either a function call or saving of some information. Examples include:\n",
    "\n",
    "- A tool call to execute SQL, which will then be run by the tool\n",
    "- A tool call to generate a summary, which will then be saved to the State of the graph\n",
    "\n",
    "Note that using tool calls is common **whether actually calling tools or not**.\n",
    "\n",
    "There are typically a few different interactions you may want to do here:\n",
    "\n",
    "1. Approve the tool call and continue\n",
    "2. Modify the tool call manually and then continue\n",
    "3. Give natural language feedback, and then pass that back to the agent instead of continuing\n",
    "\n",
    "We can implement this in LangGraph using a [breakpoint](https://langchain-ai.github.io/langgraph/how-tos/human_in_the_loop/breakpoints/): breakpoints allow us to interrupt graph execution before a specific step. At this breakpoint, we can manually update the graph state taking one of the three options above"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7cbd446a-808f-4394-be92-d45ab818953c",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "First we need to install the packages required"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "af4ce0ba-7596-4e5f-8bf8-0b0bd6e62833",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture --no-stderr\n",
    "%pip install --quiet -U langgraph langchain_anthropic"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0abe11f4-62ed-4dc4-8875-3db21e260d1d",
   "metadata": {},
   "source": [
    "Next, we need to set API keys for Anthropic (the LLM we will use)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "c903a1cf-2977-4e2d-ad7d-8b3946821d89",
   "metadata": {},
   "outputs": [],
   "source": [
    "import getpass\n",
    "import os\n",
    "\n",
    "\n",
    "def _set_env(var: str):\n",
    "    if not os.environ.get(var):\n",
    "        os.environ[var] = getpass.getpass(f\"{var}: \")\n",
    "\n",
    "\n",
    "_set_env(\"ANTHROPIC_API_KEY\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0ed46a8-effe-4596-b0e1-a6a29ee16f5c",
   "metadata": {},
   "source": [
    "<div class=\"admonition tip\">\n",
    "    <p class=\"admonition-title\">Set up <a href=\"https://smith.langchain.com\">LangSmith</a> for LangGraph development</p>\n",
    "    <p style=\"padding-top: 5px;\">\n",
    "        Sign up for LangSmith to quickly spot issues and improve the performance of your LangGraph projects. LangSmith lets you use trace data to debug, test, and monitor your LLM apps built with LangGraph — read more about how to get started <a href=\"https://docs.smith.langchain.com\">here</a>. \n",
    "    </p>\n",
    "</div>    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "035e567c-db5c-4085-ba4e-5b3814561c21",
   "metadata": {},
   "source": [
    "## Simple Usage\n",
    "\n",
    "Let's set up a very simple graph that facilitates this.\n",
    "First, we will have an LLM call that decides what action to take.\n",
    "Then we go to a human node. This node actually doesn't do anything - the idea is that we interrupt before this node and then apply any updates to the state.\n",
    "After that, we check the state and either route back to the LLM or to the correct tool.\n",
    "\n",
    "Let's see this in action!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "85e452f8-f33a-4ead-bb4d-7386cdba8edc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/4gHYSUNDX1BST0ZJTEUAAQEAAAHIAAAAAAQwAABtbnRyUkdCIFhZWiAH4AABAAEAAAAAAABhY3NwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAA9tYAAQAAAADTLQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAlkZXNjAAAA8AAAACRyWFlaAAABFAAAABRnWFlaAAABKAAAABRiWFlaAAABPAAAABR3dHB0AAABUAAAABRyVFJDAAABZAAAAChnVFJDAAABZAAAAChiVFJDAAABZAAAAChjcHJ0AAABjAAAADxtbHVjAAAAAAAAAAEAAAAMZW5VUwAAAAgAAAAcAHMAUgBHAEJYWVogAAAAAAAAb6IAADj1AAADkFhZWiAAAAAAAABimQAAt4UAABjaWFlaIAAAAAAAACSgAAAPhAAAts9YWVogAAAAAAAA9tYAAQAAAADTLXBhcmEAAAAAAAQAAAACZmYAAPKnAAANWQAAE9AAAApbAAAAAAAAAABtbHVjAAAAAAAAAAEAAAAMZW5VUwAAACAAAAAcAEcAbwBvAGcAbABlACAASQBuAGMALgAgADIAMAAxADb/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsKCwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRT/2wBDAQMEBAUEBQkFBQkUDQsNFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBT/wAARCAGGAWoDASIAAhEBAxEB/8QAHQABAAMBAAMBAQAAAAAAAAAAAAUGBwQCAwgBCf/EAF8QAAEEAQIDAQgMCAoFBw0BAAEAAgMEBQYRBxIhExQVFiIxQVaUCBcyNlFUYXST0dPUI1VxdZWytNIkNUJSY4GRkqGzNDdyseEJJTNigsLwGCYnQ0RFRldzdoWio8H/xAAbAQEBAQADAQEAAAAAAAAAAAAAAQIDBAUGB//EADYRAQABAgIGCQIFBQEBAAAAAAABAhEDURIhMUGR0QQTFDNSYXGhwZKxBRUiYoEjMkPh8ILC/9oADAMBAAIRAxEAPwD+qaIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgLmuZKpjwDatQVgfIZpAz/eVAunuaxklZTsy43CRuMZuQECe44HxuzJB5Ih1HOPGcdy3lAa5/TV0Hp2o4vbhaUkpJc6aeESyuJ85e/dx/rK7GhRR3k68o+f8Ap81tba6fCrCfjih60z608KsJ+OKHrTPrX74LYX8UUPVmfUngthfxRQ9WZ9Sf0fP2XU/PCrCfjih60z608KsJ+OKHrTPrX74LYX8UUPVmfUngthfxRQ9WZ9Sf0fP2NT88KsJ+OKHrTPrTwqwn44oetM+tfvgthfxRQ9WZ9SeC2F/FFD1Zn1J/R8/Y1HhVhT/74oetM+td1a3Bdj7SvNHPH/OieHD+0LhGl8MD/FFD1Zn1LhscPtPyydtBjIcbbAPLbxw7mmH/AG2bE/kO4+RLYM75jhPJNSxIq9QyF3C34cZlpnXI5/FqZNzWtMrgN+zlDQGiTbcgtAa7Y9GkbGwriromiQREWEEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQFXtd3Jq+ANevIYbF+eGiyQEgs7WQMc4EdQQ0uI+UBWFVjiAOxxVG8d+zoZGtZk2G+zO0DXn8ga8k/ICufAiJxab5rG1YadSHH1IKtaJsNeBjYo42DZrGgbAD5AAvcip2d4y8P9LZafF5rXOmsRk6+wmpX8vXgmj3aHDmY54I3BBG48hBXDM31yi4rOdd8b8dojWFXS8WA1BqbNy0Tk5auBpsndWq9p2YlfzPZuC8EBrOZx2PRe7/AMoXhX/8y9H/AKeq/aLLeOtK7xdr0crwzwMOpcrDVfHh9f6e1JXg72WefZ0cpDt5YRs0uYOcHcjkB6qC3YPjVncj7IbVWhJNJ5KXDY2rQfDka7IAyEyicvlmc6fmMbuza1nIwu3a/mAGxMri+PmOt65o6YyOmNT6clyU81XG5HM49sNS9LG1z3MjcHucCWsc5vO1vMG9N1BU9O610bx5yGeiwA1HiNTYnF0buTq24YO909Z8wkkdFI4Oewtn5hybnxdtvOsi0lwN1vS1Hw7yeT0AybU+C1CLmodY2MxBPYysb2zRGSEF3P2TRK2Qxv5C0MDWMcUGyReySr56lq2TTmjtS5U6emyNKzb7mrsrNtVecFnM+dpeHFoI5Qejhzcp6Cd9j7xLyvFfhdgNQZnA3cJftUK08j7DImw23Pia90tcMlkIiJJ25+V23lCjOEXDvMYPQWuMNl6wx1jMagzlqAmRkm8FmzI6KTxCfK1wOx2I8hAKh+DWtH8IeGGnNL8UI8Vw/nwtCviql3KZyoIcr2LOR8kHjhwADYyQ8AjtB06INzRUAeyC4XFheOJOkCwEAu7+1dgTvsP+k+Q/2KZ0txO0drm3NV03qzB6gtQs7WWDFZKGy9jNwOZzWOJA3IG5+FBJamw/f7BW6TSGTPbzwSn/ANVM0h0Ug+Vrw139S/dMZkah05i8oGhndtWKxyD+SXNBI/q32XVkr8OKx1q7YJbBWifNIQNyGtBJ/wAAorQeOlxGisHTsAtsRUomytI22fyjmG35d12NuDN89XDX8LuTyIi66CIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiIC9NupDfqTVrEbZq8zHRyRvG7XtI2IPyEFe5FYm2uBWcVk3aafDhsvMQ0bR0chKTyWWb7Nje49BMBsCCfH903+U1k/JRrSvL314nvPlc5gJK/blOvkKsta1BHZrSt5ZIZmB7Hj4CD0IVedoCrD0oZPL4tm+/ZVrz3Rj8jJOYNHyAAfIue+Hia6ptPt/prVKe72Ux/7JB9GPqXuiiZAzljY2Nv81o2CrPgRP6U576eL7JPAif0pz308X2SdXh+P2ktGa0oqt4ET+lOe+ni+yVUxONyt3ifqbAyapzHe/H4vHW4OWaLtO0nkuNk5vwfk2gj26D+V5fM6vD8ftJaM2qL1zVorG3axMl28nO0HZVrwIn9Kc99PF9kngRP6U576eL7JOrw/H7SWjNYO9tT4rD9GPqXkyvXph0jY4oABu5waG9PlKrw0ROCD4U547eYzxfZryj4fY2R7XZCe/muU7iPI23yxf1xbhh/raU0MKNtfCOdktD1TzM15LHXrbSaeikbLPbB8W45rg5scf86PcAvd7k7cg5t38trX41oY0NaA1oGwA8gX6uOuvStEaogmRERcaCIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICz7Txb7e2uBuebvFhdx5tu2yO3n/L5v6z5tBWfae39vXXHVu3eLC9ABv/02R8vn/t6eXbzoNBREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQFnunQPb41yeZpPeHC+Lt1H4bJdfJ/42K0JZ5p3b2+dc9Tv3hwvTb+myXn/wDH+KDQ0REBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBEXi97Y2lziGtaNySdgAg8kVKOr85lgLGFxtHva/rDPkLEjJJm+Z4jaw8rT5RudyPKAvHv5rD4jg/Wpvs13Oy4m+0fzC2XdFSO/msPiOD9am+zTv5rD4jg/Wpvs07LXnHGCy25We3WxdyahVZevRwvfXqyS9k2aQNJawv2PKCdhzbHbffYr4V4S+zwta39ki7DR8NLVTI6hdQwk8TsmHPoivLYMsrh2ALg1s7iWkjbsz1G5X17381h8RwfrU32ayDSnsfZdIcetR8VKdDDHL5iDk7lM0nZVpXbdtMw9nvzSbDf8r/AOd0dlrzjjBZ9LIqR381h8RwfrU32ad/NYfEcH61N9mnZa844wWXdFSO/msPiOD9am+zTv5rD4jg/Wpvs07LXnHGCy7oqlQ1ZkqluCHO0ateGxI2GO3SndIxsjiA1r2uY0t5idgQSN9gdtwrauDEw6sObVFrCIi4kEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQFE6tcW6VzJB2IpTEEf/TKllEav96ea+ZT/wCW5cuF3lPrCxtQenQBp/GAAAdyxdB/sBSCj9PfxBjPmsX6gUgvRr/uknaIiLCCKoVOLek72DweYgyvPjs3kTicfN3NKO2tB8jOz5SzdvjQyDmcA3xfL1G9vU2giq1/ifpnFjVps5LshpSBtnM/gJT3LG6EzB3Rvj/gwXbM5j5vL0Vio3YclSr267+0rzxtljfsRzNcNwdj1HQ+dB70RFRX9dnbTjj5xaqEfIe6I1oazvXnvbf86q/tEa0RcfSO6o9Z+GtwiIugyIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgKI1f70818yn/y3KXURq/3p5r5lP8A5bly4XeU+sLG1B6e/iDGfNYv1AvZmb0mMw965FCbMteCSVkLfLIWtJDR+XbZevT38QYz5rF+oFIL0a/7pJ2sD4G4yXL8PtP8VtQ641Dkb93Huy1+uzIuGNY10bnOhbVb4gbF5BsObmZ1J6hUjhZqLVOP4rcN7rLGo2aS1rXuvji1LqHvjNZjbW7ohm7nEYZVdsAdo3kbP2IBC3DEex+0BgdSHO4/TzKt7tZJxGyzN3MySRrmyObX5+xaXB7gSGDfmPwr16f9jtw90vlcbksZp81r2Mm7ahObth7qniuaWRc0h5Iy17gYm7MO/VvQbdfRnUjB9KEe0hwP6+TiM8H8vduQX1+qDb4DaDu4PL4aXT0XezK3++lmuyeVgFrfftoy14MLt+u8fL1J+Er8t4rigbU3cmpdIxVOd3YxzaetPe1m/ihzheAcQNtyAN/gC1ETSMa177j2Wg8/eKudv/xL1OT0bmr+KWgtMO1DnMVg5tDS35q+GyMlTtpWTVWMcXMIcNhIerSCdtiS0kHVq3C3CW8pYz2ZxtG7qXI4rvTlbdaOSKC7CQOZjoC9zS3zDmLnBpLebYnfz0vwl0poy3ibWHxZqT4rHy4qk82ppOxqyStlfGA95BHOxpBO5AAAIHRTRkfOOgMlqLGaJ4Taym1jqLKZXK6sOBvRX8g6WrYqdvZrhpg6M5wIWO7TbnLtySd+n16qhV4SaTpYHCYWHFcmNwuR77UIO6ZT2NrtHydpzF+7vHleeVxLfG222A2t6tMWFe15723/ADqr+0RrRFnevPe2/wCdVf2iNaIp0juqPWf/AJa3CIi6DIiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiKDj1thbNuhXqXm5B96SaGF9BjrEfPEN5A+SMFsfLtt45b42zfKQEE4irdHN57MMoTQ4A4mrYglfN33stFmvICRE3soudrg73R/CNLRsNubcBDpvK3oa5zOoZ5ZO4pK1qviohSrzSP8szer5o3NHRvLN06k7nYgJnJ5ejhak9rIXIKVaCF9iWWxIGNZGwbveST0a0dSfIPOoaXWsdiObvRjMhmpRRZfgMEPZQWGv8AcMZPKWxlxHUjm3A6nbcA9mL0hhsNPVs1sfD3dWqNoR35wZrfYA7iN0795HDfxjzOO53J69VMIK3bi1TlGXooZsfgo5K8XctkNdbmjlPWTnYeVmwG7W7E7nqf5qjtaaPrXtNaqfkLl3Iw3KQJqzzkQRGFhc0xsbttzOHM7ffm8h8XorbkbMlLH2bENSa/NDE6RlSuWCSZwBIYwvc1gc49BzOaNz1IHVfEmF/5TCjxB1DU0rg+FuXymTykopQ1H5GOMvc7xSCeQ8o8u5PkAJPkW6J0aoqyWH1hp7+IMZ81i/UCkFBRVtQabrxY9uEmzcFdjYoblSxCxz2AbAyNle3Z2w67Eg+XpvsPLvtn/Q3J+tU/t161VMVTNUVRafOOa2TaKE77Z/0NyfrVP7dO+2f9Dcn61T+3Wer/AHR9VPMsm0UJ32z/AKG5P1qn9unfbP8Aobk/Wqf26dX+6Pqp5lk2irGH1Xl87iaeRqaNzHc1uFs8XbyVYZOVw3HMx8wcw9erXAEHoQCuzvtn/Q3J+tU/t06v90fVTzLJtFCd9s/6G5P1qn9unfbP+huT9ap/bp1f7o+qnmWenXnvbf8AOqv7RGtEWT64zOTxWlL2bymmci3D4gMyFqnUkhmvWWxPa/kjY2Tk5QRzPJfvysIDXF24znhx/wAoXw24pa0w+lsJiNVPy+UsNrQRvxrHAE+6e7klcQxjQXOdts1rXE9AV1ukTGjTRe8xedWvbbkTss+nkRF0WRERAREQEREBERAREQEREBERAREQEREBFF53UuM01Qs3MjbbBDWY2SUAF7w1zuVuzGguO7ugAB3PQLhu53M2HZGvh8C59itJAyKzlZxWq2GvAMjmFgkk/BtPUOjaHO2aD7pzQsS48jmKGHFbu+9Wo90zsrQd0zNj7WVx2ZG3cjmcfM0dSouxp/J5Kez3ZnpoqptxT1ocbEK7mRM8sUjyXF4c7q4t5OmwG3XfrxulsTiJLMlWjEySxcffkkdu9xsPHK6Td25B5QG9PIBsNh0QccOsRkXQd68Tk8jE68+lNP2HczK/J7uU9uWF8e/QOiD+YnpuASPytBqe+akluxj8QI7b3zVqjXWjPXHSNnavDORx8rtmH4AenMbEiCu1ND0I3UZb01vNWqVmS3Xs5GYyOjkf0JDRs0bDo0BvijydSSZypUgoV2QVoY68DBs2KJga1v5AOgXuRARFwZvO0dO0DcyExhg52RDlY57nve4NY1rGguc4uIAABJ3Qd6i8rqSliblei95mydqKaatQhAM07Y2gv5QSAAN2jmcQ3d7Rvu4Llac3lrYJHeKpVvkcp5J5L1dreh+CIOeT/OdytHuC7xe3A4GlprGRY+gyRleMucDNO+eRznOLnOfJI5z3uLnEkuJJJQRE+FyWrakkeZmfjcXcpRNkxdOR8VqKUkOlDrUcnUbbM2jDenOeZ3MA2i6H9izorQHG3U3E3G15HZnNtL+xnPOypNI5xsSRE9R2pIJH8nd4B5XcrdiRAREQEREBERBAaNkLaV6o52VldTv2IjPl2ASSAvMgMbgAHxASBjHeXZgB3IJU+q7h2uray1FAe+8jJo6twSWyHUmlzXxGKsfMR2Ae9nmMrXfy1YkBERAXy/oj2CemdC8b9S68xmSt4xlhjJcFHjniJ+JsP5xZ2aWlkjS3lDA4FvLJI1zDs1y+oEQZ8daZzQjez1lR7txbBsNS4eBzomjoN7Ncc0kPXcl7OeMAFznRjorxjslUzFCC9QtQ3aVhgkhs1pBJHI0+RzXAkEH4QulUbIcM+9t+XK6OyB0vkpXmWeqyPtMddcSS7tq24AcSSTLEWSE7cznAcpC8oqnpjXEt/InCZ7GuwOo2MLxWLzLWtsHlkqz8rRK0edpDZG9C5jQ5pdbEBERAREQEREBERAREQEREBF+EgAknYBVzGtfq11PLvtsOJbJ3TjhjrTnRXIXxbMlmPKOYHnJawbt9y7dx5eUPIaygybmx4GE5wy1pp4bdd38BLmOLBG6wAWhxeC3Zoc5uxJbsOvi/A5bNxyjL5R1Srax7K82PxL3RGKY9ZJGWhyy/9VpbyEDc7c2xbYK1aKnXir14mQQRMDI4o2hrWNA2AAHQADzL2II7HaexmItWLVOhXr3LLI2WLTIx204jbys7ST3T+VvQcxOykURAREQEREBFy5PJ1cLjrV+9OyrTqxOmmmkOzWMaN3OP5AFFdz5DP2uad8+LxtezDYrCtMWS3GhnMWzgsBjbzuHiNIJ7Lxjyvcwh4v1FLm9otOmC5DLHZjOaa9k9SrPE4x8jmNeHSOEgcCxpAHZSBz2O5Q7sxGnoMXYluvkkuZSxDDDZuzOPNL2bSGkNHisG7nu5WBo3e47dVI168VOvFBBEyCCJoZHFG0NaxoGwAA6AAeZexAREQEREBERAREQEREFemiEPEGrIGZZ5s4uZpex2+Pj7OWLYPHmmd2x5fhbHJ/NCsKrua8TWWm3/APPLuZtqLal1ojdjXc1ofD4m0Z+Fzh51YkBERAREQEREERqjTFPVmM7jtmSF7HiatbruDZ6szd+SWJxB2cNz5QQQS1wLXEGK4f6mvZetexWbbHHqTDSire7JhZHYBbzRWYmknaOVmzttzyuEke5MZKtizvW+2lOIukdTxgsgyEvg5knA7NLJeZ9R7vhLbA7Nvzt/l8waIiIgIihMxrbT2n7QrZPOY+hZI5uxsWWMft8PKTvst00VVzamLytrptFVvbU0d6UYn1yP609tTR3pRifXI/rXL2bG8E8JXRnJaUVW9tTR3pRifXI/rT21NHelGJ9cj+tOzY3gnhJozktKKre2po70oxPrkf1p7amjvSjE+uR/WnZsbwTwk0ZyWlFVvbU0d6UYn1yP609tTR3pRifXI/rTs2N4J4SaM5LQ5oe0tcA5pGxB8hVE4X6+0/mcdS05BqXTOS1NjanZ3sbgbUe0BiIieWwb88bGu2bsQOUkD4FVuN9Lhpx04cZTSWb1LhxFZbz1rPdMbn1bDQezlb18oJO/k3BcPOvmL/k9uG+P4Laj19l9WZfG0ck2QYek91tgZPC13PJKzc+MxxEWzh8B+BOzY3gnhJozk/oEiq3tqaO9KMT65H9ae2po70oxPrkf1p2bG8E8JNGclpRVb21NHelGJ9cj+tPbU0d6UYn1yP607NjeCeEmjOS0oqt7amjvSjE+uR/WntqaO9KMT65H9admxvBPCTRnJaVx5TLVsPBFLae5olmjrxtYxz3Pke4NaAGgnynqfI0AuJABIq+Q4x6NoiFrdRY6ead5jiZHYDm83K53juG4Y3Zp8Z2w32HVzmg8eD17pWs91/I6uxM2WswQssiLI71oywO8WGNztmjme8823M7ccxIa0NdmxvBPCTRnJZcdi7V21XyeXHZXIRMyKpXnc6vGxzwWuc3oHy8rGeMQeUl4YdnOLptVb21NHelGJ9cj+tPbU0d6UYn1yP607NjeCeEmjOS0oq7S4i6WyNiOvV1Hi555HBjI2W4y57j5ABv1Pl6fIrEuKvDrw5tXEx6paY2iIiwgiIgIiICIiAiIgrmoiG6l0of+eTvbmbtjv9F/0aU72/6Pp4v9IY/hVjVd1KSNQaT2OYH8Pl3GNH8GP8En/wBM/ov5v9L2SsSAiIgIiICIiAqJx0oS3uEep31tu7KNQ5Srvvt29VwsReTr7uJvkV7Xov0oslRsVJ288E8bopG/C1w2I/sKD8x96HJ0K1yu7nr2ImzRu+FrgCD/AGFdCoXAO7Le4JaFkncX2WYWrBM4jbmkjiax5/vNKvqDizVx2Ow960wAvggklaD8LWkj/cqjpKrHX0/SkA5p7MLJ55ndXzSOaC57iepJP9nk8gVn1V72Mx8zm/UKr2mfe5ivmkX6gXoYGrCn1a3JJERbZEREBERAREQEREBERAREQEREBERAREQeq1VhvV5K9iGOxBK0sfFK0Oa9p8oIPQheXDu5La0yGSyvnNW3aqNklJc4sinkjZuSSSQ1rRuTudtz5V5rl4Ze965+dsj+1yqYncT6x9pXctqIi81BERAREQEREBERBW9TOaNRaQDpsrETkJQ1lDfueQ9yWOlr+i84/pBErIq7qSQs1BpNvNlxz35Rtjmg1z/BJz/C/gi6eL/S9krEgIiICIiAiIgIiIM74B7M4axVwCG08rlqQBO+whyNmLb/APRaIs74IHl09qKEANEWqc30H/WyE8n+Jfv/AFrREEXqr3sZj5nN+oVXtM+9zFfNIv1ArDqr3sZj5nN+oVXtM+9zFfNIv1AvRwe5n1+GtySXzD7Hzjlqapw94Yx6rwFyxitQSDFw6otZVtmxNbd2rmGWIguDH9m5oeXk9Bu0bhfTywjA8CM/i+E/CbTEtzGuv6SzVTJXpGSyGKSOIylwiJZuXHtG7BwaOh6hSb31Mpif2QPY8IMprnvDzdw5p2H7g7s93tkhS7TtOz6eXn5eU/zd/wCUoTVnsmMvpuvrbKV9COyOntH5XvbkrrcuyOZ45Ync8MJj8cgTNJa5zB5NnO67RGoOA/EGbRee0Pirum/Bu5qHv3Bcty2Bb7N19lx0DmNjLWlrg7aQOdzAAcrd+YTWpeBGfzPDzjDgYLmNbc1jmnZGg+SWQRxRmKqzaUhhIdvA/o0OHVvXy7Z/ULNgeLmdtamzmmczo0YvUdPDjN0adfKx2I7sJc5nIZSxgjkD2hpB3aOYHmI6qtaR9lAzPWdXY6/hcdXzOBwsubbBiM/Dk4J4o9w6N0sbB2UgdygtLT0eCN12cXOBuY4kao1NdqZWvi6mV0dJp6OXmf2zJzZ7XdzQNuyLfFOzt+p6KBrcDta3M/fyluvo/CwWdH3dLR4vCOnbFXMha6KUPMQ5gXAgt5W8o22Lzur+oWjRfHbKZ/P6MqZnR5wGP1hSkt4e23JMsyOLIRMY5owxojJjJcCHP8mx5T0WwLG5eGmSwEPB7JWrFZ1bQNCYZQV2TTSTf83Or/weNkZdIefrtsCR5AT0Vlq8cdL3LUNeODUgkleGNMmlMqxu5Ow3c6sA0fKSAPOrE22ilae9kxavcPMpr/NaTbgtGY9lpr7rso2WeeaKwYGsii7NoLXuGwe97dnAjbbZxiMR7Lk5axexsGAw2RzpxVrJ42jgtU18k2yYGh768r4mEwyFp3b4rmuIIDuinsf7H63e9jZa4Z5jJQ1r08lmZl+jvLHDI68+1A8BwaXcpMfMNhvsQD51beHNDX8F17tZVtJQwR1+SN+nxO6WaXcbyO7RrRG0jfxBzdT7rop+rUITI8cnZl2DqaRwzM/Ll9NzajL5Mh3I2tX5WCEFwjf40j3uaOni9m87HbZUjF+yYxujOGPDaBvZW83nsMzIsGq9Sw1RHAA0F892Vg7R5c4ABse7tnHYBpKu/CXgQzhbHrURXG2zl7MjMcHb7UqHjvhqjp0aySac9N+jh8CqOF4Aav0Ti+HOUwFvAW9U6d08NO5KllHS9wXYN2P3ZK1hexzZGbglh3BIICfqE3oT2UGO1xkdLwR4uOCrlr9/D2L0WRjsQVb9aJswibIwFkrJYi5zZA4e525dz09unfZO4nWWnNP5LA4196zmdSnT8NKSfs3NYC6Q2SeU+L3K0T7bdQ9rd+vMuniVwezfFzg9Dp7M3cbhtTtuxXW3sI2RkFZzZSCY+bxi7sHPZudty4nYA7Dxr8CsRpDjBBxAqudXwuLwJqMxFeN8nZzsY2ITsjaCXO7mjEWwBcQ0AAp+obEsJr+yUyjqbs3Z0Qa2koNQv07aynfVjpY5BcNVkzYOz8aMv5ObdzXAuIDXABzrrDx10tPMyNsGpeZ7g0c2k8q0bn4Sa2w/KVSbPAjPzcGczpFtzGjJXdUuzkcplk7EQHLNucpPJvz9m0jbYjm6b7dVZm+wdWpPZGX8XLqzJYrRc+a0dpO0+nmcy3IMila+INdYMFctJlEQd4xL2blrgN9ly8S+OWYyNHW2M0HpuXPQYXEukyOdbk20m1JJa5lY2DdpMsjY3NediwDdo5tyuXUvBDXRpa90pp3KYGDR2tL1i5bt3hN3fQFpoFtkUbW9nKHeOWlzmcvOd99gv3J8EtbaVs61x2hLWnpNM6rqNZLBnHzsnoziq2sXRmNjhI1zGMJDuUgg7LP6hF1vZPV9F6S0Fg3yYzKakn0vj8pfsaj1HDi4w2SFoB7WYOdLK9zXkgDydXOHMN9m4UcScdxb0HjdUYyN0Na32jHQve2QxyRyOjkbzMJa4BzHbOadnDYjyrK8XwQ1poK/gM1pabTmQyI01QwOax2bdM2tJJVZtHYglZG5wI5nt5XMAI28hW46dgv1sFQiyvcffNsLe6jj43MrmXbxzG1xJDd99tzutU33iRXLwy971z87ZH9rlXUuXhl73rn52yP7XKtYncVesfK7ltREXmoIiICIiAio2sOLOM0xZko1YZMvk2dHwV3BscJ232kkPRp+QBzuo6bHdUefjZqeV5MONxNZvmZI+WUj/tDk3/sXqYP4Z0rHp06abR56ls3FFhPt0at+K4X6Ob99Pbo1b8Vwv0c3767H5N0vKOJ/LG/ZIezpzHBXjDBpi/w8uS96bPddWenqIww5WGSGSNnaR9yu3aDJzcgceWSNvU8vX7F0plbmd0vh8lkca7D5C5ThsWcc+TtDVlewOfEXbDmLSS3fYb7eQL5O4pabHF3XGjtU57G4l+R0zY7eBscb+SyNw5scoJPMxrwHADbz/CtQ9ujVvxXC/Rzfvp+TdLyjify3ZFhPt0at+K4X6Ob99fo40atB61MK4fAGTD/HnT8m6XlHE/luqLKMHx0BlbHnsSaUZIHddGQzsHyuYWhzR+Tm/wB+2o07lfIVYbVWeOzWmYHxzQvD2PaeoLSOhB+ELzukdFxuizbFpt9uJZ7kRF1EEREGd8GPFh1ozzM1Tkf8Xh3/AHloizzg+WibXjWgjl1Rc33O/UsiP/8Aq0NBF6q97GY+ZzfqFV7TPvcxXzSL9QKw6q97GY+ZzfqFV7TPvcxXzSL9QL0cHuZ9fhrckkXpu1u7ac9ftZYO1jdH2sLuV7Nxtu0+YjygqN8GIPjmS9el/eSZllMIofwYg+OZL16X95PBiD45kvXpf3kvOQmEUP4MQfHMl69L+8ngxB8cyXr0v7yXnITCKH8GIPjmS9el/eTwYg+OZL16X95LzkJhFD+DEHxzJevS/vJ4MQfHMl69L+8l5yEwih/BiD45kvXpf3k8GIPjmS9el/eS85CYRQ/gxB8cyXr0v7yeDEHxzJevS/vJechMIofwYg+OZL16X95PBiD45kvXpf3kvOQmEUP4MQfHMl69L+8ngxB8cyXr0v7yXnITCKH8GIPjmS9el/eUhRpMoQdkySaUb7808rpHf2uJKRM5DoXLwy971z87ZH9rlXUuXhl73rn52yP7XKridxV6x8ruW1EReagiIgKi8WdYy6Zw0FOjIY8nknOiikafGhjaAZJR8o3a0fA57T1AKvSwvjRNI/iBUicT2UeLa6MfK6V4f+oz/Ber+GYNOP0qmmvZGvgsKXHG2JnK3fbckkkkkk7kknqST1JPlXkiL9DcYiL5zgPEHiPd1Tk8NbfUtUctax9I+EElaGp2L+VjZKgrvZJvsHO53EuDunKNl18XG6q0WvM5D6MRYHnYczl7/Fm1PqPMULOArQWKMGPvPjggm73skcQ0e6aXt9y7dvUnbckqRwl3JcWdYOpX87k8JTx2Dx95lfEWTVfZmssc58rnN6uazlDQ33O56hcXabzoxTrmbR/EzyVrWnNRY/VmEqZfFWO6sfaaXwzcjmcw3I8jgCOoPlCkVnHschy8E9KDcnas7qfP+EetHXYwq5xMOmudsxEgrjwq1bJp3UEOJmkJxWSkLGMcfFr2DuQW/AJDuCP53KdgS4mnL02pXwGtNF/00VqCSP8A22ytLdvl3AWOkYNPSMKrCq3/AHzWnbZ9WIiL8vUREQZ3wg2F3iCAf/imz/kwLRFnfCD/AE7iF/8AdNn/ACIFoiCL1V72Mx8zm/UKr2mfe5ivmkX6gVh1V72Mx8zm/UKr2mfe5ivmkX6gXo4Pcz6/DW5JIiLTIiIgIsoyfHBuns/r45THWm4TTD8bVArV2yWbFi07YcnLKQ9v4SDZpax43PR24Ufq72Q82O0pqKbE6UyrdUYvIUcUMNkhXa4zW3MbXfuyctcx3aN6B/MD7oNAJGdKBs6LL9Vcf8XpBlttrT+csXMbjm5TMVKraz3Yiu7mLTYeZxHzEMeQyN73ENJAPTeSq8YsfkNV5XCUsPl7bcSyCXIZJsUUdWqyaHtmlxfI15IZsS1rS4cw3Gx3S8C/Isy05x5x2o8dpK+NOZ/HVNUzwQYp96Ku10/aV5Zy8sbM5zWMZC7mJA35mlvMDuv23x6xMeTONpYTN5e+7L2sLDXpQwkzT14WyyuaXytaIxzcvM4t8ZpB2GxK8DTEWM5f2Qs1uto52l9LZTMXM1mLWMtY6UV4bNM1WyGzG4STsb2gMZAPOWeXxty1rtQ1Hq3B6Ox8d7UGYx+BpySCJtjJ2o68ZkIJDA55ALtmuO2/mPwK3iRLIo3T+psPq3HNyGDytHM0HOLBax9lk8RcPKOZhI3HwbrIuFnHiTLWGQ6kq5KvXzWbylfC5ievCyjJFBNN2UAc13OHdjA5/M9gDtnbOJCl4G3osrxPsiMJm7dZlTC5t1S/StX8VfkigjhycUDQ55hDpQ9oII5XStjY7ceN1UBwz4xZeXRWDzuo6ebyef1cTbxGmqsFIObW5BKHQEPaBEI5I+Z9mUO5tujeZrS0oG5ostpeyExGZnwtTD4DPZjKZOC7OMdWhgZLVNSdsE7JzJMxjHCRxaPGLTyHYndvN77HHzBRaljxcOOytuk7LtwLs1BFF3Ey+Ty9h40gkeQ7xXOZG5rSDuRsdl4GloiLQLl4Ze965+dsj+1yrqXLwy971z87ZH9rlUxO4q9Y+V3LaiIvNQREQFlvHHT0k1ShqCBpcMfzxWwPNA/Y9of9hzRv8DXPPmWpLxexsjHMe0PY4bFrhuCPgK7XRsero2NTi07v+lXyzKHmN4jLWybHlLhuAfNuOm6p3e/iH+PtM/oSx97W76t4NXKs8lrTRilquJccZO7szF8kT9tuX4GO2267O22aKVNpjUlZ3LLpnKBw8oZGyQf2scR/ivvcPpfRuk0xVTXbyvafuzozuZ73v4h/j7TP6Esfe15ZLhDpPMZ52auYhr8nI5kk0kM8sUcz2bcrpI2vDHkbDYuBPQK+d4c/6NZf1b/ineHP+jWX9W/4rmv0edVVUT6zE/eTRlWZNF4aV+oHvp7uz7BHkj2r/wAO0RdkB5fF8QbeLt8Pl6qLy/CPSecdjH3MTzS42s2nWlisSxPbABsI3OY8F7enuXEjy/CVeu8Of9Gsv6t/xTvDn/RrL+rf8VZno9WqZp4x6mjLPq+kM7palVxOkLmExOAqRiOvUu4+xakZ1JP4Tulu43J8o/rXs738Qvx9pn9CWPvavveHP+jWX9W/4oMBqBx2Gmsvufhrbf7ys3wI2VxH/r/ZoygNPQ5qCrIM5coXbJfux+PqPrsDNh0LXyyEnffruPN0Vu0Lp+TVOsKFdrSatGRl628eRoa7eNh+V72jp5w1/wAGy7sJwt1Pm5mieozB1dxzTXHtkkI8/LGxx6+bxiPh6+fZ9L6WoaQxTaGPY4N355ZpDzSzyEAF73ecnYDzAAAAAAAeX078SwsHCnDwatKqdW29v5zWItrS6Ii+HBERBnnCE81vX53B31Ra8g28kUI6/wBi0NZ5wcG7NaydPwmqL56HfflLWf8AdWhoIvVXvYzHzOb9QqvaZ97mK+aRfqBWHVXvYzHzOb9QqvaZ97mK+aRfqBejg9zPr8NbkkiItMq1ndV5TEZB1epozOZuENBFuhNRbESfNtNZjfuP9nb4N1wO17nGnpw31O7oDuLOL++q6IpYZXLwWdlX5G1Zyr45Mrqmlqe1FJVBcGVmQCKoSJCPFNaMl4JHl6edc+oeBVzLWc1kaepI6mZvamqajhsT47toYhXgihirvjErTI0CMu5g5h5nb+brriKWgZBkfY+VcjxSdrOd+ByE1ruV12LK6fjtzCSFoZz1ZnSA1+ZrW7gh+xG42PVTr+FM3g7xIoxZrs8hrGezN3f3LuafaVY60Y5efx+zbE0+Vu/yLQkS0DO9S8K7Vx2hp8BmK+It6TL21Rcom3BIx1Y1yCxssZDg09HB3TruCCs3xXB/V2ndeaex+J1AzujGUcvlLepMjhHTQWreQute5oY2ZjRI1jHbbPOw23bs7ZfRiJNMDJaPAmfT1/Rd3B6j7G3gpb8t2XJURZOTfdkZJZldyvj7OUua4tcNw0PI5SOi0XUmmaeqqDKl2bIQRMkEodjclYoybgEbGSCRjiOp8UnbyHbcDaVRW0CIxOnItO4WTH4yzc32eY58ldnvyNeR0JfNI57gD/J5tvg2WVaV9jfLj8Zp7Eaj1Oc/g9O1ZoMZSr0BUPaSxPifYnf2j+1kDJZQ0gMA5ySCeq2xFLQPnuLgXkeFvCPVWG05SwuYz2SxXeXHWsTgocZZ8djoxNbmEhEu3M17nAN9wSGkuV4yvCK7Df0jktM52DCZPT2KlwrHXMebkEtaQQ7+IJYy14NdhDg7byghwK0xE0YGb8O+CtTh1qFuSrZKa8xmJbjmtsRjtHSutTWbNl7wdi6WSUEtDQBydN99hG8O/Y/0OHmtchmoBg7cE9m3bglfgY25SF88rpHNdd593sBe8AcgPLyguIataRLQCIi0C5eGXveufnbI/tcq6ly8Mve9c/O2R/a5VMTuKvWPldy2oiLzUEREBERAREQEREBERAREQEREBERARF+OcGNLnENaBuSfIEGe8Dvwulcza5eXunU2cePla3JWI2n+sRg/1rQ1n3ACJw4N6VsvjfE/IVe+bmSDZzTZe6wQfl3l6rQUEdqOF9jT2UijaXSPqyta0eclhAVa0u9smmcQ5p3a6nCQfhHIFdlU7XD9nbvfjM1ksHC9xe6rSEDoQ49SWtlify7nrs0gbknbqu7gYlMUzRVNt6xss6UXD4AX/TPN/Q0fuyeAF/0zzf0NH7sue+H449+S2jN3IuHwAv8Apnm/oaP3ZPAC/wCmeb+ho/dkvh+OPfkWjN3IuHwAv+meb+ho/dk8AL/pnm/oaP3ZL4fjj35FozdyLh8AL/pnm/oaP3ZPAC/6Z5v6Gj92S+H449+RaM3ci4fAC/6Z5v6Gj92TwAv+meb+ho/dkvh+OPfkWjN3Iqxo3S+Tz2l8bkJdf3shJYhD3WsdXpivIfhYHVydvylTPgBf9M839DR+7JfD8ce/ItGbuRcPgBf9M839DR+7J4AX/TPN/Q0fuyXw/HHvyLRm7kXD4AX/AEzzf0NH7sngBf8ATPN/Q0fuyXw/HHvyLRm7kXD4AX/TPN/Q0fuyeAF/0zzf0NH7sl8Pxx78i0Zu5Fw+AF/0zzf0NH7sngBf9M839DR+7JfD8ce/ItGbuXPw0YW6csO8rZMnfe07bbg25divWzh/O/xLWqc3bgPuot68PMPOOeKFj2+XytcD8BCtNSpBj6sNWrDHXrQsEcUMTQ1jGgbBoA6AAeZcOLiUaGhTN7zHtfP1Nj3IiLosiIiAiIgIiICIiAiIgIiICIiAiIgKmcZctPheFeqbFRrn3n0JK1RjDsXWJR2UI32O28j2DfYq5rPeIYGpNa6L0qGuki7qOfu7HxWw0ywxB3TymzJWc0dNxE8j3J2C54DDwadwWOxVUbVqNaOrENtvEY0Nb/gAu9EQEREBERAREQEREBERAREQV7Ql3uzTrGOvVMhPVsWKc0tGHsoxJFM9jmcn8ktLdj5txuOhCsKr0E78JqmatZt89XLO7SjAykWiKRjPwrXTN8UlwAeA/Z3STYuA2bYUBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQeE88daGSaaRsUUbS98j3ANa0Dckk+QBULhfXlz1vM65txmOXPOZHj2O8seNh5hX/rkL5Z/kE7Wnq1eGqP/SjlJtK1SJNMV3uj1DZA3ZZ2H8Xs8x5tx2x6gMBj91ITHoaAiIgIiICIiAiIgIiICIiAiIg5cnj48rj7FOZ88UU7Cxz6074JGg+dsjCHNPyggqK7+2MJO2DNtaIrF1lSjbqxySiXmZu0zhrNoDzB7NyeQns9nB0gjE+iD8a4PaHNIc0jcEHcEL9Vdh0iMKabcBZ7z0qzbB71RRMNSZ8pLg5w5edvK8lw5HNGznAg9OXwGrpcPB/5x0TixXx3d1zJRSCTHxFrtpGCU8r929HbuY0Fp3BPK4NCyovVWsw3K8VivKyeCVgkjljcHNe0jcOBHQgjruvagIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIq9q3XmG0VHXGSsuNy0S2pj6sbp7dtw8rYoWAvft5yBs0dXEDqgsKoFnUt3iLamxelppauDY50N7U0WwBI3DoaRIIkeD0dNsWMO7Wl7w4R+Hg3qDiLJ2up3SYDT2+8enKc+1iwPN3bYjdsQem8ER5PKHvla7lF9q1YaNaGtWhjr14WCOOGJoaxjQNg1oHQAAbABBzYXC0dOYqtjcbWZUo1mckULN9gPlJ6kk7kk7kkkkkldyIgIiICIiAiIgIiICIiAiIgIiICIiAiIgr+T0XSuPyNmjLPgsregjgkyeN5GzAMO7Ds9ro3FvkHOx3QkdR0XjeuaiwxydkUos/Ub2HcVOgGwWyPczc7pZBG4j3berOm7fKATYkQRNbVGNtXbVTt3QWK1htV7LUT4eeRzeZojLwBICNyCzcHYjfcECWXJksTRzNcV8hTgvQNe2UR2YmyND2kOa4AjyggEHygjdRPgxaoTNficxaqslyJvWobrnXGSscAJIWdo7eJvTmaGENaSdmkeKgsKKvV9QZSpZp1cthJGSWrM0LbOLkNqvGxvWN8pLWPYXjzcrmtcCC8jlc6B13x20Tw74fx61yubru07JahqMtVXtl5nvm7J3K0HdxjIkc9rd3BsUniktIQX9FA5HWFKDGY+3jyMwck0PoNpyNc2wwtDu0D9+URhpDi/fbYgDmc5rTEu1RqzfxdO4jbYe6zMgP+FUrsUYGJiRpRs85iPvK2XRFSvCjVvo5h/01L91Two1b6OYf9NS/dVvsuL5fVTzWy6oqV4Uat9HMP+mpfuqeFGrfRzD/AKal+6p2XF8vqp5ll1RUrwo1b6OYf9NS/dU8KNW+jmH/AE1L91TsuL5fVTzLLqipXhRq30cw/wCmpfuqeFGrfRzD/pqX7qnZcXy+qnmWXVFSvCjVvo5h/wBNS/dV2Y3WFxt+vVzWLjxpsu7OCxWs90QmTbfkc4sY5pPXlJbykjbcOLQ6T0bFiL6uMT9pS0rSi8ZHtiY573BjGglznHYAfCVQp+M+GvyS19KVrmursb+zc3ANZJXjf13D7T3NgaQQd285cNvck9F1UX9V3VvEHT2h2wDM5OOtYskitSiY6e3aI8rYa8YdJKfkY0lV04HXusA05nN19G49w8bH6d2sW3D4H3JmANBG24jhDgd9pD0KsOlOHun9FOnlxOOZFdsACzkJ3usXLO3k7WxIXSSf9pxQV91/XOuXFmPrDQWFJ27uvsjs5SZvwxQbuig38odKZHeZ0LSrBpTh/hdHzWbVOCSxlbYHdeWvSusXLO3UB8zyXcoJPKwbMbvs1rR0VjRAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQF8vezR9i9qv2TL9MUcFfwOGxmLMs1izfD3WZpH7BrW8sRLWNAcfd7OMnVo5AT9Qog+WvY38DtVex+t0tLag1bFqigyhalx0UcDmCiHTQdqxrnEnlceV3L0AIJ86+glGag/wBZGJ/NNv8Azq6k16v+Oj0+ZancIiLLIiIgIvF8jIgC9zWAkNBcdtyfIF5ICIuOzmcfTyNLH2L1aC/dDzVqyTNbLOGAF/I0nd3KCCdgdtxug7FWeIzbT9LltGaKtdN2kIJ54u1ZHJ3VFyuLA5pcAdjsHNJ28o8qsyr+uP4kg/OND9shXLg95T6wsbYI+C+OzD2z6yyd/XM4Id2GWeG0Gn/q04w2E7eYyNe8fzvLvf69eKpBHBBEyGGNoYyONoa1rR0AAHkAXsReOgiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgpOoP9ZGJ/NNv/ADq6k1Gag/1kYn802/8AOrqTXq/46PT5lqdzI+M2oNVVuIPDLTmms4MDHn7d6G9OasU57KKq6UFoe07OBb4p8m+3MHAFp8dE6n1FDxG4j6byWcmzFfT+HxMtWexXgjkM0sVkzSO7NjQS8xMO22w26Add71n9B4/UerdLahszWWXdOy2JakcTmiN5mhdC/tAWkkBriRsR18u/kUBq3gpjNVaot56PNZzA28hSZj8kzD2mwsvwMLyxsm7HOBb2jwHxljtnEbritN7ssh4ba94g8Ur2hMUNaSYXvnoWLPXrlfG1ZJpLXbiMuaHsLGg8w3HKRs3oGk7rYOAWtcnxB4S4PN5p0UmWkNitalhZyMkkgsSQF4b5ubsubbzbr80FwQwXDvJYO7jbeRnlw+n26cgFqSNzXVmyiQPfysG8m7R1Gw2/krjw2mdS8JsJS03orAY/P4SAz2O683nnVLAlmsSzPbyx03tLQZOh3B26EdNzIiY2ir+ymw+Qy1zhO2hnreDc7WNaEPqwwSEPdBOWy/hWOHMzlcAPcntDuDs3b2WcjrrW3ErUGjcHrR+nINJYugZ8i7G17FjJ27DJHB8jXN5GRgRdWxtaS5ztiAABas5obJcXdNNo6zoM0tco34b+Ot6czDrM0E0e5bK18leMNcN3N5SxwIJXJnPY/Y/NWob0eq9U4rMGg3GXspjr0cVjJQNLi0WPwRaXAvfs9jWuHMQCAlpvcZtw94ua34/3NP4zFZyPQ7o9Nsy+Tu0qUVqSxadZmrhkTZg5rYQa8jydi48zRuNt16tHa6yXELiHwRyOZEBzFaTU2NuSVmlsU0tfkhMjB5g/kDtvNzEeZafkfY7acMWCGn8hmdF2cNju9Fe3p602KV9Pfm7GQyMeHjm3cHEcwcSQdyV1V+AOl8dR0RWxbshiDpCw6xj56dnaR/aHedkxcHdo2brz79TudiCpareNIVf1x/EkH5xoftkKsCr+uP4kg/OND9shXawe8p9YWNsNDREXjoIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiIKTqD/AFkYn802/wDOrqTX5qnDW5r9HL4+MWbVSOSB9Uv5O2ikLC4NJOweDG0jfofGG45txCHP5Jp2Ok83v59hXO3/APZerRbEw6dGY1RbbEb5za2pxFB+EOS9E85/dr/bJ4Q5L0Tzn92v9stdXOccY5lpTiKD8Icl6J5z+7X+2TwhyXonnP7tf7ZOrnOOMcy0pxFB+EOS9E85/dr/AGyeEOS9E85/dr/bJ1c5xxjmWlOIoPwhyXonnP7tf7ZPCHJeiec/u1/tk6uc44xzLSnFX9cfxJB+caH7ZCvZ4Q5L0Tzn92v9svZHQyWq56kVjF2cRjobEVmV9x8faTGN7ZGMY1j3bDnaOYu26NIAPNuN0x1dUV1TFo84Ii03XtEReKyIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiIP//Z",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from typing_extensions import TypedDict, Literal\n",
    "from langgraph.graph import StateGraph, START, END, MessagesState\n",
    "from langgraph.checkpoint.memory import MemorySaver\n",
    "from langchain_anthropic import ChatAnthropic\n",
    "from langchain_core.tools import tool\n",
    "from langchain_core.messages import AIMessage\n",
    "from IPython.display import Image, display\n",
    "\n",
    "\n",
    "@tool\n",
    "def weather_search(city: str):\n",
    "    \"\"\"Search for the weather\"\"\"\n",
    "    print(\"----\")\n",
    "    print(f\"Searching for: {city}\")\n",
    "    print(\"----\")\n",
    "    return \"Sunny!\"\n",
    "\n",
    "\n",
    "model = ChatAnthropic(model_name=\"claude-3-5-sonnet-20240620\").bind_tools(\n",
    "    [weather_search]\n",
    ")\n",
    "\n",
    "\n",
    "class State(MessagesState):\n",
    "    \"\"\"Simple state.\"\"\"\n",
    "\n",
    "\n",
    "def call_llm(state):\n",
    "    return {\"messages\": [model.invoke(state[\"messages\"])]}\n",
    "\n",
    "\n",
    "def human_review_node(state):\n",
    "    pass\n",
    "\n",
    "\n",
    "def run_tool(state):\n",
    "    new_messages = []\n",
    "    tools = {\"weather_search\": weather_search}\n",
    "    tool_calls = state[\"messages\"][-1].tool_calls\n",
    "    for tool_call in tool_calls:\n",
    "        tool = tools[tool_call[\"name\"]]\n",
    "        result = tool.invoke(tool_call[\"args\"])\n",
    "        new_messages.append(\n",
    "            {\n",
    "                \"role\": \"tool\",\n",
    "                \"name\": tool_call[\"name\"],\n",
    "                \"content\": result,\n",
    "                \"tool_call_id\": tool_call[\"id\"],\n",
    "            }\n",
    "        )\n",
    "    return {\"messages\": new_messages}\n",
    "\n",
    "\n",
    "def route_after_llm(state) -> Literal[END, \"human_review_node\"]:\n",
    "    if len(state[\"messages\"][-1].tool_calls) == 0:\n",
    "        return END\n",
    "    else:\n",
    "        return \"human_review_node\"\n",
    "\n",
    "\n",
    "def route_after_human(state) -> Literal[\"run_tool\", \"call_llm\"]:\n",
    "    if isinstance(state[\"messages\"][-1], AIMessage):\n",
    "        return \"run_tool\"\n",
    "    else:\n",
    "        return \"call_llm\"\n",
    "\n",
    "\n",
    "builder = StateGraph(State)\n",
    "builder.add_node(call_llm)\n",
    "builder.add_node(run_tool)\n",
    "builder.add_node(human_review_node)\n",
    "builder.add_edge(START, \"call_llm\")\n",
    "builder.add_conditional_edges(\"call_llm\", route_after_llm)\n",
    "builder.add_conditional_edges(\"human_review_node\", route_after_human)\n",
    "builder.add_edge(\"run_tool\", \"call_llm\")\n",
    "\n",
    "# Set up memory\n",
    "memory = MemorySaver()\n",
    "\n",
    "# Add\n",
    "graph = builder.compile(checkpointer=memory, interrupt_before=[\"human_review_node\"])\n",
    "\n",
    "# View\n",
    "display(Image(graph.get_graph().draw_mermaid_png()))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d246d39f-4b36-459b-bd54-bf363753e590",
   "metadata": {},
   "source": [
    "## Example with no review\n",
    "\n",
    "Let's look at an example when no review is required (because no tools are called)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1b3aa6fc-c7fb-4819-8d7f-ba6057cc4edf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'messages': [HumanMessage(content='hi!', id='393fa21d-4bfb-445b-8faa-78e22b92e346')]}\n",
      "{'messages': [HumanMessage(content='hi!', id='393fa21d-4bfb-445b-8faa-78e22b92e346'), AIMessage(content=\"Hello! Welcome to our conversation. How can I assist you today? Is there anything specific you'd like to know or discuss?\", response_metadata={'id': 'msg_017S671xYvZm1mi9EcsKvPzF', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 355, 'output_tokens': 29}}, id='run-8ec507a1-5caf-47d6-89eb-1a2e8f38423c-0', usage_metadata={'input_tokens': 355, 'output_tokens': 29, 'total_tokens': 384})]}\n"
     ]
    }
   ],
   "source": [
    "# Input\n",
    "initial_input = {\"messages\": [{\"role\": \"user\", \"content\": \"hi!\"}]}\n",
    "\n",
    "# Thread\n",
    "thread = {\"configurable\": {\"thread_id\": \"1\"}}\n",
    "\n",
    "# Run the graph until the first interruption\n",
    "for event in graph.stream(initial_input, thread, stream_mode=\"values\"):\n",
    "    print(event)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d59dc607-e70d-497b-aac9-78c847c27042",
   "metadata": {},
   "source": [
    "If we check the state, we can see that it is finished"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "213323cc-0320-4313-ab11-19042e28b495",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pending Executions!\n",
      "()\n"
     ]
    }
   ],
   "source": [
    "print(\"Pending Executions!\")\n",
    "print(graph.get_state(thread).next)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c1985f7-54f1-420f-a2b6-5e6154909966",
   "metadata": {},
   "source": [
    "## Example of approving tool\n",
    "\n",
    "Let's now look at what it looks like to approve a tool call"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "2561a38f-edb5-4b44-b2d7-6a7b70d2e6b7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='8bda37cc-4bd3-4a14-bca5-b992934e710b')]}\n",
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='8bda37cc-4bd3-4a14-bca5-b992934e710b'), AIMessage(content=[{'text': 'To get the weather information for San Francisco, I can use the weather_search function. Let me do that for you.', 'type': 'text'}, {'id': 'toolu_01MW3ETLpq4b8s6VaAMgDBZP', 'input': {'city': 'San Francisco'}, 'name': 'weather_search', 'type': 'tool_use'}], response_metadata={'id': 'msg_019FjC1prjVv8BuQX7DmF65F', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 360, 'output_tokens': 80}}, id='run-1b580410-173c-4fe0-a149-22e8f516b259-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco'}, 'id': 'toolu_01MW3ETLpq4b8s6VaAMgDBZP', 'type': 'tool_call'}], usage_metadata={'input_tokens': 360, 'output_tokens': 80, 'total_tokens': 440})]}\n"
     ]
    }
   ],
   "source": [
    "# Input\n",
    "initial_input = {\"messages\": [{\"role\": \"user\", \"content\": \"what's the weather in sf?\"}]}\n",
    "\n",
    "# Thread\n",
    "thread = {\"configurable\": {\"thread_id\": \"2\"}}\n",
    "\n",
    "# Run the graph until the first interruption\n",
    "for event in graph.stream(initial_input, thread, stream_mode=\"values\"):\n",
    "    print(event)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ef6d51c-e2b6-4266-8de7-acf1a0b62a57",
   "metadata": {},
   "source": [
    "If we now check, we can see that it is waiting on human review"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "33d68f0f-d435-4dd1-8013-6a59186dc9f5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pending Executions!\n",
      "('human_review_node',)\n"
     ]
    }
   ],
   "source": [
    "print(\"Pending Executions!\")\n",
    "print(graph.get_state(thread).next)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14c99fdd-4204-4c2d-b1af-02f38ab6ad57",
   "metadata": {},
   "source": [
    "To approve the tool call, we can just continue the thread with no edits. To do this, we just create a new run with no inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f9a0d5d4-52ff-49e0-a6f4-41f9a0e844d8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----\n",
      "Searching for: San Francisco\n",
      "----\n",
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='8bda37cc-4bd3-4a14-bca5-b992934e710b'), AIMessage(content=[{'text': 'To get the weather information for San Francisco, I can use the weather_search function. Let me do that for you.', 'type': 'text'}, {'id': 'toolu_01MW3ETLpq4b8s6VaAMgDBZP', 'input': {'city': 'San Francisco'}, 'name': 'weather_search', 'type': 'tool_use'}], response_metadata={'id': 'msg_019FjC1prjVv8BuQX7DmF65F', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 360, 'output_tokens': 80}}, id='run-1b580410-173c-4fe0-a149-22e8f516b259-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco'}, 'id': 'toolu_01MW3ETLpq4b8s6VaAMgDBZP', 'type': 'tool_call'}], usage_metadata={'input_tokens': 360, 'output_tokens': 80, 'total_tokens': 440}), ToolMessage(content='Sunny!', name='weather_search', id='835b0fe3-8aa0-45d5-ac29-03bbe57cc767', tool_call_id='toolu_01MW3ETLpq4b8s6VaAMgDBZP')]}\n",
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='8bda37cc-4bd3-4a14-bca5-b992934e710b'), AIMessage(content=[{'text': 'To get the weather information for San Francisco, I can use the weather_search function. Let me do that for you.', 'type': 'text'}, {'id': 'toolu_01MW3ETLpq4b8s6VaAMgDBZP', 'input': {'city': 'San Francisco'}, 'name': 'weather_search', 'type': 'tool_use'}], response_metadata={'id': 'msg_019FjC1prjVv8BuQX7DmF65F', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 360, 'output_tokens': 80}}, id='run-1b580410-173c-4fe0-a149-22e8f516b259-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco'}, 'id': 'toolu_01MW3ETLpq4b8s6VaAMgDBZP', 'type': 'tool_call'}], usage_metadata={'input_tokens': 360, 'output_tokens': 80, 'total_tokens': 440}), ToolMessage(content='Sunny!', name='weather_search', id='835b0fe3-8aa0-45d5-ac29-03bbe57cc767', tool_call_id='toolu_01MW3ETLpq4b8s6VaAMgDBZP'), AIMessage(content=\"Based on the search results, the weather in San Francisco is sunny! It's a beautiful day in the city. Is there anything else you'd like to know about the weather or any other information I can help you with?\", response_metadata={'id': 'msg_01UY2d6RCzvwagwMb1J5etek', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 453, 'output_tokens': 49}}, id='run-7137f52c-abe6-4dc1-b536-92dd1d9187b0-0', usage_metadata={'input_tokens': 453, 'output_tokens': 49, 'total_tokens': 502})]}\n"
     ]
    }
   ],
   "source": [
    "for event in graph.stream(None, thread, stream_mode=\"values\"):\n",
    "    print(event)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d30c4a7-b480-4ede-b2b4-8ec11de95e30",
   "metadata": {},
   "source": [
    "## Edit Tool Call\n",
    "\n",
    "Let's now say we want to edit the tool call. E.g. change some of the parameters (or even the tool called!) but then execute that tool."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "ec77831c-e6b8-4903-9146-e098a4b2fda1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='0c488edd-7b9c-4416-ba02-8a2d7e9f2597')]}\n",
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='0c488edd-7b9c-4416-ba02-8a2d7e9f2597'), AIMessage(content=[{'text': \"Certainly! I can help you check the weather in San Francisco. To get this information, I'll use the weather search tool. Let me fetch that for you.\", 'type': 'text'}, {'id': 'toolu_01CpbVmprQnjxpQzx8MzE1g8', 'input': {'city': 'San Francisco'}, 'name': 'weather_search', 'type': 'tool_use'}], response_metadata={'id': 'msg_01Mv7iqdtPgZEX2LiBBqWDuY', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 360, 'output_tokens': 88}}, id='run-52a09799-efb5-4fff-82c3-884e20119ad3-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco'}, 'id': 'toolu_01CpbVmprQnjxpQzx8MzE1g8', 'type': 'tool_call'}], usage_metadata={'input_tokens': 360, 'output_tokens': 88, 'total_tokens': 448})]}\n"
     ]
    }
   ],
   "source": [
    "# Input\n",
    "initial_input = {\"messages\": [{\"role\": \"user\", \"content\": \"what's the weather in sf?\"}]}\n",
    "\n",
    "# Thread\n",
    "thread = {\"configurable\": {\"thread_id\": \"5\"}}\n",
    "\n",
    "# Run the graph until the first interruption\n",
    "for event in graph.stream(initial_input, thread, stream_mode=\"values\"):\n",
    "    print(event)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "edcffbd7-829b-4d0c-88bf-cd531bc0e6b2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pending Executions!\n",
      "('human_review_node',)\n"
     ]
    }
   ],
   "source": [
    "print(\"Pending Executions!\")\n",
    "print(graph.get_state(thread).next)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87358aca-9b8f-48c7-98d4-3d755f6b0104",
   "metadata": {},
   "source": [
    "To do this, we first need to update the state. We can do this by passing a message in with the **same** id of the message we want to overwrite. This will have the effect of **replacing** that old message. Note that this is only possible because of the **reducer** we are using that replaces messages with the same ID - read more about that [here](https://langchain-ai.github.io/langgraph/concepts/low_level/#working-with-messages-in-graph-state)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "df4a9900-d953-4465-b8af-bd2858cb63ea",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current State:\n",
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='0c488edd-7b9c-4416-ba02-8a2d7e9f2597'), AIMessage(content=[{'text': \"Certainly! I can help you check the weather in San Francisco. To get this information, I'll use the weather search tool. Let me fetch that for you.\", 'type': 'text'}, {'id': 'toolu_01CpbVmprQnjxpQzx8MzE1g8', 'input': {'city': 'San Francisco'}, 'name': 'weather_search', 'type': 'tool_use'}], response_metadata={'id': 'msg_01Mv7iqdtPgZEX2LiBBqWDuY', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 360, 'output_tokens': 88}}, id='run-52a09799-efb5-4fff-82c3-884e20119ad3-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco'}, 'id': 'toolu_01CpbVmprQnjxpQzx8MzE1g8', 'type': 'tool_call'}], usage_metadata={'input_tokens': 360, 'output_tokens': 88, 'total_tokens': 448})]}\n",
      "\n",
      "Current Tool Call ID:\n",
      "toolu_01CpbVmprQnjxpQzx8MzE1g8\n",
      "----\n",
      "Searching for: San Francisco, USA\n",
      "----\n",
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='0c488edd-7b9c-4416-ba02-8a2d7e9f2597'), AIMessage(content=[{'text': \"Certainly! I can help you check the weather in San Francisco. To get this information, I'll use the weather search tool. Let me fetch that for you.\", 'type': 'text'}, {'id': 'toolu_01CpbVmprQnjxpQzx8MzE1g8', 'input': {'city': 'San Francisco'}, 'name': 'weather_search', 'type': 'tool_use'}], id='run-52a09799-efb5-4fff-82c3-884e20119ad3-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco, USA'}, 'id': 'toolu_01CpbVmprQnjxpQzx8MzE1g8', 'type': 'tool_call'}]), ToolMessage(content='Sunny!', name='weather_search', id='ff968b9f-9b87-4893-9f32-dfb88dbe0536', tool_call_id='toolu_01CpbVmprQnjxpQzx8MzE1g8')]}\n",
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='0c488edd-7b9c-4416-ba02-8a2d7e9f2597'), AIMessage(content=[{'text': \"Certainly! I can help you check the weather in San Francisco. To get this information, I'll use the weather search tool. Let me fetch that for you.\", 'type': 'text'}, {'id': 'toolu_01CpbVmprQnjxpQzx8MzE1g8', 'input': {'city': 'San Francisco'}, 'name': 'weather_search', 'type': 'tool_use'}], id='run-52a09799-efb5-4fff-82c3-884e20119ad3-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco, USA'}, 'id': 'toolu_01CpbVmprQnjxpQzx8MzE1g8', 'type': 'tool_call'}]), ToolMessage(content='Sunny!', name='weather_search', id='ff968b9f-9b87-4893-9f32-dfb88dbe0536', tool_call_id='toolu_01CpbVmprQnjxpQzx8MzE1g8'), AIMessage(content=\"Great news! The weather in San Francisco is currently sunny. It's a beautiful day in the city by the bay. Is there anything else you'd like to know about the weather or any other information I can help you with?\", response_metadata={'id': 'msg_01PhwUeRWkSJB6kzHZS361XZ', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 464, 'output_tokens': 50}}, id='run-5aebcf37-626e-4675-b225-476bc99bdbb8-0', usage_metadata={'input_tokens': 464, 'output_tokens': 50, 'total_tokens': 514})]}\n"
     ]
    }
   ],
   "source": [
    "# To get the ID of the message we want to replace, we need to fetch the current state and find it there.\n",
    "state = graph.get_state(thread)\n",
    "print(\"Current State:\")\n",
    "print(state.values)\n",
    "print(\"\\nCurrent Tool Call ID:\")\n",
    "current_content = state.values[\"messages\"][-1].content\n",
    "current_id = state.values[\"messages\"][-1].id\n",
    "tool_call_id = state.values[\"messages\"][-1].tool_calls[0][\"id\"]\n",
    "print(tool_call_id)\n",
    "\n",
    "# We now need to construct a replacement tool call.\n",
    "# We will change the argument to be `San Francisco, USA`\n",
    "# Note that we could change any number of arguments or tool names - it just has to be a valid one\n",
    "new_message = {\n",
    "    \"role\": \"assistant\",\n",
    "    \"content\": current_content,\n",
    "    \"tool_calls\": [\n",
    "        {\n",
    "            \"id\": tool_call_id,\n",
    "            \"name\": \"weather_search\",\n",
    "            \"args\": {\"city\": \"San Francisco, USA\"},\n",
    "        }\n",
    "    ],\n",
    "    # This is important - this needs to be the same as the message you replacing!\n",
    "    # Otherwise, it will show up as a separate message\n",
    "    \"id\": current_id,\n",
    "}\n",
    "graph.update_state(\n",
    "    # This is the config which represents this thread\n",
    "    thread,\n",
    "    # This is the updated value we want to push\n",
    "    {\"messages\": [new_message]},\n",
    "    # We push this update acting as our human_review_node\n",
    "    as_node=\"human_review_node\",\n",
    ")\n",
    "\n",
    "# Let's now continue executing from here\n",
    "for event in graph.stream(None, thread, stream_mode=\"values\"):\n",
    "    print(event)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e14acc96-3d50-44b1-8616-b8d9131e46c4",
   "metadata": {},
   "source": [
    "## Give feedback to a tool call\n",
    "\n",
    "Sometimes, you may not want to execute a tool call, but you also may not want to ask the user to manually modify the tool call. In that case it may be better to get natural language feedback from the user. You can then insert these feedback as a mock **RESULT** of the tool call.\n",
    "\n",
    "There are multiple ways to do this:\n",
    "\n",
    "1. You could add a new message to the state (representing the \"result\" of a tool call)\n",
    "2. You could add TWO new messages to the state - one representing an \"error\" from the tool call, other HumanMessage representing the feedback\n",
    "\n",
    "Both are similar in that they involve adding messages to the state. The main difference lies in the logic AFTER the `human_node` and how it handles different types of messages.\n",
    "\n",
    "For this example we will just add a single tool call representing the feedback. Let's see this in action!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "d57d5131-7912-4216-aa87-b7272507fa51",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='601c4c75-f506-4d91-896d-5e382123de24')]}\n",
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='601c4c75-f506-4d91-896d-5e382123de24'), AIMessage(content=[{'text': \"Certainly! I can help you check the weather in San Francisco. To get the most accurate and up-to-date information, I'll use the weather search tool. Let me fetch that for you right away.\", 'type': 'text'}, {'id': 'toolu_014UTKh5uqfc885Fj4RRqGdg', 'input': {'city': 'San Francisco'}, 'name': 'weather_search', 'type': 'tool_use'}], response_metadata={'id': 'msg_013nHyPYxNXFSoXeS6q4oWua', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 360, 'output_tokens': 98}}, id='run-0537e15e-86a4-4c6f-8dfb-6e4c160812c4-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco'}, 'id': 'toolu_014UTKh5uqfc885Fj4RRqGdg', 'type': 'tool_call'}], usage_metadata={'input_tokens': 360, 'output_tokens': 98, 'total_tokens': 458})]}\n"
     ]
    }
   ],
   "source": [
    "# Input\n",
    "initial_input = {\"messages\": [{\"role\": \"user\", \"content\": \"what's the weather in sf?\"}]}\n",
    "\n",
    "# Thread\n",
    "thread = {\"configurable\": {\"thread_id\": \"6\"}}\n",
    "\n",
    "# Run the graph until the first interruption\n",
    "for event in graph.stream(initial_input, thread, stream_mode=\"values\"):\n",
    "    print(event)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "e33ad664-0307-43c5-b85a-1e02eebceb5c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pending Executions!\n",
      "('human_review_node',)\n"
     ]
    }
   ],
   "source": [
    "print(\"Pending Executions!\")\n",
    "print(graph.get_state(thread).next)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "483d9455-8625-4c6a-9b98-f731403b2ed3",
   "metadata": {},
   "source": [
    "To do this, we first need to update the state. We can do this by passing a message in with the same **tool call id** of the tool call we want to respond to. Note that this is a **different** ID from above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3f05f8b6-6128-4de5-8884-862fc93f1227",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current State:\n",
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='601c4c75-f506-4d91-896d-5e382123de24'), AIMessage(content=[{'text': \"Certainly! I can help you check the weather in San Francisco. To get the most accurate and up-to-date information, I'll use the weather search tool. Let me fetch that for you right away.\", 'type': 'text'}, {'id': 'toolu_014UTKh5uqfc885Fj4RRqGdg', 'input': {'city': 'San Francisco'}, 'name': 'weather_search', 'type': 'tool_use'}], response_metadata={'id': 'msg_013nHyPYxNXFSoXeS6q4oWua', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 360, 'output_tokens': 98}}, id='run-0537e15e-86a4-4c6f-8dfb-6e4c160812c4-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco'}, 'id': 'toolu_014UTKh5uqfc885Fj4RRqGdg', 'type': 'tool_call'}], usage_metadata={'input_tokens': 360, 'output_tokens': 98, 'total_tokens': 458})]}\n",
      "\n",
      "Current Tool Call ID:\n",
      "toolu_014UTKh5uqfc885Fj4RRqGdg\n",
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='601c4c75-f506-4d91-896d-5e382123de24'), AIMessage(content=[{'text': \"Certainly! I can help you check the weather in San Francisco. To get the most accurate and up-to-date information, I'll use the weather search tool. Let me fetch that for you right away.\", 'type': 'text'}, {'id': 'toolu_014UTKh5uqfc885Fj4RRqGdg', 'input': {'city': 'San Francisco'}, 'name': 'weather_search', 'type': 'tool_use'}], response_metadata={'id': 'msg_013nHyPYxNXFSoXeS6q4oWua', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 360, 'output_tokens': 98}}, id='run-0537e15e-86a4-4c6f-8dfb-6e4c160812c4-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco'}, 'id': 'toolu_014UTKh5uqfc885Fj4RRqGdg', 'type': 'tool_call'}], usage_metadata={'input_tokens': 360, 'output_tokens': 98, 'total_tokens': 458}), ToolMessage(content='User requested changes: pass in the country as well', name='weather_search', id='e20ceddc-a0d3-469d-b31e-512f3042a07e', tool_call_id='toolu_014UTKh5uqfc885Fj4RRqGdg'), AIMessage(content=[{'text': \"I apologize for the oversight. It seems that the weather search function requires more specific information. Let's try again with a more detailed search, including the country. Since San Francisco is commonly associated with the one in California, USA, I'll use that. Here's the updated search:\", 'type': 'text'}, {'id': 'toolu_01AaipBbWDLjHnPcoApx8wRq', 'input': {'city': 'San Francisco, USA'}, 'name': 'weather_search', 'type': 'tool_use'}], response_metadata={'id': 'msg_018rErqC2cLe2VVhebdJf81e', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 480, 'output_tokens': 116}}, id='run-fcba65ed-400a-4783-9ecd-e22051682399-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco, USA'}, 'id': 'toolu_01AaipBbWDLjHnPcoApx8wRq', 'type': 'tool_call'}], usage_metadata={'input_tokens': 480, 'output_tokens': 116, 'total_tokens': 596})]}\n"
     ]
    }
   ],
   "source": [
    "# To get the ID of the message we want to replace, we need to fetch the current state and find it there.\n",
    "state = graph.get_state(thread)\n",
    "print(\"Current State:\")\n",
    "print(state.values)\n",
    "print(\"\\nCurrent Tool Call ID:\")\n",
    "tool_call_id = state.values[\"messages\"][-1].tool_calls[0][\"id\"]\n",
    "print(tool_call_id)\n",
    "\n",
    "# We now need to construct a replacement tool call.\n",
    "# We will change the argument to be `San Francisco, USA`\n",
    "# Note that we could change any number of arguments or tool names - it just has to be a valid one\n",
    "new_message = {\n",
    "    \"role\": \"tool\",\n",
    "    # This is our natural language feedback\n",
    "    \"content\": \"User requested changes: pass in the country as well\",\n",
    "    \"name\": \"weather_search\",\n",
    "    \"tool_call_id\": tool_call_id,\n",
    "}\n",
    "graph.update_state(\n",
    "    # This is the config which represents this thread\n",
    "    thread,\n",
    "    # This is the updated value we want to push\n",
    "    {\"messages\": [new_message]},\n",
    "    # We push this update acting as our human_review_node\n",
    "    as_node=\"human_review_node\",\n",
    ")\n",
    "\n",
    "# Let's now continue executing from here\n",
    "for event in graph.stream(None, thread, stream_mode=\"values\"):\n",
    "    print(event)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d2e79ab-7cdb-42ce-b2ca-2932f8782c90",
   "metadata": {},
   "source": [
    "We can see that we now get to another breakpoint - because it went back to the model and got an entirely new prediction of what to call. Let's now approve this one and continue."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "a30d40ad-611d-4ec3-84be-869ea05acb89",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pending Executions!\n",
      "('human_review_node',)\n",
      "----\n",
      "Searching for: San Francisco, USA\n",
      "----\n",
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='601c4c75-f506-4d91-896d-5e382123de24'), AIMessage(content=[{'text': \"Certainly! I can help you check the weather in San Francisco. To get the most accurate and up-to-date information, I'll use the weather search tool. Let me fetch that for you right away.\", 'type': 'text'}, {'id': 'toolu_014UTKh5uqfc885Fj4RRqGdg', 'input': {'city': 'San Francisco'}, 'name': 'weather_search', 'type': 'tool_use'}], response_metadata={'id': 'msg_013nHyPYxNXFSoXeS6q4oWua', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 360, 'output_tokens': 98}}, id='run-0537e15e-86a4-4c6f-8dfb-6e4c160812c4-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco'}, 'id': 'toolu_014UTKh5uqfc885Fj4RRqGdg', 'type': 'tool_call'}], usage_metadata={'input_tokens': 360, 'output_tokens': 98, 'total_tokens': 458}), ToolMessage(content='User requested changes: pass in the country as well', name='weather_search', id='e20ceddc-a0d3-469d-b31e-512f3042a07e', tool_call_id='toolu_014UTKh5uqfc885Fj4RRqGdg'), AIMessage(content=[{'text': \"I apologize for the oversight. It seems that the weather search function requires more specific information. Let's try again with a more detailed search, including the country. Since San Francisco is commonly associated with the one in California, USA, I'll use that. Here's the updated search:\", 'type': 'text'}, {'id': 'toolu_01AaipBbWDLjHnPcoApx8wRq', 'input': {'city': 'San Francisco, USA'}, 'name': 'weather_search', 'type': 'tool_use'}], response_metadata={'id': 'msg_018rErqC2cLe2VVhebdJf81e', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 480, 'output_tokens': 116}}, id='run-fcba65ed-400a-4783-9ecd-e22051682399-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco, USA'}, 'id': 'toolu_01AaipBbWDLjHnPcoApx8wRq', 'type': 'tool_call'}], usage_metadata={'input_tokens': 480, 'output_tokens': 116, 'total_tokens': 596}), ToolMessage(content='Sunny!', name='weather_search', id='3f3ee262-70f5-422c-8e3f-6a9af758514d', tool_call_id='toolu_01AaipBbWDLjHnPcoApx8wRq')]}\n",
      "{'messages': [HumanMessage(content=\"what's the weather in sf?\", id='601c4c75-f506-4d91-896d-5e382123de24'), AIMessage(content=[{'text': \"Certainly! I can help you check the weather in San Francisco. To get the most accurate and up-to-date information, I'll use the weather search tool. Let me fetch that for you right away.\", 'type': 'text'}, {'id': 'toolu_014UTKh5uqfc885Fj4RRqGdg', 'input': {'city': 'San Francisco'}, 'name': 'weather_search', 'type': 'tool_use'}], response_metadata={'id': 'msg_013nHyPYxNXFSoXeS6q4oWua', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 360, 'output_tokens': 98}}, id='run-0537e15e-86a4-4c6f-8dfb-6e4c160812c4-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco'}, 'id': 'toolu_014UTKh5uqfc885Fj4RRqGdg', 'type': 'tool_call'}], usage_metadata={'input_tokens': 360, 'output_tokens': 98, 'total_tokens': 458}), ToolMessage(content='User requested changes: pass in the country as well', name='weather_search', id='e20ceddc-a0d3-469d-b31e-512f3042a07e', tool_call_id='toolu_014UTKh5uqfc885Fj4RRqGdg'), AIMessage(content=[{'text': \"I apologize for the oversight. It seems that the weather search function requires more specific information. Let's try again with a more detailed search, including the country. Since San Francisco is commonly associated with the one in California, USA, I'll use that. Here's the updated search:\", 'type': 'text'}, {'id': 'toolu_01AaipBbWDLjHnPcoApx8wRq', 'input': {'city': 'San Francisco, USA'}, 'name': 'weather_search', 'type': 'tool_use'}], response_metadata={'id': 'msg_018rErqC2cLe2VVhebdJf81e', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 480, 'output_tokens': 116}}, id='run-fcba65ed-400a-4783-9ecd-e22051682399-0', tool_calls=[{'name': 'weather_search', 'args': {'city': 'San Francisco, USA'}, 'id': 'toolu_01AaipBbWDLjHnPcoApx8wRq', 'type': 'tool_call'}], usage_metadata={'input_tokens': 480, 'output_tokens': 116, 'total_tokens': 596}), ToolMessage(content='Sunny!', name='weather_search', id='3f3ee262-70f5-422c-8e3f-6a9af758514d', tool_call_id='toolu_01AaipBbWDLjHnPcoApx8wRq'), AIMessage(content=\"Great news! The weather in San Francisco, USA is currently sunny. \\n\\nHere's a summary of the weather information:\\n- Location: San Francisco, USA\\n- Current conditions: Sunny\\n\\nIt's a beautiful day in San Francisco! The sunny weather is perfect for outdoor activities or simply enjoying the city. Remember to wear sunscreen and stay hydrated if you plan to spend time outside. \\n\\nIs there anything else you'd like to know about the weather in San Francisco or any other location?\", response_metadata={'id': 'msg_017Pnjyte2ZXAREgUvEqbUVt', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 609, 'output_tokens': 107}}, id='run-30c0d0ef-09a3-40ad-b410-80019b284983-0', usage_metadata={'input_tokens': 609, 'output_tokens': 107, 'total_tokens': 716})]}\n"
     ]
    }
   ],
   "source": [
    "print(\"Pending Executions!\")\n",
    "print(graph.get_state(thread).next)\n",
    "\n",
    "for event in graph.stream(None, thread, stream_mode=\"values\"):\n",
    "    print(event)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}