update tool use overview

fern/pages/v2/tool-use/tool-use-overview.mdx

@@ -44,7 +44,7 @@ The pre-requisite, or Step 0, before we can run a tool use workflow, is to defin
 
 ### Creating the tool
 
-A tool can be any function that you create or extrenal services that return an object for a given input. Some examples: a web search engine, an email service, an SQL database, a vector database, a weather data service, a sports data service, or even another LLM.
+A tool can be any function that you create or external services that return an object for a given input. Some examples: a web search engine, an email service, an SQL database, a vector database, a weather data service, a sports data service, or even another LLM.
 
 In this example, we define a `get_weather` function that returns the temperature for a given query, which is the location. You can implement any logic here, but to simplify the example, here we are hardcoding the return value to be the same for all queries.
 
@@ -112,7 +112,7 @@ At its most basic, these four components interact in a workflow through four ste
 <img src="https://cohere.com/_next/image?url=https%3A%2F%2Fcohere-ai.ghost.io%2Fcontent%2Fimages%2F2024%2F07%2Ftool-use-workflow-2.png&w=3840&q=75" alt="Tool use workflow" />
 
 ### Step 1: Get user message
-In the first step, we get the user's message and put it in the `messages` list with the `role` set to `user`.
+In the first step, we get the user's message and append it to the `messages` list with the `role` set to `user`.
 
 ```python PYTHON
 messages = [{"role": "user", "content": "What's the weather in Toronto?"}]
@@ -145,7 +145,7 @@ The endpoint will send back a list of tool calls to be made if the model determi
 - `tool_plan`: its reflection on the next steps it should take, given the user query.
 - `tool_calls`: a list of tool calls to be made (if any), together with the tool call IDs.
 
-We then add this information to the `messages` list with the `role` set to `assistant`.
+We then append this information to the `messages` list with the `role` set to `assistant`.
 
 ```python PYTHON
 response = co.chat(
@@ -158,23 +158,34 @@ if response.message.tool_calls:
     messages.append(
         {
             "role": "assistant",
-            "tool_calls": response.message.tool_calls,
             "tool_plan": response.message.tool_plan,
+            "tool_calls": response.message.tool_calls,
         }
     )
+    print(response.message.tool_plan, "\n")
     print(response.message.tool_calls)
 ```
 
 ```mdx wordWrap
-[ToolCallV2(id='get_weather_776n8ctsgycn', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"Toronto"}'))]
+I will search for the weather in Toronto. 
+
+[
+    ToolCallV2(
+        id="get_weather_1byjy32y4hvq",
+        type="function",
+        function=ToolCallV2Function(
+            name="get_weather", arguments='{"location":"Toronto"}'
+        ),
+    )
+]
 ```
 
 ### Step 3: Get tool results
 During this step, we perform the function calling. We call the necessary tools based on the tool call payloads given by the endpoint.
 
 For each tool call, we append the tool results to the `tool_content` list with `type` set to `document` and `document` set to the tool results (in JSON string format).
 
-We then add this information to the `messages` list with the `role` set to `tool`, together with the tool call IDs that were generated in the previous step.
+We then append this information to the `messages` list with the `role` set to `tool`, together with the tool call IDs that were generated in the previous step.
 
 ```python PYTHON
 import json
@@ -200,45 +211,242 @@ In this step, we call the Chat endpoint to generate the response to the user, ag
 
 The model generates a response to the user, grounded on the information provided by the tool.
 
-It also generates fine-grained citations, which are included out-of-the-box with the Command family of models. Here, we see the model generating two citations, one for each specific span in its response, where it uses the tool result to answer the question.
+We then append the response to the `messages` list with the `role` set to `assistant`.
 
 ```python PYTHON
 response = co.chat(
     model="command-r-plus-08-2024",
     messages=messages,
     tools=tools
 )
+
+messages.append(
+    {"role": "assistant", "content": response.message.content[0].text}
+)
+
 print(response.message.content[0].text)
 ```
 ```mdx wordWrap
-It is 20C in Toronto.
+It's 20°C in Toronto.
+```
+It also generates fine-grained citations, which are included out-of-the-box with the Command family of models. Here, we see the model generating two citations, one for each specific span in its response, where it uses the tool result to answer the question.
+
+```python PYTHON
+print(response.message.citations)
 ```
 ```mdx wordWrap
-start=6 end=9 text='20C' sources=[ToolSource(type='tool', id='get_weather_776n8ctsgycn:0', tool_output={'temperature': '20C'})] 
+[Citation(start=5, end=9, text='20°C', sources=[ToolSource(type='tool', id='get_weather_1byjy32y4hvq:0', tool_output={'temperature': '20C'})], type='TEXT_CONTENT')]
 ```
 
 ## Parallel tool calling
-[[TODO - demonstrate example of parallel tool calling]]
+The LLM can also determine that more than one tool call is required, where it will call multiple tools in parallel. This can be calling the same tool multiple times or different tools for any number of calls.
+
+In the example below, the user asks for the weather in Toronto and New York. This requires calling the `get_weather` function twice, one for each location. This is reflected in the model's response, where two parallel tool calls are generated.
+
+```python PYTHON
+messages = [{"role": "user", "content": "What's the weather in Toronto and New York?"}]
+
+response = co.chat(
+    model="command-r-plus-08-2024",
+    messages=messages,
+    tools=tools
+)
+
+if response.message.tool_calls:
+    messages.append(
+        {
+            "role": "assistant",
+            "tool_plan": response.message.tool_plan,
+            "tool_calls": response.message.tool_calls,
+        }
+    )
+    print(response.message.tool_plan, "\n")
+    print(response.message.tool_calls)
+```
+```mdx wordWrap
+I will search for the weather in Toronto and New York.
+
+[
+    ToolCallV2(
+        id="get_weather_9b0nr4kg58a8",
+        type="function",
+        function=ToolCallV2Function(
+            name="get_weather", arguments='{"location":"Toronto"}'
+        ),
+    ),
+    ToolCallV2(
+        id="get_weather_0qq0mz9gwnqr",
+        type="function",
+        function=ToolCallV2Function(
+            name="get_weather", arguments='{"location":"New York"}'
+        ),
+    ),
+]
+```
 
 ## Directly answering
-[[TODO - describe the scenario where the LLM decides not to use tools but instead directly answers the user]]
+A key attribute of tool use systems is the model’s ability to choose the right tools for a task. This includes the ability to decide to *not* use any tool, and instead, respond to a user message directly.
+
+In the example below, the user asks for a simple arithmetic question. The model determines that it does not need to use any of the available tools (only one, `get_weather`, in this case), and instead, directly answers the user.
+
+```python PYTHON
+messages = [{"role": "user", "content": "What's 2+2?"}]
+
+response = co.chat(
+    model="command-r-plus-08-2024",
+    messages=messages,
+    tools=tools
+)
+
+if response.message.tool_calls:
+    print(response.message.tool_plan, "\n")
+    print(response.message.tool_calls)
+
+else:
+    print(response.message.content[0].text)
+```
+```mdx wordWrap
+The answer to 2+2 is 4.
+```
 
 ## Forcing tool usage
-[[TODO - describe the tool choice parameter]]
 
-## Response object
+<Note>This feature is only compatible with the [Command R7B](https://docs.cohere.com/v2/docs/command-r7b) and newer models.</Note>
 
-### Tool calling step
-[[TODO - describe the response object for tool calling step]]
+As we saw in the previous examples, during the tool calling step, the model may decide to either:
+- make tool call(s)
+- or, respond to a user message directly.
 
-### Response generation step
-[[TODO - describe the response object for response generation step]]
+You can, however, force the model to choose one of these options. This is done via the `tool_choice` parameter.
+- You can force the model to make tool call(s), i.e. to not respond directly, by setting the `tool_choice` parameter to `REQUIRED`.
+- Alternatively, you can force the model to respond directly, i.e. to not make tool call(s), by setting the `tool_choice` parameter to `NONE`.
 
+By default, if you don’t specify the `tool_choice` parameter, then the model will decide between making tool calls or responding directly.
+
+```python PYTHON {5}
+response = co.chat(
+    model="command-r-plus-08-2024",
+    messages=messages,
+    tools=tools,
+    tool_choice="REQUIRED" # optional, to force tool calls
+    # tool_choice="NONE" # optional, to force a direct response
+)
+```
 ## State management
-[[TODO - describe the state management via the messages list - single and multi turn. show examples]]
+This section gives a more detailed look at how the state is managed via the `messages` list.
+
+As described in the [tool use workflow](#tool-use-workflow) section above, at each step of the workflow, the endpoint requires that we append specific types of information to the `messages` list. This is to ensure that the model has the necessary context to generate its response at a given point.
 
 ### Single turn
-[[TODO - describe state management wrt single turn scenarios]]
+In summary, each single turn of a conversation that involves tool calling consists of:
+1. a `user` message containing the user message (`content`)
+2. an `assistant` message, containing the tool calling information (`tool_plan` and `tool_calls`)
+3. a `tool` message, containing the tool results (`tool_call_id` and `content`)
+4. a final `assistant` message, containing the response to the user (`content`)
+
+These correspond to the four steps described in the [tool use workflow](#tool-use-workflow) section above.
+
+The following is the list of messages from the example in the [tool use workflow](#tool-use-workflow) section.
+
+```python PYTHON
+for message in messages:
+    print(message, "\n")
+```
+
+```json
+{   
+    "role": "user", 
+    "content": "What's the weather in Toronto?"
+}
+
+{
+    "role": "assistant",
+    "tool_plan": "I will search for the weather in Toronto.",
+    "tool_calls": [
+        ToolCallV2(
+            id="get_weather_1byjy32y4hvq",
+            type="function",
+            function=ToolCallV2Function(
+                name="get_weather", arguments='{"location":"Toronto"}'
+            ),
+        )
+    ],
+}
+
+{
+    "role": "tool",
+    "tool_call_id": "get_weather_1byjy32y4hvq",
+    "content": [{"type": "document", "document": {"data": '{"temperature": "20C"}'}}],
+}
+
+{   
+    "role": "assistant", 
+    "content": "It's 20°C in Toronto."
+}
+```
+
+Note that this sequence of messages will be different in these scenarios:
+- **Multi-step**: In a multi-step tool use scenario, instead of just one occurence of assistant-tool messages (items 2 and 3 above), there will be a sequence of assistant-tool messages to reflect the multiple steps of tool calling involved. This is further described in the [multi-step tool use documentation]([[TODO - add link]])
+- **Directly answering**: When the model decides to respond directly to the user, there will be no items 2 and 3 above (the tool calling and tool response messages). Instead, the final `assistant` message will contain the model's direct response to the user.
+
+### Chatbots
+
+When building chatbots, we'll need to maintain the state of a conversation over multiple turns. In this case, we can keep appending to the `messages` list with the sequence of described in the [state management](#state-management) section above.
+
+As an example, here's the messages list from the first conversation turn.
+```python PYTHON
+from cohere import ToolCallV2, ToolCallV2Function
+
+messages = [
+    {"role": "user", "content": "What's the weather in Toronto?"},
+    {
+        "role": "assistant",
+        "tool_plan": "I will search for the weather in Toronto.",
+        "tool_calls": [
+            ToolCallV2(
+                id="get_weather_1byjy32y4hvq",
+                type="function",
+                function=ToolCallV2Function(
+                    name="get_weather", arguments='{"location":"Toronto"}'
+                ),
+            )
+        ],
+    },
+    {
+        "role": "tool",
+        "tool_call_id": "get_weather_1byjy32y4hvq",
+        "content": [
+            {"type": "document", "document": {"data": '{"temperature": "20C"}'}}
+        ],
+    },
+    {"role": "assistant", "content": "It's 20°C in Toronto."},
+]
+```
+Then, given a follow-up user message, the model correctly infers that the context of the user's message is about the weather.
+
+```python PYTHON
+messages.append({"role": "user", "content": "What about London?"})
+
+response = co.chat(
+    model="command-r-plus-08-2024",
+    messages=messages,
+    tools=tools
+)
+
+if response.message.tool_calls:
+    messages.append(
+        {
+            "role": "assistant",
+            "tool_plan": response.message.tool_plan,
+            "tool_calls": response.message.tool_calls,
+        }
+    )
+    print(response.message.tool_plan, "\n")
+    print(response.message.tool_calls)
+```
+```mdx wordWrap
+I will search for the weather in London. 
+
+[ToolCallV2(id='get_weather_8hwpm7d4wr14', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"London"}'))]
+```
 
-### Multi turn
-[[TODO - describe state management wrt multi turn scenarios]]

fern/pages/v2/tool-use/tool-use-tool-definition.mdx

@@ -17,7 +17,7 @@ updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)"
 ## Tool creation
 
 ### Example: Custom functions
-[[TODO - an example of a tool with basic working logic - e.g. sales database]]
+[[TODO - an example of a tool with basic working logic - e.g. sales database - example more than 1 tool]]
 
 ### Example: External services
 [[TODO - an example of a tool with basic working logic - e.g. web search]]