examples/Creating_slides_with_Assistants_API_and_DALL-E3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating slides with the Assistants API (GPT-4), and DALL·E-3\n",
    "\n",
    "This notebook illustrates the use of the new [Assistants API](https://platform.openai.com/docs/assistants/overview) (GPT-4), and DALL·E-3 in crafting informative and visually appealing slides. <br>\n",
    "Creating slides is a pivotal aspect of many jobs, but can be laborious and time-consuming. Additionally, extracting insights from data and articulating them effectively on slides can be challenging. <br><br> This cookbook recipe will demonstrate how you can utilize the new Assistants API to facilitate the end to end slide creation process for you without you having to touch Microsoft PowerPoint or Google Slides, saving you valuable time and effort!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 0. Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import display, Image\n",
    "from openai import OpenAI\n",
    "import os\n",
    "import pandas as pd\n",
    "import json\n",
    "import io\n",
    "from PIL import Image\n",
    "import requests\n",
    "\n",
    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n",
    "\n",
    "#Lets import some helper functions for assistants from https://cookbook.openai.com/examples/assistants_api_overview_python\n",
    "def show_json(obj):\n",
    "    display(json.loads(obj.model_dump_json()))\n",
    "\n",
    "def submit_message(assistant_id, thread, user_message,file_ids=None):\n",
    "    params = {\n",
    "        'thread_id': thread.id,\n",
    "        'role': 'user',\n",
    "        'content': user_message,\n",
    "    }\n",
    "    if file_ids:\n",
    "        params['file_ids']=file_ids\n",
    "\n",
    "    client.beta.threads.messages.create(\n",
    "        **params\n",
    ")\n",
    "    return client.beta.threads.runs.create(\n",
    "    thread_id=thread.id,\n",
    "    assistant_id=assistant_id,\n",
    ")\n",
    "\n",
    "def get_response(thread):\n",
    "    return client.beta.threads.messages.list(thread_id=thread.id)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Creating the content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this recipe, we will be creating a brief fictional presentation for the quarterly financial review of our company, NotReal Corporation. We want to highlight some key trends we are seeing that are affecting the profitability of our company.<br> Let's say we have the some financial data at our disposal. Let's load in the data, and take a look..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Quarter</th>\n",
       "      <th>Distribution channel</th>\n",
       "      <th>Revenue ($M)</th>\n",
       "      <th>Costs ($M)</th>\n",
       "      <th>Customer count</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2021</td>\n",
       "      <td>Q1</td>\n",
       "      <td>Online Sales</td>\n",
       "      <td>1.50</td>\n",
       "      <td>1.301953</td>\n",
       "      <td>150</td>\n",
       "      <td>2021 Q1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2021</td>\n",
       "      <td>Q1</td>\n",
       "      <td>Direct Sales</td>\n",
       "      <td>1.50</td>\n",
       "      <td>1.380809</td>\n",
       "      <td>151</td>\n",
       "      <td>2021 Q1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2021</td>\n",
       "      <td>Q1</td>\n",
       "      <td>Retail Partners</td>\n",
       "      <td>1.50</td>\n",
       "      <td>1.348246</td>\n",
       "      <td>152</td>\n",
       "      <td>2021 Q1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2021</td>\n",
       "      <td>Q2</td>\n",
       "      <td>Online Sales</td>\n",
       "      <td>1.52</td>\n",
       "      <td>1.308608</td>\n",
       "      <td>152</td>\n",
       "      <td>2021 Q2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2021</td>\n",
       "      <td>Q2</td>\n",
       "      <td>Direct Sales</td>\n",
       "      <td>1.52</td>\n",
       "      <td>1.413305</td>\n",
       "      <td>153</td>\n",
       "      <td>2021 Q2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Year Quarter Distribution channel  Revenue ($M)  Costs ($M)  \\\n",
       "0  2021      Q1         Online Sales          1.50    1.301953   \n",
       "1  2021      Q1         Direct Sales          1.50    1.380809   \n",
       "2  2021      Q1      Retail Partners          1.50    1.348246   \n",
       "3  2021      Q2         Online Sales          1.52    1.308608   \n",
       "4  2021      Q2         Direct Sales          1.52    1.413305   \n",
       "\n",
       "   Customer count     Time  \n",
       "0             150  2021 Q1  \n",
       "1             151  2021 Q1  \n",
       "2             152  2021 Q1  \n",
       "3             152  2021 Q2  \n",
       "4             153  2021 Q2  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "financial_data_path = 'data/NotRealCorp_financial_data.json'\n",
    "financial_data = pd.read_json(financial_data_path)\n",
    "financial_data.head(5)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, this data has quarterly revenue, costs and customer data across different distribution channels. Let's create an Assistant\n",
    "that can act as a personal analyst and make a nice visualization for our PowerPoint!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we need to upload our file so our Assistant can access it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "file = client.files.create(\n",
    "  file=open('data/NotRealCorp_financial_data.json',\"rb\"),\n",
    "  purpose='assistants',\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we're ready to create our Assistant. We can instruct our assistant to act as a data scientist, and take any queries we give it and run the necessary code to output the proper data visualization. The instructions parameter here is akin to system instructions in the ChatCompletions endpoint, and can help guide the assistant. We can also turn on the tool of Code Interpreter, so our Assistant will be able to code. Finally, we can specifiy any files we want to use, which in this case is just the `financial_data` file we created above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "assistant = client.beta.assistants.create(\n",
    "  instructions=\"You are a data scientist assistant. When given data and a query, write the proper code and create the proper visualization\",\n",
    "  model=\"gpt-4-1106-preview\",\n",
    "  tools=[{\"type\": \"code_interpreter\"}],\n",
    "  file_ids=[file.id]\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a thread now, and as our first request ask the Assistant to calculate quarterly profits, and then plot the profits by distribution channel over time. The assistant will automatically calculate the profit for each quarter, and also create a new column combining quarter and year, without us having to ask for that directly. We can also specify the colors of each line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "thread = client.beta.threads.create(\n",
    "  messages=[\n",
    "    {\n",
    "      \"role\": \"user\",\n",
    "      \"content\": \"Calculate profit (revenue minus cost) by quarter and year, and visualize as a line plot across the distribution channels, where the colors of the lines are green, light red, and light blue\",\n",
    "      \"file_ids\": [file.id]\n",
    "    }\n",
    "  ]\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "No we can execute the run of our thread"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "run = client.beta.threads.runs.create(\n",
    "    thread_id=thread.id,\n",
    "    assistant_id=assistant.id,\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now start a loop that will check if the image has been created. Note: This may take a few minutes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "messages = client.beta.threads.messages.list(thread_id=thread.id)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Assistant still working...\n",
      "Plot created!\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "while True:\n",
    "    messages = client.beta.threads.messages.list(thread_id=thread.id)\n",
    "    try:\n",
    "        #See if image has been created\n",
    "        messages.data[0].content[0].image_file\n",
    "        #Sleep to make sure run has completed\n",
    "        time.sleep(5)\n",
    "        print('Plot created!')\n",
    "        break\n",
    "    except:\n",
    "        time.sleep(10)\n",
    "        print('Assistant still working...')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see the messages the Assistant added."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[MessageContentImageFile(image_file=ImageFile(file_id='file-0rKABLygI02MgwwhpgWdRFY1'), type='image_file'),\n",
       " MessageContentText(text=Text(annotations=[], value=\"The profit has been calculated for each distribution channel by quarter and year. Next, I'll create a line plot to visualize these profits. As specified, I will use green for the 'Online Sales', light red for 'Direct Sales', and light blue for 'Retail Partners' channels. Let's create the plot.\"), type='text'),\n",
       " MessageContentText(text=Text(annotations=[], value=\"The JSON data has been successfully restructured into a tabular dataframe format. It includes the year, quarter, distribution channel, revenue, costs, customer count, and a combined 'Time' representation of 'Year Quarter'. Now, we have the necessary information to calculate the profit (revenue minus cost) by quarter and year.\\n\\nTo visualize the profit across the different distribution channels with a line plot, we will proceed with the following steps:\\n\\n1. Calculate the profit for each row in the dataframe.\\n2. Group the data by 'Time' (which is a combination of Year and Quarter) and 'Distribution channel'.\\n3. Aggregate the profit for each group.\\n4. Plot the aggregated profits as a line plot with the distribution channels represented in different colors as requested.\\n\\nLet's calculate the profit for each row and then continue with the visualization.\"), type='text'),\n",
       " MessageContentText(text=Text(annotations=[], value='The structure of the JSON data shows that it is a dictionary with \"Year\", \"Quarter\", \"Distribution channel\", and potentially other keys that map to dictionaries containing the data. The keys of the inner dictionaries are indices, indicating that the data is tabular but has been converted into a JSON object.\\n\\nTo properly convert this data into a DataFrame, I will restructure the JSON data into a more typical list of dictionaries, where each dictionary represents a row in our target DataFrame. Subsequent to this restructuring, I can then load the data into a Pandas DataFrame. Let\\'s restructure and load the data.'), type='text'),\n",
       " MessageContentText(text=Text(annotations=[], value=\"The JSON data has been incorrectly loaded into a single-row DataFrame with numerous columns representing each data point. This implies the JSON structure is not as straightforward as expected, and a direct conversion to a flat table is not possible without further processing.\\n\\nTo better understand the JSON structure and figure out how to properly normalize it into a table format, I will print out the raw JSON data structure. We will analyze its format and then determine the correct approach to extract the profit by quarter and year, as well as the distribution channel information. Let's take a look at the JSON structure.\"), type='text'),\n",
       " MessageContentText(text=Text(annotations=[], value=\"It seems that the file content was successfully parsed as JSON, and thus, there was no exception raised. The variable `error_message` is not defined because the `except` block was not executed.\\n\\nI'll proceed with displaying the data that was parsed from JSON.\"), type='text'),\n",
       " MessageContentText(text=Text(annotations=[], value=\"It appears that the content of the dataframe has been incorrectly parsed, resulting in an empty dataframe with a very long column name that seems to contain JSON data rather than typical CSV columns and rows.\\n\\nTo address this issue, I will take a different approach to reading the file. I will attempt to parse the content as JSON. If this is not successful, I'll adjust the loading strategy accordingly. Let's try to read the contents as JSON data first.\"), type='text'),\n",
       " MessageContentText(text=Text(annotations=[], value=\"Before we can calculate profits and visualize the data as requested, I need to first examine the contents of the file that you have uploaded. Let's go ahead and read the file to understand its structure and the kind of data it contains. Once I have a clearer picture of the dataset, we can proceed with the profit calculations. I'll begin by loading the file into a dataframe and displaying the first few entries to see the data schema.\"), type='text'),\n",
       " MessageContentText(text=Text(annotations=[], value='Calculate profit (revenue minus cost) by quarter and year, and visualize as a line plot across the distribution channels, where the colors of the lines are green, light red, and light blue'), type='text')]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "messages = client.beta.threads.messages.list(thread_id=thread.id)\n",
    "[message.content[0] for message in messages.data]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the last message (latest message is shown first) from the assistant contains the image file we are looking for. An interesting note here is that the Assistant was able to attempt several times to parse the JSON data, as the first parsing was unsuccessful, demonstrating the assistant's adaptability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Quick helper function to convert our output file to a png\n",
    "def convert_file_to_png(file_id, write_path):\n",
    "    data = client.files.content(file_id)\n",
    "    data_bytes = data.read()\n",
    "    with open(write_path, \"wb\") as file:\n",
    "        file.write(data_bytes)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_file_id = messages.data[0].content[0].image_file.file_id\n",
    "image_path = \"../images/NotRealCorp_chart.png\"\n",
    "convert_file_to_png(plot_file_id,image_path)\n",
    "\n",
    "#Upload\n",
    "plot_file = client.files.create(\n",
    "  file=open(image_path, \"rb\"),\n",
    "  purpose='assistants'\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's load in the plot!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![The Image](../images/NotRealCorp_chart.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nice! So, with just one sentence, we were able to have our assistant use code interpreter to\n",
    "calculate the profitability, and graph the three lineplots of the various distribution channels.<br><br>\n",
    "Now we have a nice visual for our slide, but we want some insights to go along with it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Generating insights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get insights from our image, we simply need to add a new message to our thread. Our Assistant will know to use the message history to give us some concise takeaways from the visual provided. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Run(id='run_NWoygMcBfHUr58fCE4Cn6rxN', assistant_id='asst_3T362kLlTyAq0FUnkvjjQczO', cancelled_at=None, completed_at=None, created_at=1701827074, expires_at=1701827674, failed_at=None, file_ids=['file-piTokyHGllwGITzIpoG8dok3'], instructions='You are a data scientist assistant. When given data and a query, write the proper code and create the proper visualization', last_error=None, metadata={}, model='gpt-4-1106-preview', object='thread.run', required_action=None, started_at=None, status='queued', thread_id='thread_73TgtFoJMlEJvb13ngjTnAo3', tools=[ToolAssistantToolsCode(type='code_interpreter')])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submit_message(assistant.id,thread,\"Give me two medium length sentences (~20-30 words per sentence) of the \\\n",
    "      most important insights from the plot you just created.\\\n",
    "             These will be used for a slide deck, and they should be about the\\\n",
    "                     'so what' behind the data.\"\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, once the run has completed, we can see the latest message"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The plot reveals a consistent upward trend in profits for all distribution channels, indicating successful business growth over time. Particularly, 'Online Sales' shows a notable increase, underscoring the importance of digital presence in revenue generation.\n"
     ]
    }
   ],
   "source": [
    "# Hard coded wait for a response, as the assistant may iterate on the bullets.\n",
    "time.sleep(10)\n",
    "response = get_response(thread)\n",
    "bullet_points = response.data[0].content[0].text.value\n",
    "print(bullet_points)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cool! So our assistant was able to identify the noteworthy growth in Online Sales profit, and infer that this shows the importance of a large digital presence. Now let's get a compelling title for the slide."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Run(id='run_q6E85J31jCw3QkHpjJKl969P', assistant_id='asst_3T362kLlTyAq0FUnkvjjQczO', cancelled_at=None, completed_at=None, created_at=1701827084, expires_at=1701827684, failed_at=None, file_ids=['file-piTokyHGllwGITzIpoG8dok3'], instructions='You are a data scientist assistant. When given data and a query, write the proper code and create the proper visualization', last_error=None, metadata={}, model='gpt-4-1106-preview', object='thread.run', required_action=None, started_at=None, status='queued', thread_id='thread_73TgtFoJMlEJvb13ngjTnAo3', tools=[ToolAssistantToolsCode(type='code_interpreter')])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submit_message(assistant.id,thread,\"Given the plot and bullet points you created,\\\n",
    " come up with a very brief title for a slide. It should reflect just the main insights you came up with.\"\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the title is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\"Ascending Profits & Digital Dominance\"\n"
     ]
    }
   ],
   "source": [
    "#Wait as assistant may take a few steps\n",
    "time.sleep(10)\n",
    "response = get_response(thread)\n",
    "title = response.data[0].content[0].text.value\n",
    "print(title)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. DALL·E-3 title image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nice, now we have a title, a plot and two bullet points. We're almost ready to put this all on a slide, but as a final step, let's have DALL·E-3 come up with an image to use as the title slide of the presentation. <br><br>\n",
    "*Note:* DALL·E-3 is not yet available within the assistants API but is coming soon! <br> <br>\n",
    "We'll feed in a brief description of our company (NotRealCorp) and have DALL·E-3 do the rest!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "company_summary = \"NotReal Corp is a prominent hardware company that manufactures and sells processors, graphics cards and other essential computer hardware.\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = client.images.generate(\n",
    "  model='dall-e-3',\n",
    "  prompt=f\"given this company summary {company_summary}, create an inspirational \\\n",
    "    photo showing the growth and path forward. This will be used at a quarterly\\\n",
    "       financial planning meeting\",\n",
    "       size=\"1024x1024\",\n",
    "       quality=\"hd\",\n",
    "       n=1\n",
    ")\n",
    "image_url = response.data[0].url\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cool, now we can add this image to our thread. First, we can save the image locally, then upload it to the assistants API using the `File` upload endpoint. Let's also take a look at our image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "dalle_img_path = '../images/dalle_image.png'\n",
    "img = requests.get(image_url)\n",
    "\n",
    "#Save locally\n",
    "with open(dalle_img_path,'wb') as file:\n",
    "  file.write(img.content)\n",
    "\n",
    "#Upload\n",
    "dalle_file = client.files.create(\n",
    "  file=open(dalle_img_path, \"rb\"),\n",
    "  purpose='assistants'\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "  \n",
    "![Image](../images/dalle_image.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Creating the slides"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now have all the content we need to create the slides. While we could simply add a message asking for slides, but let's instead give the assistant a slide template, using the `python-pptx` library, to use. This will ensure we get a deck in the style we want. See the `Extensions` section at the end of the notebook for notes on creating the template."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "title_template = \"\"\"\n",
    "from pptx import Presentation\n",
    "from pptx.util import Inches, Pt\n",
    "from pptx.enum.text import PP_PARAGRAPH_ALIGNMENT\n",
    "from pptx.dml.color import RGBColor\n",
    "\n",
    "# Create a new presentation object\n",
    "prs = Presentation()\n",
    "\n",
    "# Add a blank slide layout\n",
    "blank_slide_layout = prs.slide_layouts[6]\n",
    "slide = prs.slides.add_slide(blank_slide_layout)\n",
    "\n",
    "# Set the background color of the slide to black\n",
    "background = slide.background\n",
    "fill = background.fill\n",
    "fill.solid()\n",
    "fill.fore_color.rgb = RGBColor(0, 0, 0)\n",
    "\n",
    "# Add image to the left side of the slide with a margin at the top and bottom\n",
    "left = Inches(0)\n",
    "top = Inches(0)\n",
    "height = prs.slide_height\n",
    "width = prs.slide_width * 3/5\n",
    "pic = slide.shapes.add_picture(image_path, left, top, width=width, height=height)\n",
    "\n",
    "# Add title text box positioned higher\n",
    "left = prs.slide_width * 3/5\n",
    "top = Inches(2)\n",
    "width = prs.slide_width * 2/5\n",
    "height = Inches(1)\n",
    "title_box = slide.shapes.add_textbox(left, top, width, height)\n",
    "title_frame = title_box.text_frame\n",
    "title_p = title_frame.add_paragraph()\n",
    "title_p.text = title_text\n",
    "title_p.font.bold = True\n",
    "title_p.font.size = Pt(38)\n",
    "title_p.font.color.rgb = RGBColor(255, 255, 255)\n",
    "title_p.alignment = PP_PARAGRAPH_ALIGNMENT.CENTER\n",
    "\n",
    "# Add subtitle text box\n",
    "left = prs.slide_width * 3/5\n",
    "top = Inches(3)\n",
    "width = prs.slide_width * 2/5\n",
    "height = Inches(1)\n",
    "subtitle_box = slide.shapes.add_textbox(left, top, width, height)\n",
    "subtitle_frame = subtitle_box.text_frame\n",
    "subtitle_p = subtitle_frame.add_paragraph()\n",
    "subtitle_p.text = subtitle_text\n",
    "subtitle_p.font.size = Pt(22)\n",
    "subtitle_p.font.color.rgb = RGBColor(255, 255, 255)\n",
    "subtitle_p.alignment = PP_PARAGRAPH_ALIGNMENT.CENTER\n",
    "\"\"\"\n",
    "\n",
    "data_vis_template = \"\"\"\n",
    "from pptx import Presentation\n",
    "from pptx.util import Inches, Pt\n",
    "from pptx.enum.text import PP_PARAGRAPH_ALIGNMENT\n",
    "from pptx.dml.color import RGBColor\n",
    "\n",
    "# Create a new presentation object\n",
    "prs = Presentation()\n",
    "\n",
    "# Add a blank slide layout\n",
    "blank_slide_layout = prs.slide_layouts[6]\n",
    "slide = prs.slides.add_slide(blank_slide_layout)\n",
    "\n",
    "# Set the background color of the slide to black\n",
    "background = slide.background\n",
    "fill = background.fill\n",
    "fill.solid()\n",
    "fill.fore_color.rgb = RGBColor(0, 0, 0)\n",
    "\n",
    "# Define placeholders\n",
    "image_path = data_vis_img\n",
    "title_text = \"Maximizing Profits: The Dominance of Online Sales & Direct Sales Optimization\"\n",
    "bullet_points = \"• Online Sales consistently lead in profitability across quarters, indicating a strong digital market presence.\\n• Direct Sales show fluctuations, suggesting variable performance and the need for targeted improvements in that channel.\"\n",
    "\n",
    "# Add image placeholder on the left side of the slide\n",
    "left = Inches(0.2)\n",
    "top = Inches(1.8)\n",
    "height = prs.slide_height - Inches(3)\n",
    "width = prs.slide_width * 3/5\n",
    "pic = slide.shapes.add_picture(image_path, left, top, width=width, height=height)\n",
    "\n",
    "# Add title text spanning the whole width\n",
    "left = Inches(0)\n",
    "top = Inches(0)\n",
    "width = prs.slide_width\n",
    "height = Inches(1)\n",
    "title_box = slide.shapes.add_textbox(left, top, width, height)\n",
    "title_frame = title_box.text_frame\n",
    "title_frame.margin_top = Inches(0.1)\n",
    "title_p = title_frame.add_paragraph()\n",
    "title_p.text = title_text\n",
    "title_p.font.bold = True\n",
    "title_p.font.size = Pt(28)\n",
    "title_p.font.color.rgb = RGBColor(255, 255, 255)\n",
    "title_p.alignment = PP_PARAGRAPH_ALIGNMENT.CENTER\n",
    "\n",
    "# Add hardcoded \"Key Insights\" text and bullet points\n",
    "left = prs.slide_width * 2/3\n",
    "top = Inches(1.5)\n",
    "width = prs.slide_width * 1/3\n",
    "height = Inches(4.5)\n",
    "insights_box = slide.shapes.add_textbox(left, top, width, height)\n",
    "insights_frame = insights_box.text_frame\n",
    "insights_p = insights_frame.add_paragraph()\n",
    "insights_p.text = \"Key Insights:\"\n",
    "insights_p.font.bold = True\n",
    "insights_p.font.size = Pt(24)\n",
    "insights_p.font.color.rgb = RGBColor(0, 128, 100)\n",
    "insights_p.alignment = PP_PARAGRAPH_ALIGNMENT.LEFT\n",
    "insights_frame.add_paragraph()\n",
    "\n",
    "\n",
    "bullet_p = insights_frame.add_paragraph()\n",
    "bullet_p.text = bullet_points\n",
    "bullet_p.font.size = Pt(12)\n",
    "bullet_p.font.color.rgb = RGBColor(255, 255, 255)\n",
    "bullet_p.line_spacing = 1.5\n",
    "\"\"\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's set a few quick variables for our slides. We want the company name, NotRealCorp, to be on the title slide, and the title of the presentation should 'Quartlerly financial planning metting, Q3, 2023'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "title_text = \"NotRealCorp\"\n",
    "subtitle_text = \"Quarterly financial planning meeting, Q3 2023\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And for the data slide, we have:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we have a template to create a Title Slide. The template below was created by uploading the image of a desirable title slide to GPT-V, and asking for the `python-pptx` code to create that template. The inputs to the template are the image_path, title_text, and subtitle_text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Run(id='run_taLrnOnlDhoywgQFFBOLPlg0', assistant_id='asst_3T362kLlTyAq0FUnkvjjQczO', cancelled_at=None, completed_at=None, created_at=1701827118, expires_at=1701827718, failed_at=None, file_ids=['file-piTokyHGllwGITzIpoG8dok3'], instructions='You are a data scientist assistant. When given data and a query, write the proper code and create the proper visualization', last_error=None, metadata={}, model='gpt-4-1106-preview', object='thread.run', required_action=None, started_at=None, status='queued', thread_id='thread_73TgtFoJMlEJvb13ngjTnAo3', tools=[ToolAssistantToolsCode(type='code_interpreter')])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submit_message(assistant.id,thread,f\"Use the included code template to create a PPTX slide that follows the template format, but uses the image, company name/title, and document name/subtitle included:\\\n",
    "{title_template}. IMPORTANT: Use the image file included in this message as the image_path image in this first slide, and use the Company Name {title_text} as the title_text variable, and \\\n",
    "  use the subtitle_text {subtitle_text} a the subtitle_text variable. \\\n",
    "    NEST, create a SECOND slide using the following code template: {data_vis_template} to create a PPTX slide that follows the template format, but uses the company name/title, and document name/subtitle included:\\\n",
    "{data_vis_template}. IMPORTANT: Use the line plot image, that is the second attached image in this message, that you created earlier in the thread as the data_vis_img image, and use the data visualization title that you created earlier for the variable title_text, and\\\n",
    "  the bullet points of insights you created earlier for the bullet_points variable. Output these TWO SLIDES as a .pptx file. Make sure the output is two slides, with each slide matching the respective template given in this message.\",\n",
    "              file_ids=[dalle_file.id, plot_file.id]\n",
    ")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Assistant still working on PPTX...\n",
      "Assistant still working on PPTX...\n",
      "Assistant still working on PPTX...\n",
      "Assistant still working on PPTX...\n",
      "Assistant still working on PPTX...\n",
      "Assistant still working on PPTX...\n",
      "Assistant still working on PPTX...\n",
      "Assistant still working on PPTX...\n",
      "Assistant still working on PPTX...\n",
      "Assistant still working on PPTX...\n",
      "Successfully retrieved pptx_id: file-oa0i63qPH4IaJXYj90aA6L4Q\n"
     ]
    }
   ],
   "source": [
    "#May take 1-3 mins\n",
    "while True:\n",
    "    try:\n",
    "        response = get_response(thread)\n",
    "        pptx_id = response.data[0].content[0].text.annotations[0].file_path.file_id\n",
    "        print(\"Successfully retrieved pptx_id:\", pptx_id)\n",
    "        break\n",
    "    except Exception as e:\n",
    "        print(\"Assistant still working on PPTX...\")\n",
    "        time.sleep(10)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "pptx_id = response.data[0].content[0].text.annotations[0].file_path.file_id\n",
    "ppt_file= client.files.content(pptx_id)\n",
    "file_obj = io.BytesIO(ppt_file.read())\n",
    "with open(\"data/created_slides.pptx\", \"wb\") as f:\n",
    "    f.write(file_obj.getbuffer())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we have a PPTX file saved with all of our created content!. <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at the screenshots of the .pptx we just created using JUST the assistants API and DALL·E-3. We don't have a `seed` parameter yet in the Assistants API, so the DALL·E-3 image and wordings will be slightly different from what you see when you run this notebook, due to the non-determinism of LLMs, but the outputs should be directionally the same."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The title slide:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Title Slide](../images/title_slide.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the data slide:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Data Slide](../images/data_vis_slide.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Woo! While these slides could use some formatting tweaks, we have made some great content using the Assistants API, GPT-4 and DALL·E-3. We were able to take a `.csv` file with financial data, and use our assisant to calculate profit by quarter across distribution channels, plot the results, identify insights and key takeaways from the visualization, and create a summarative title. And, given just a description of our company, NotRealCorp, we used DALL·E-3 to make an awesome title image. <br><br>\n",
    "While we are still a ways away from entirely automating this process without a human in the loop, hopefully this notebook can make the slide creation process a bit easier for you. More importantly, this notebook can ideally give you a glimpse into the potential of the assistants API! We're excited to see what you build."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Extensions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- When  DALL·E-3 is incorporated in the Assistants API, we will have the ability to request the generated title image within the thread. \n",
    "- GPT-4-Vision is not yet supported in the Assistants API, but could have been used to gather insights from the line plot image.\n",
    "- GPT-4-Vision was used to generate the `python-pptx` template included in this recipe, so a potential extension project could be demonstrating best practices around converting images to slide templates."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "openai",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}