Exploring Limitations of Gemini Models

{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Limitations of : gemini-pro and  gemini-pro-vision","metadata":{}},{"cell_type":"code","source":"import google.generativeai as genai\nfrom kaggle_secrets import UserSecretsClient\nfrom pprint import pprint\nfrom IPython.display import Markdown\n\n\nfrom PIL import Image","metadata":{"execution":{"iopub.status.busy":"2023-12-14T06:09:38.294028Z","iopub.execute_input":"2023-12-14T06:09:38.294465Z","iopub.status.idle":"2023-12-14T06:09:38.299898Z","shell.execute_reply.started":"2023-12-14T06:09:38.294433Z","shell.execute_reply":"2023-12-14T06:09:38.298915Z"},"trusted":true},"execution_count":52,"outputs":[]},{"cell_type":"code","source":"user_secrets = UserSecretsClient()\napi_key = user_secrets.get_secret(\"google_api_key\")\ngenai.configure(api_key = api_key)","metadata":{"execution":{"iopub.status.busy":"2023-12-14T05:51:10.984680Z","iopub.execute_input":"2023-12-14T05:51:10.985062Z","iopub.status.idle":"2023-12-14T05:51:11.303970Z","shell.execute_reply.started":"2023-12-14T05:51:10.985033Z","shell.execute_reply":"2023-12-14T05:51:11.302649Z"},"trusted":true},"execution_count":16,"outputs":[]},{"cell_type":"markdown","source":"### Available Models","metadata":{}},{"cell_type":"code","source":"for models in genai.list_models():\n    print(\"Model Name -\",models.name.split('/')[1])\n    print(\"Model Description -\",models.description)\n    print(\"--------------\")\n    ","metadata":{"execution":{"iopub.status.busy":"2023-12-14T05:53:48.610783Z","iopub.execute_input":"2023-12-14T05:53:48.611537Z","iopub.status.idle":"2023-12-14T05:53:48.956770Z","shell.execute_reply.started":"2023-12-14T05:53:48.611485Z","shell.execute_reply":"2023-12-14T05:53:48.955591Z"},"trusted":true},"execution_count":26,"outputs":[{"name":"stdout","text":"Model Name - chat-bison-001\nModel Description - A legacy text-only model optimized for chat conversations\n--------------\nModel Name - text-bison-001\nModel Description - A legacy model that understands text and generates text as an output\n--------------\nModel Name - embedding-gecko-001\nModel Description - Obtain a distributed representation of a text.\n--------------\nModel Name - gemini-pro\nModel Description - The best model for scaling across a wide range of tasks\n--------------\nModel Name - gemini-pro-vision\nModel Description - The best image understanding model to handle a broad range of applications\n--------------\nModel Name - embedding-001\nModel Description - Obtain a distributed representation of a text.\n--------------\nModel Name - aqa\nModel Description - Model trained to return answers to questions that are grounded in provided sources, along with estimating answerable probability.\n--------------\n","output_type":"stream"}]},{"cell_type":"code","source":"# model = genai.GenerativeModel(model_name='gemini-pro')\n# response = model.generate_content('Who are you ?')\n# print(response.text)","metadata":{"execution":{"iopub.status.busy":"2023-12-14T06:16:54.670222Z","iopub.execute_input":"2023-12-14T06:16:54.670674Z","iopub.status.idle":"2023-12-14T06:16:54.675370Z","shell.execute_reply.started":"2023-12-14T06:16:54.670639Z","shell.execute_reply":"2023-12-14T06:16:54.674206Z"},"trusted":true},"execution_count":53,"outputs":[]},{"cell_type":"code","source":"def generate_resposne(question):\n    print(question)\n    resposne = model.generate_content(question)\n    return response.text","metadata":{"execution":{"iopub.status.busy":"2023-12-14T06:16:54.886365Z","iopub.execute_input":"2023-12-14T06:16:54.886781Z","iopub.status.idle":"2023-12-14T06:16:54.893237Z","shell.execute_reply.started":"2023-12-14T06:16:54.886750Z","shell.execute_reply":"2023-12-14T06:16:54.892018Z"},"trusted":true},"execution_count":54,"outputs":[]},{"cell_type":"markdown","source":"## Testing Gemini-Pro","metadata":{}},{"cell_type":"markdown","source":"#### Reasoning Questions","metadata":{}},{"cell_type":"code","source":"output= model.generate_content(\"Reena's mother has five daughters: Reena, Teena, Sheena and Sheela. Who is the fifth daughter?\")\nprint(output.text)","metadata":{"execution":{"iopub.status.busy":"2023-12-14T06:24:25.511775Z","iopub.execute_input":"2023-12-14T06:24:25.512172Z","iopub.status.idle":"2023-12-14T06:24:26.546739Z","shell.execute_reply.started":"2023-12-14T06:24:25.512142Z","shell.execute_reply":"2023-12-14T06:24:26.545664Z"},"trusted":true},"execution_count":65,"outputs":[{"name":"stdout","text":"Reena\n","output_type":"stream"}]},{"cell_type":"code","source":"output = model.generate_content('A man builds a house with all 4 sides facing south. A bear walks past the house. What color is the bear?')\nprint(output.text)","metadata":{"execution":{"iopub.status.busy":"2023-12-14T06:25:08.532982Z","iopub.execute_input":"2023-12-14T06:25:08.533396Z","iopub.status.idle":"2023-12-14T06:25:09.677860Z","shell.execute_reply.started":"2023-12-14T06:25:08.533363Z","shell.execute_reply":"2023-12-14T06:25:09.676582Z"},"trusted":true},"execution_count":66,"outputs":[{"name":"stdout","text":"White, because it is a North Pole bear and the house is at the South Pole.\n","output_type":"stream"}]},{"cell_type":"code","source":"output = model.generate_content(\"\"\"\nAmla, Bamla, Camla, Damla and Emala are good friends. Only two of them are always honest. The other three persons are sometimes honest and sometimes not. When they were asked about who is a liar. Here are the answers from them:\nAmla said: Bamla is not a lier.\nBamla said: Camla is a liar.\nCamla said: Damla is a liar.\nDamla said: Emala is a liar.\nEmala said: Bamla is a liar.\nAmla said: Emala is not a liar.\nEmala said: Camla is a liar.\nFind out which two persons are always honest?\"\"\")\ntry:\n    print(output.text)\nexcept Exception as e:\n    print(\"Caught Exception :\",e)\n    pprint(output.prompt_feedback)","metadata":{"execution":{"iopub.status.busy":"2023-12-14T06:28:44.325890Z","iopub.execute_input":"2023-12-14T06:28:44.326757Z","iopub.status.idle":"2023-12-14T06:28:45.627684Z","shell.execute_reply.started":"2023-12-14T06:28:44.326696Z","shell.execute_reply":"2023-12-14T06:28:45.626555Z"},"trusted":true},"execution_count":69,"outputs":[{"name":"stdout","text":"Caught Exception : The `response.parts` quick accessor only works for a single candidate, but none were returned. Check the `response.prompt_feedback` to see if the prompt was blocked.\nblock_reason: SAFETY\nsafety_ratings {\n  category: HARM_CATEGORY_SEXUALLY_EXPLICIT\n  probability: NEGLIGIBLE\n}\nsafety_ratings {\n  category: HARM_CATEGORY_HATE_SPEECH\n  probability: NEGLIGIBLE\n}\nsafety_ratings {\n  category: HARM_CATEGORY_HARASSMENT\n  probability: HIGH\n}\nsafety_ratings {\n  category: HARM_CATEGORY_DANGEROUS_CONTENT\n  probability: NEGLIGIBLE\n}\n\n","output_type":"stream"}]},{"cell_type":"markdown","source":"> As it was a logic reasoning question, Gemini classified the prompt as `HARM_CATEGORY_HARASSMENT`.<br>\n> **Intreseting**","metadata":{}},{"cell_type":"code","source":"output = model.generate_content('A brother and a sister were born in summer and in winter. The sister was not born in winter. Who was born in summer?')\nprint(output.text)","metadata":{"execution":{"iopub.status.busy":"2023-12-14T06:33:58.559619Z","iopub.execute_input":"2023-12-14T06:33:58.560025Z","iopub.status.idle":"2023-12-14T06:33:59.679069Z","shell.execute_reply.started":"2023-12-14T06:33:58.559995Z","shell.execute_reply":"2023-12-14T06:33:59.677803Z"},"trusted":true},"execution_count":70,"outputs":[{"name":"stdout","text":"The brother.\n","output_type":"stream"}]},{"cell_type":"markdown","source":"> As question says , sister was not in winter, the answer is sister was born in summer.","metadata":{}},{"cell_type":"markdown","source":"### Visual Based Puzzels","metadata":{}},{"cell_type":"code","source":"# output = !wget puzzle.jpg https://s3-ap-southeast-1.amazonaws.com/sawaal.com/qaimg/22089055_1429224250518008_2132246368445954029_n1507282378.jpg \nmodel_2 = genai.GenerativeModel(model_name='gemini-pro-vision')\nresponse = model_2.generate_content(Image.open('22089055_1429224250518008_2132246368445954029_n1507282378.jpg'))\nprint(response.text)","metadata":{"execution":{"iopub.status.busy":"2023-12-14T06:19:00.474766Z","iopub.execute_input":"2023-12-14T06:19:00.475195Z","iopub.status.idle":"2023-12-14T06:19:07.559305Z","shell.execute_reply.started":"2023-12-14T06:19:00.475162Z","shell.execute_reply":"2023-12-14T06:19:07.558006Z"},"trusted":true},"execution_count":60,"outputs":[{"name":"stdout","text":" The answer is 16.\n","output_type":"stream"}]},{"cell_type":"markdown","source":"##### Image is as follows:<br>\n<!-- ![Image](22089055_1429224250518008_2132246368445954029_n1507282378.jpg) -->","metadata":{}},{"cell_type":"code","source":"console = !curl -o pzzle.jpg \"https://s3-ap-southeast-1.amazonaws.com/sawaal.com/qaimg/16939430_1229474037159698_4440410928025505694_n1499418020.jpg\"","metadata":{"execution":{"iopub.status.busy":"2023-12-14T06:44:21.794735Z","iopub.execute_input":"2023-12-14T06:44:21.795165Z","iopub.status.idle":"2023-12-14T06:44:22.116682Z","shell.execute_reply.started":"2023-12-14T06:44:21.795133Z","shell.execute_reply":"2023-12-14T06:44:22.115520Z"},"trusted":true},"execution_count":74,"outputs":[]},{"cell_type":"code","source":"response = model_2.generate_content(Image.open('pzzle.jpg'))\nprint(response.text)","metadata":{"execution":{"iopub.status.busy":"2023-12-14T06:47:34.867754Z","iopub.execute_input":"2023-12-14T06:47:34.868391Z","iopub.status.idle":"2023-12-14T06:47:40.162185Z","shell.execute_reply.started":"2023-12-14T06:47:34.868359Z","shell.execute_reply":"2023-12-14T06:47:40.160928Z"},"trusted":true},"execution_count":83,"outputs":[{"name":"stdout","text":" The answer is 371.\n","output_type":"stream"}]},{"cell_type":"markdown","source":"> **Image is as follows**<br>\n>![Image](pzzle.jpg)\n>\n><br>\n> you dont find answer 371 in from image","metadata":{}},{"cell_type":"markdown","source":"## Thank you","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}