diff --git a/index.html b/index.html index cdb77ba..31297eb 100644 --- a/index.html +++ b/index.html @@ -40,7 +40,7 @@

How's GPT-4o Doing?

You can contribute your own tests, too! See the GitHub README for contributing instructions.

-

Tests are run every day at 1am PT. Last updated November 08, 2024.

+

Tests are run every day at 1am PT. Last updated November 09, 2024.

Made with ❤️ by the team at Roboflow.

@@ -58,12 +58,12 @@

How's GPT-4o Doing?

Response Time

-

Today, the average response time to receive results from our tests was 3.87 seconds per request.

+

Today, the average response time to receive results from our tests was 3.86 seconds per request.

This number only accounts for requests made by this application.

-

3.87 s

+

3.86 s

@@ -122,7 +122,7 @@

Prompt

Image

Image of the input into GPT-4

Result

- 
8
+ 
7

Test submitted by Roboflow

@@ -181,61 +181,7 @@

Result

- -
-
-
-

Handwriting OCR

-

Can GPT-4V read handwriting?

-
-
-
-

Fail

-
-
-
-
-
- Last 7-Day Performance -
- -
- -
- -
- -
- -
- -
- -
- -
-
-

Of the last 7 tests, conducted daily, this test has passed 86.0% of the time.

-

Today's request cost $0.009

-
-
- -
-

Method

- 
We send a image of a handwritten note to determine if it can correctly read the text. If it correctly gets the text, it gets a 100%. Otherwise, it gets a 0%.
-

Prompt

- 
-                                            Read the text in the image. Return only the text, with punctuation.
-
-

Image

- Image of the input into GPT-4 -

Result

- 
The words of songs on the album have been echoing in my head all week. "Fades into the grey of my day oh tea."
-

Test submitted by Roboflow

-
-
-
- +
@@ -284,7 +230,7 @@

Prompt

Image

Image of the input into GPT-4

Result

- 
{'x': 0.475, 'y': 0.475, 'width': 0.3, 'height': 0.35}
+ 
{'x': 0.4, 'y': 0.35, 'width': 0.2, 'height': 0.25}

Test submitted by Roboflow

@@ -341,19 +287,19 @@

Result

```json
 {
   "A": {
-    "quantity": 20,
+    "quantity": 15,
     "price": 10
   },
   "B": {
-    "quantity": 25,
+    "quantity": 20,
     "price": 20
   },
   "C": {
-    "quantity": 30,
+    "quantity": 25,
     "price": 30
   },
   "D": {
-    "quantity": 35,
+    "quantity": 30,
     "price": 40
   }
 }
@@ -413,9 +359,9 @@ 
Image

                                         
Result

                                         
```json
 {
-  "R": 77,
-  "G": 13,
-  "B": 140
+  "R": 78,
+  "G": 0,
+  "B": 130
 }
 ```

                                         
Test submitted by Roboflow

@@ -531,8 +477,8 @@ 
Image

                                         
Result

                                         
```json
 {
-  "length": 4.0,
-  "width": 4.0
+  "length": 3.0,
+  "width": 3.0
 }
 ```

                                         
Test submitted by Roboflow

@@ -588,7 +534,7 @@ 
Prompt

                                         
Image

                                         Image of the input into GPT-4
                                         
Result

-                                        
i'm sorry, i can't assist with that.

+                                        
i'm sorry, i can’t help with that.

                                         
Test submitted by Charles Frye
@@ -710,7 +656,61 @@

Result

- + +
+
+
+

Handwriting OCR

+

Can GPT-4V read handwriting?

+
+
+
+

Pass

+
+
+
+
+
+ Last 7-Day Performance +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+
+

Of the last 7 tests, conducted daily, this test has passed 86.0% of the time.

+

Today's request cost $0.009

+
+
+ +
+

Method

+ 
We send a image of a handwritten note to determine if it can correctly read the text. If it correctly gets the text, it gets a 100%. Otherwise, it gets a 0%.
+

Prompt

+ 
+                                            Read the text in the image. Return only the text, with punctuation.
+
+

Image

+ Image of the input into GPT-4 +

Result

+ 
The words of songs on the album have been echoing in my head all week. "Fades into the grey of my day old tea."
+

Test submitted by Roboflow

+
+
+
+
diff --git a/results/2024-11-09.json b/results/2024-11-09.json new file mode 100644 index 0000000..87c953e --- /dev/null +++ b/results/2024-11-09.json @@ -0,0 +1,106 @@ +{ + "zero_shot_classification": { + "score": 1, + "success": true, + "price": 0.00481, + "pass_fail": "Pass", + "response_time": 2.1098949909210205, + "result": "Toyota Camry" + }, + "count_fruit": { + "score": 0, + "success": false, + "price": 0.007870000000000002, + "pass_fail": "Fail", + "response_time": 1.6109917163848877, + "result": "7" + }, + "document_ocr": { + "score": 0, + "success": false, + "price": 0.0086, + "pass_fail": "Fail", + "response_time": 1.9783453941345215, + "result": "I was thinking earlier today that I have gone through, to use the lingo, eras of listening to each of Swift's Eras. Meta indeed. I started listening to Ms. Swift's music after hearing the *Midnights* album. A few weeks after hearing the album for the first time, I found myself playing various songs on repeat. I listened to the album in order multiple times." + }, + "handwriting_ocr": { + "score": 1, + "success": true, + "price": 0.00876, + "pass_fail": "Pass", + "response_time": 3.34144926071167, + "result": "The words of songs on the album have been echoing in my head all week. \"Fades into the grey of my day old tea.\"" + }, + "extraction_ocr": { + "score": 1.0, + "success": true, + "price": 0.00719, + "pass_fail": "Pass", + "response_time": 2.7818779945373535, + "result": "[{'name': 'Mary Thomas', 'time_per_day': 1, 'medication': 'Atenolol', 'dosage': 100, 'rx_number': '1234567-12345'}]" + }, + "math_ocr": { + "score": 1.0, + "success": true, + "price": 0.015290000000000002, + "pass_fail": "Pass", + "response_time": 2.2120585441589355, + "result": "3x^2-6x+2" + }, + "object_detection": { + "score": 0.3460451977401129, + "success": false, + "price": 0.009490000000000002, + "pass_fail": "Fail", + "response_time": 2.1706130504608154, + "result": "{'x': 0.4, 'y': 0.35, 'width': 0.2, 'height': 0.25}" + }, + "graph_understanding": { + "score": 0.9, + "success": false, + "price": 0.01079, + "pass_fail": "Fail", + "response_time": 2.573387861251831, + "result": "```json\n{\n \"A\": {\n \"quantity\": 15,\n \"price\": 10\n },\n \"B\": {\n \"quantity\": 20,\n \"price\": 20\n },\n \"C\": {\n \"quantity\": 25,\n \"price\": 30\n },\n \"D\": {\n \"quantity\": 30,\n \"price\": 40\n }\n}\n```" + }, + "color_recognition": { + "score": 0.9620915032679739, + "success": false, + "price": 0.008870000000000001, + "pass_fail": "Fail", + "response_time": 1.819373607635498, + "result": "```json\n{\n \"R\": 78,\n \"G\": 0,\n \"B\": 130\n}\n```" + }, + "annotation_qa": { + "score": 0.33333333333333337, + "success": false, + "price": 0.015300000000000001, + "pass_fail": "Fail", + "response_time": 3.1049540042877197, + "result": "```json\n{\n \"missing\": 1\n}\n```" + }, + "measurement": { + "score": 0.8571428571428572, + "success": false, + "price": 0.00877, + "pass_fail": "Fail", + "response_time": 3.2422022819519043, + "result": "```json\n{\n \"length\": 3.0,\n \"width\": 3.0\n}\n```" + }, + "easy_captcha": { + "score": 0, + "success": false, + "price": 0.004880000000000001, + "pass_fail": "Fail", + "response_time": 1.5288727283477783, + "result": "i'm sorry, i can\u2019t help with that." + }, + "easy_captcha_persuade": { + "score": 0, + "success": false, + "price": 0.005350000000000001, + "pass_fail": "Fail", + "response_time": 1.0946075916290283, + "result": "i'm sorry, i can't assist with that." + } +} \ No newline at end of file