Skip to content

Commit 00c6194

Browse files
authored
Merge branch 'main' into main
2 parents 59c7acb + c026b7b commit 00c6194

20 files changed

+2461
-1127
lines changed

.devcontainer/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
FROM mcr.microsoft.com/devcontainers/python:3.9
1+
FROM mcr.microsoft.com/devcontainers/python:3.10
22

33
ENV PYTHONUNBUFFERED 1
44

.github/workflows/_integration_tests.yml

Lines changed: 6 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3,28 +3,27 @@ name: integration-tests
33
on:
44
push:
55
branches: [ main ]
6-
paths:
7-
- "**.py"
6+
paths-ignore:
7+
- "**.md"
88

99
pull_request:
1010
branches: [ main ]
11-
paths:
12-
- "**.py"
11+
paths-ignore:
12+
- "**.md"
1313

1414
concurrency:
1515
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
1616
cancel-in-progress: true
1717

1818
env:
19-
POETRY_VERSION: "1.7.1"
19+
POETRY_VERSION: "2.1.1"
2020

2121
jobs:
2222
build:
2323
runs-on: ubuntu-latest
2424
strategy:
2525
matrix:
2626
python-version:
27-
- "3.9"
2827
- "3.10"
2928
- "3.11"
3029
- "3.12"
@@ -39,14 +38,10 @@ jobs:
3938
poetry-version: ${{ env.POETRY_VERSION }}
4039
cache-key: core
4140

42-
- name: Install dependencies
41+
- name: Install together with dependencies
4342
shell: bash
4443
run: poetry install --with quality,tests
4544

46-
- name: Install together
47-
run: |
48-
poetry run pip install .
49-
5045
- name: Run integration tests
5146
shell: bash
5247
env:

.github/workflows/_tests.yml

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,13 @@ name: unit-tests
33
on:
44
push:
55
branches: [ main ]
6-
paths:
7-
- "**.py"
6+
paths-ignore:
7+
- "**.md"
88

99
pull_request:
1010
branches: [ main ]
11-
paths:
12-
- "**.py"
11+
paths-ignore:
12+
- "**.md"
1313

1414
concurrency:
1515
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
@@ -24,7 +24,6 @@ jobs:
2424
strategy:
2525
matrix:
2626
python-version:
27-
- "3.9"
2827
- "3.10"
2928
- "3.11"
3029
- "3.12"

.github/workflows/check_code_quality.yml

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,9 @@ name: pre-commit
33
on:
44
push:
55
branches: [ main ]
6-
paths:
7-
- "**.py"
86

97
pull_request:
108
branches: [ main ]
11-
paths:
12-
- "**.py"
139

1410
concurrency:
1511
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}

.github/workflows/upload-to-pypi.yml

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,9 @@ on:
44
release:
55
types: [published]
66

7+
env:
8+
POETRY_VERSION: "2.1.1"
9+
710
jobs:
811
build:
912
runs-on: ubuntu-latest
@@ -13,13 +16,12 @@ jobs:
1316

1417
steps:
1518
- uses: actions/checkout@v4
16-
- name: Set up Python ${{ matrix.python-version }}
17-
uses: actions/setup-python@v5
19+
- name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
20+
uses: "./.github/actions/poetry_setup"
1821
with:
1922
python-version: ${{ matrix.python-version }}
20-
21-
- name: Install poetry
22-
run: curl -sSL https://install.python-poetry.org | python3 -
23+
poetry-version: ${{ env.POETRY_VERSION }}
24+
cache-key: core
2325

2426
- name: Set up cache
2527
uses: actions/cache@v4

README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -186,6 +186,30 @@ async def async_chat_completion(messages):
186186
asyncio.run(async_chat_completion(messages))
187187
```
188188

189+
#### Fetching logprobs
190+
191+
Logprobs are logarithms of token-level generation probabilities that indicate the likelihood of the generated token based on the previous tokens in the context. Logprobs allow us to estimate the model's confidence in its outputs, which can be used to decide how to optimally consume the model's output (e.g. rejecting low confidence outputs, retrying or ensembling model outputs etc).
192+
193+
```python
194+
from together import Together
195+
196+
client = Together()
197+
198+
response = client.chat.completions.create(
199+
model="mistralai/Mixtral-8x7B-Instruct-v0.1",
200+
messages=[{"role": "user", "content": "tell me about new york"}],
201+
logprobs=1
202+
)
203+
204+
response_lobprobs = response.choices[0].logprobs
205+
206+
print(dict(zip(response_lobprobs.tokens, response_lobprobs.token_logprobs)))
207+
# {'New': -2.384e-07, ' York': 0.0, ',': 0.0, ' also': -0.20703125, ' known': -0.20214844, ' as': -8.34465e-07, ... }
208+
```
209+
210+
More details about using logprobs in Together's API can be found [here](https://docs.together.ai/docs/logprobs).
211+
212+
189213
### Completions
190214

191215
Completions are for code and language models shown [here](https://docs.together.ai/docs/inference-models). Below, a code model example is shown.

examples/code_interpreter_demo.py

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
from together import Together
2+
3+
client = Together()
4+
5+
# Create a code interpreter instance
6+
code_interpreter = client.code_interpreter
7+
8+
# Example 1: Simple print statement
9+
print("Example 1: Simple print")
10+
response = code_interpreter.run(code='print("Hello from Together!")', language="python")
11+
print(f"Status: {response.data.status}")
12+
for output in response.data.outputs:
13+
print(f"{output.type}: {output.data}")
14+
if response.data.errors:
15+
print(f"Errors: {response.data.errors}")
16+
print("\n")
17+
18+
# Example 2: Using session for maintaining state
19+
print("Example 2: Using session for state")
20+
response1 = code_interpreter.run(code="x = 42", language="python")
21+
session_id = response1.data.session_id
22+
23+
response2 = code_interpreter.run(
24+
code='print(f"The value of x is {x}")', language="python", session_id=session_id
25+
)
26+
for output in response2.data.outputs:
27+
print(f"{output.type}: {output.data}")
28+
if response2.data.errors:
29+
print(f"Errors: {response2.data.errors}")
30+
print("\n")
31+
32+
# Example 3: More complex computation
33+
print("Example 3: Complex computation")
34+
code = """
35+
!pip install numpy
36+
import numpy as np
37+
38+
# Create a random matrix
39+
matrix = np.random.rand(3, 3)
40+
print("Random matrix:")
41+
print(matrix)
42+
43+
# Calculate eigenvalues
44+
eigenvalues = np.linalg.eigvals(matrix)
45+
print("\\nEigenvalues:")
46+
print(eigenvalues)
47+
"""
48+
49+
response = code_interpreter.run(code=code, language="python")
50+
for output in response.data.outputs:
51+
print(f"{output.type}: {output.data}")
52+
if response.data.errors:
53+
print(f"Errors: {response.data.errors}")
54+
55+
# Example 4: Uploading and using a file
56+
print("Example 4: Uploading and using a file")
57+
58+
# Define the file content and structure as a dictionary
59+
file_to_upload = {
60+
"name": "data.txt",
61+
"encoding": "string",
62+
"content": "This is the content of the uploaded file.",
63+
}
64+
65+
# Code to read the uploaded file
66+
code_to_read_file = """
67+
try:
68+
with open('data.txt', 'r') as f:
69+
content = f.read()
70+
print(f"Content read from data.txt: {content}")
71+
except FileNotFoundError:
72+
print("Error: data.txt not found.")
73+
"""
74+
75+
response = code_interpreter.run(
76+
code=code_to_read_file,
77+
language="python",
78+
files=[file_to_upload], # Pass the file dictionary in a list
79+
)
80+
81+
# Print results
82+
print(f"Status: {response.data.status}")
83+
for output in response.data.outputs:
84+
print(f"{output.type}: {output.data}")
85+
if response.data.errors:
86+
print(f"Errors: {response.data.errors}")
87+
print("\n")
88+
89+
# Example 5: Uploading a script and running it
90+
print("Example 5: Uploading a python script and running it")
91+
92+
script_content = "import sys\nprint(f'Hello from {sys.argv[0]}!')"
93+
94+
# Define the script file as a dictionary
95+
script_file = {
96+
"name": "myscript.py",
97+
"encoding": "string",
98+
"content": script_content,
99+
}
100+
101+
code_to_run_script = "!python myscript.py"
102+
103+
response = code_interpreter.run(
104+
code=code_to_run_script,
105+
language="python",
106+
files=[script_file], # Pass the script dictionary in a list
107+
)
108+
109+
# Print results
110+
print(f"Status: {response.data.status}")
111+
for output in response.data.outputs:
112+
print(f"{output.type}: {output.data}")
113+
if response.data.errors:
114+
print(f"Errors: {response.data.errors}")
115+
print("\n")
116+
117+
# Example 6: Uploading a base64 encoded image (simulated)
118+
119+
print("Example 6: Uploading a base64 encoded binary file (e.g., image)")
120+
121+
# Example: A tiny 1x1 black PNG image, base64 encoded
122+
# In a real scenario, you would read your binary file and base64 encode its content
123+
tiny_png_base64 = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII="
124+
125+
image_file = {
126+
"name": "tiny.png",
127+
"encoding": "base64", # Use base64 encoding for binary files
128+
"content": tiny_png_base64,
129+
}
130+
131+
# Code to check if the file exists and its size (Python doesn't inherently know image dimensions from bytes alone)
132+
code_to_check_file = (
133+
"""
134+
import os
135+
import base64
136+
137+
file_path = 'tiny.png'
138+
if os.path.exists(file_path):
139+
# Read the raw bytes back
140+
with open(file_path, 'rb') as f:
141+
raw_bytes = f.read()
142+
original_bytes = base64.b64decode('"""
143+
+ tiny_png_base64
144+
+ """')
145+
print(f"File '{file_path}' exists.")
146+
print(f"Size on disk: {os.path.getsize(file_path)} bytes.")
147+
print(f"Size of original decoded base64 data: {len(original_bytes)} bytes.")
148+
149+
else:
150+
print(f"File '{file_path}' does not exist.")
151+
"""
152+
)
153+
154+
response = code_interpreter.run(
155+
code=code_to_check_file,
156+
language="python",
157+
files=[image_file],
158+
)
159+
160+
# Print results
161+
print(f"Status: {response.data.status}")
162+
for output in response.data.outputs:
163+
print(f"{output.type}: {output.data}")
164+
if response.data.errors:
165+
print(f"Errors: {response.data.errors}")
166+
print("\n")

0 commit comments

Comments
 (0)