diff --git a/changelog.md b/changelog.md new file mode 100644 index 00000000..1df49ee0 --- /dev/null +++ b/changelog.md @@ -0,0 +1,42 @@ +# Updates to Admin APIs, Real-time Sessions, and Fine-tuning Request Payloads + +1. **New admin API key endpoints**: You can now manage admin-level keys with “GET,” “POST,” “GET by ID,” and “DELETE” calls at `/organization/admin_api_keys`. This makes it easier to create, review, and delete high-permission keys without mixing them up with normal API keys. +2. **New realtime sessions endpoint**: There is now a “POST /realtime/sessions” method for ephemeral token creation. This lets you quickly generate short-lived client tokens to power more dynamic, real-time applications. +Example Payload: +```json +{ + "model": "gpt-4o-realtime-preview-2024-12-17", + "modalities": ["audio", "text"], + "instructions": "You are a helpful assistant.", + "voice": "alloy", + "input_audio_format": "pcm16", + "output_audio_format": "pcm16", + "input_audio_transcription": { + "model": "whisper-1" + }, + "turn_detection": null, + "tools": [], + "tool_choice": "auto", + "temperature": 0.8, + "max_response_output_tokens": "inf" +} +``` + +3. **Fine-tuning job creation changed**: the request now requires you to nest fields under “method” with “type” and “hyperparameters” inside. The [finetuning UI](https://platform.openai.com/finetune) remains unchanged +```json +{ + "training_file": "file-abc123", + "model": "gpt-4o-mini", + "method": { + "type": "supervised", + "supervised": { + "hyperparameters": { + "n_epochs": 2 + } + } + } +} +``` + +![Fine-tuning Dashboard](images/finetuning-dashboard.png) +Access fine-tuning UI at https://platform.openai.com/finetune \ No newline at end of file diff --git a/finetuning-guide.md b/finetuning-guide.md new file mode 100644 index 00000000..02449004 --- /dev/null +++ b/finetuning-guide.md @@ -0,0 +1,83 @@ +# OpenAI Fine-tuning Guide +## Overview +Fine-tuning allows you to customize OpenAI's models for specific use cases by training them on your own data([1](https://platform.openai.com/docs/guides/fine-tuning)). This is particularly useful for: +- Setting specific style, tone, or format +- Improving reliability for desired outputs +- Handling complex prompts and edge cases +- Teaching new skills or tasks +## Currently Available Models +- `gpt-4o-mini` (recommended for most users) +- `gpt-3.5-turbo` +- `babbage-002` +- `davinci-002` +Note: GPT-4 fine-tuning is currently in experimental access([1](https://platform.openai.com/docs/guides/fine-tuning)). +## Step-by-Step Guide +### 1. Prepare Your Dataset +Create a JSONL file with your training examples. Each example should be a conversation in this format: +```jsonl +{"messages": [ + {"role": "system", "content": "System message here"}, + {"role": "user", "content": "User message here"}, + {"role": "assistant", "content": "Assistant response here"} +]} +``` +Best practices: +- Include at least 10 examples +- Make examples diverse and representative +- Target specific cases where the base model isn't performing as desired +- Include ideal responses in the assistant messages +### 2. Validate and Upload Your Data +```python +from openai import OpenAI +client = OpenAI() +# Upload the training file +file = client.files.create( + file=open("training_data.jsonl", "rb"), + purpose="fine-tune" +) +``` +### 3. Create Fine-tuning Job +Based on the recent changelog, use this updated format: +```python +client.fine_tuning.jobs.create( + training_file="file-abc123", + model="gpt-3.5-turbo", + method={ + "type": "supervised", + "supervised": { + "hyperparameters": { + "n_epochs": 2 + } + } + } +) +``` +### 4. Configure Hyperparameters (Optional) +Key hyperparameters to consider: +- `n_epochs`: Increase by 1-2 if model isn't following training data enough +- Decrease epochs if model becomes less diverse than desired +- Adjust learning rate if model isn't converging +### 5. Monitor Training Progress +```python +# Get the status of your fine-tuning job +job = client.fine_tuning.jobs.retrieve("job-id") +print(job.status) +``` +## Cost Considerations +- Fine-tuning costs vary based on model and data size +- Training tokens and usage tokens have different pricing +- Consider testing with a smaller dataset first +## Best Practices +1. **Start Simple**: Begin with base model prompt engineering before fine-tuning +2. **Quality Data**: Ensure training data is high-quality and well-formatted +3. **Test Thoroughly**: Compare fine-tuned model against base model using test cases +4. **Iterate**: Monitor performance and adjust hyperparameters as needed +## Rate Limits +- Fine-tuned models share rate limits with their base models +- For example, if you use 50% of `gpt-3.5-turbo`'s TPM limit, your fine-tuned version will only have the remaining 50% available +## Evaluation +To evaluate your fine-tuned model: +1. Generate samples from both base and fine-tuned models +2. Compare responses side-by-side +3. Consider using OpenAI's evals framework for comprehensive testing +Remember that fine-tuning can be complementary to retrieval strategies - they're not mutually exclusive approaches to improving model performance. \ No newline at end of file diff --git a/images/finetuning-dashboard.png b/images/finetuning-dashboard.png new file mode 100644 index 00000000..50b71338 Binary files /dev/null and b/images/finetuning-dashboard.png differ