Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ Stream chats with the Responses API, transcribe and translate audio with Whisper
- [Translate](#translate)
- [Transcribe](#transcribe)
- [Speech](#speech)
- [Real-Time](#real-time)
- [Usage](#usage)
- [Errors](#errors-1)
- [Development](#development)
Expand Down Expand Up @@ -1657,6 +1658,33 @@ File.binwrite('demo.mp3', response)
# => mp3 file that plays: "This is a speech test!"
```

### Realtime

The [Realtime API](https://platform.openai.com/docs/guides/realtime) allows you to create a live speech-to-speech session with an OpenAI model. It responds with a session object, plus a client_secret key which contains a usable ephemeral API token that can be used to [authenticate browser clients for a WebRTC connection](https://platform.openai.com/docs/guides/realtime#connect-with-webrtc).

```ruby
response = client.realtime.create(parameters: { model: "gpt-4o-realtime-preview-2024-12-17" })
puts "ephemeral key: #{response.dig('client_secret', 'value')}"
# => "ephemeral key: ek_abc123"
```

Then in the client-side Javascript application, make a POST request to the Real-Time API with the ephemeral key and the SDP offer.

```js
const OPENAI_REALTIME_URL = 'https://api.openai.com/v1/realtime/sessions'
const MODEL = 'gpt-4o-realtime-preview-2024-12-17'

const response = await fetch(`${OPENAI_REALTIME_URL}?model=${MODEL}`, {
method: 'POST',
headers: {
'Content-Type': 'application/sdp',
'Authorization': `Bearer ${ephemeralKey}`,
'OpenAI-Beta': 'realtime=v1'
},
body: offer.sdp
})
```

### Usage

The Usage API provides information about the cost of various OpenAI services within your organization.
Expand Down
1 change: 1 addition & 0 deletions lib/openai.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
require_relative "openai/assistants"
require_relative "openai/threads"
require_relative "openai/messages"
require_relative "openai/realtime"
require_relative "openai/runs"
require_relative "openai/run_steps"
require_relative "openai/vector_stores"
Expand Down
6 changes: 6 additions & 0 deletions lib/openai/client.rb
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# rubocop:disable Metrics/ClassLength
module OpenAI
class Client
include OpenAI::HTTP
Expand Down Expand Up @@ -92,6 +93,10 @@ def batches
@batches ||= OpenAI::Batches.new(client: self)
end

def realtime
@realtime ||= OpenAI::Realtime.new(client: self)
end

def moderations(parameters: {})
json_post(path: "/moderations", parameters: parameters)
end
Expand Down Expand Up @@ -132,3 +137,4 @@ def inspect
end
end
end
# rubocop:enable Metrics/ClassLength
19 changes: 19 additions & 0 deletions lib/openai/realtime.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
module OpenAI
class Realtime
def initialize(client:)
@client = client.beta(realtime: "v1")
end

# Create a new real-time session with OpenAI.
#
# This method sets up a new session for real-time voice interaction with an OpenAI model.
# It returns session details that can be used to establish a WebRTC connection.
#
# @param parameters [Hash] parameters for the session (see: https://platform.openai.com/docs/api-reference/realtime-sessions/create)
# @return [Hash] Session details including session ID, ICE servers, and other
# connection information
def create(parameters: {})
@client.json_post(path: "/realtime/sessions", parameters: parameters)
end
end
end
112 changes: 112 additions & 0 deletions spec/fixtures/cassettes/realtime_session_create.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

113 changes: 113 additions & 0 deletions spec/fixtures/cassettes/realtime_session_create_with_params.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

31 changes: 31 additions & 0 deletions spec/openai/client/realtime_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
RSpec.describe OpenAI::Realtime do
let(:client) { OpenAI::Client.new }
let(:realtime) { client.realtime }

describe "#create" do
it "uses the specified model" do
model = "gpt-4o-realtime-preview-2024-12-18"
VCR.use_cassette("realtime_session_create") do
response = realtime.create(parameters: { model: model })
expect(response["model"]).to eq(model)
end
end

context "with additional parameters" do
it "sends all parameters to the API" do
parameters = {
model: "gpt-4o-realtime-preview-2024-12-17",
voice: "alloy",
instructions: "You are a helpful assistant."
}

VCR.use_cassette("realtime_session_create_with_params") do
response = realtime.create(parameters: parameters)
expect(response["model"]).to eq(parameters[:model])
expect(response["voice"]).to eq(parameters[:voice])
expect(response["instructions"]).to eq(parameters[:instructions])
end
end
end
end
end