Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"name": "agora",
"source": "./",
"description": "Real-time communication with Agora SDKs — RTC, RTM, Conversational AI, and token generation",
"version": "1.1.0"
"version": "1.2.0"
}
]
}
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "agora",
"description": "Real-time communication with Agora SDKs — RTC, RTM, Conversational AI, and token generation",
"version": "1.1.0",
"version": "1.2.0",
"author": {
"name": "Agora"
},
Expand Down
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,32 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [1.2.0]

### Added

- RTC React Native reference (`references/rtc/react-native.md`) — `react-native-agora`: engine init, events, `RtcSurfaceView`, cleanup
- RTC Flutter reference (`references/rtc/flutter.md`) — `agora_rtc_engine`: engine init, `AgoraVideoView`, `RtcEngineEventHandler`, cleanup
- RTM iOS reference (`references/rtm/ios.md`) — `AgoraRtmClientKit` v2 (Swift): init, login, subscribe, publish, presence, delegate
- RTM Android reference (`references/rtm/android.md`) — `RtmClient` v2 (Kotlin): init, login, subscribe, publish, event listener
- ConvoAI iOS toolkit reference (`references/conversational-ai/agent-toolkit-ios.md`) — `ConversationalAIAPIImpl` Swift patterns
- ConvoAI Android toolkit reference (`references/conversational-ai/agent-toolkit-android.md`) — `ConversationalAIAPIImpl` Kotlin patterns
- Multi-product integration guide (`references/integration-patterns.md`) — RTC+RTM+ConvoAI init order, UID strategy, channel naming, token matrix, codec selection, cleanup sequence
- Testing guidance expanded — RTC React Native, Flutter, RTM Web/iOS/Android mocking patterns; token renewal section; table of contents

### Changed

- `rtc/react.md`: add codec interop note — `vp8` recommended; `vp9` hardware-limited on older iOS Safari; `h264` does not scale for multi-user
- `rtc/cross-platform-coordination.md`: corrected codec table — `vp8` is the safe default; `vp9` requires iPhone 15 Pro / M3+ hardware on iOS Safari; `h264` avoid for multi-user
- `rtc/README.md`: updated codec interop note to match corrected recommendation
- `rtm/ios.md`, `rtm/android.md`: added v2 to titles to prevent v1 API misuse
- `rtm/README.md`: added Platform Scope section clarifying client-side only, all v2, no server/desktop variant
- `rtm/web.md`: removed RTM v1 legacy section; constructor wrapped in try/catch; token-only login form
- `conversational-ai/README.md`: added SDK-vs-REST routing table; RTM channel name = RTC channel name gotcha; scoped auth section to direct REST implementors
- `conversational-ai/auth-flow.md`: scoped to REST API implementors; added SDK-skip callout at top
- `SKILL.md`: bumped version to 1.2.0; added Multi-Product Integration entry; expanded RTC platform list (React Native, Flutter) and RTM platform list (iOS, Android)
- `CLAUDE.md`, `README.md`: updated file structure trees and product lists to reflect all new files and platforms

## [1.1.0]

### Added
Expand Down
17 changes: 11 additions & 6 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,10 @@ skills/
└── references/
├── doc-fetching.md ← two-tier lookup procedure (agent-facing)
├── mcp-tools.md ← Agora MCP server install guide (user-facing)
├── rtc/
├── rtm/
├── conversational-ai/
├── integration-patterns.md ← multi-product coordination (RTC+RTM+ConvoAI)
├── rtc/ ← Web, React, Next.js, iOS, Android, React Native, Flutter
├── rtm/ ← Web, iOS, Android (all v2)
├── conversational-ai/ ← REST API, server SDKs, iOS/Android toolkits, React hooks
├── server/
├── cloud-recording/
├── server-gateway/
Expand Down Expand Up @@ -45,9 +46,13 @@ Before adding any inline content, ask: **will this still be correct in 6 months
|---|---|
| RTC initialization, track management, event registration | Inline — `references/rtc/` |
| RTM messaging and presence patterns | Inline — `references/rtm/` |
| Token generation (RTC, RTM, AccessToken2) | Inline — `references/server/` |
| ConvoAI gotchas, field-type rules, lifecycle | Inline — `references/conversational-ai/README.md` |
| ConvoAI quick-start code (Python, Go, Java) | Level 2 fetch |
| Token generation (RTC, RTM, AccessToken2) | Inline — `references/server/` (`server/` is token generation only — not ConvoAI SDKs) |
| ConvoAI gotchas, field-type rules, REST API lifecycle | Inline — `references/conversational-ai/README.md` |
| ConvoAI server SDK patterns (auth modes, state machine, session lifecycle, avatar+TTS) | Inline — `references/conversational-ai/server-sdks.md` |
| ConvoAI auth flow (three-token distinction, sequence diagram) | Inline — `references/conversational-ai/auth-flow.md` |
| ConvoAI Python SDK patterns (sync/async, deprecations, debug) | Inline — `references/conversational-ai/python-sdk.md` |
| ConvoAI Go SDK patterns (context.Context, functional options, status constants) | Inline — `references/conversational-ai/go-sdk.md` |
| ConvoAI quick-start code with vendor-specific imports (Python, Go, Java) | Level 2 fetch |
| ConvoAI full request/response schemas | Level 2 fetch |
| TTS / ASR / LLM vendor configs and model names | Level 2 fetch |
| Cloud Recording REST API field details | Level 2 fetch |
Expand Down
37 changes: 25 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,13 +66,14 @@ This repo contains markdown skill files that give AI coding assistants deep know

**Products covered:**

- **RTC (Video/Voice SDK)** — Web, React, Next.js, iOS (Swift), Android (Kotlin/Java)
- **RTM (Signaling)** — Web (JS/TS) messaging, presence, metadata, stream channels
- **Conversational AI** — REST API, agent config, Gemini Live + OpenAI Realtime MLLM, 6 recipe repos (agent-samples, agent-toolkit, agent-client-toolkit-react, agent-ui-kit, server-custom-llm, server-mcp)
- **RTC (Video/Voice SDK)** — Web, React, Next.js, iOS (Swift), Android (Kotlin/Java), React Native, Flutter
- **RTM (Signaling)** — Web (JS/TS), iOS (Swift), Android (Kotlin) — all v2; messaging, presence, metadata, stream channels
- **Conversational AI** — REST API, agent config, Gemini Live + OpenAI Realtime MLLM, iOS/Android toolkits, 6 recipe repos (agent-samples, agent-toolkit, agent-client-toolkit-react, agent-ui-kit, server-custom-llm, server-mcp)
- **Cloud Recording** — REST API acquire/start/query/stop lifecycle
- **Server Gateway** — Linux SDK (C++) for server-side RTC
- **Server-Side** — Token generation for Node.js, Python, Go
- **Testing Guidance** — ConvoAI and RTC testing patterns
- **Multi-Product Integration** — RTC + RTM + ConvoAI initialization order, UID strategy, codec selection, token matrix
- **Testing Guidance** — Mocking patterns for all platforms (Web, React, iOS, Android, React Native, Flutter, RTM)

## Design — 4-Layer Progressive Disclosure

Expand Down Expand Up @@ -104,30 +105,42 @@ ConvoAI files are aligned 1:1 with repos in [AgoraIO-Conversational-AI](https://
```
skills/
└── agora/ Skill root
├── SKILL.md Entry point, product index
├── SKILL.md Entry point, product index (v1.2.0)
├── intake/
│ └── SKILL.md Multi-product needs analysis router
└── references/
├── doc-fetching.md Two-tier lookup procedure (agent-facing)
├── mcp-tools.md MCP tool reference and graceful degradation
├── integration-patterns.md RTC+RTM+ConvoAI: init order, UID strategy, codec, tokens
├── rtc/ RTC (Video/Voice SDK)
│ ├── README.md Critical rules, encoder profiles, cross-platform notes
│ ├── web.md agora-rtc-sdk-ng: client, tracks, events, screen share
│ ├── react.md agora-rtc-react: hooks, custom patterns
│ ├── react.md agora-rtc-react: hooks, codec interop, custom patterns
│ ├── nextjs.md Next.js / SSR dynamic import patterns
│ ├── ios.md AgoraRtcEngineKit (Swift): setup, delegation
│ └── android.md RtcEngine (Kotlin/Java): setup, callbacks
├── rtm/ RTM (Signaling / Messaging)
│ ├── README.md Key concepts, platform links
│ └── web.md agora-rtm v2: messaging, presence, stream channels
│ ├── android.md RtcEngine (Kotlin/Java): setup, callbacks
│ ├── react-native.md react-native-agora: engine init, events, video views
│ ├── flutter.md agora_rtc_engine (Dart): engine init, AgoraVideoView
│ └── cross-platform-coordination.md UID strategy, codec interop, screen share
├── rtm/ RTM Signaling SDK v2
│ ├── README.md Key concepts, gotchas, platform links
│ ├── web.md agora-rtm v2: messaging, presence, stream channels
│ ├── ios.md AgoraRtmClientKit (Swift): init, login, subscribe, publish
│ └── android.md RtmClient (Kotlin): init, login, subscribe, publish
├── conversational-ai/ Conversational AI (Voice AI Agents)
│ ├── README.md Architecture, endpoints, auth, lifecycle, gotchas
│ ├── agent-samples.md Backend, React clients, profiles, MLLM, deployment
│ ├── agent-toolkit.md @agora/conversational-ai SDK: API, helpers, hooks
│ ├── agent-client-toolkit-react.md React hooks: provider, transcript, state
│ ├── agent-ui-kit.md @agora/agent-ui-kit React components
│ ├── agent-toolkit-ios.md iOS ConversationalAIAPIImpl Swift toolkit
│ ├── agent-toolkit-android.md Android ConversationalAIAPIImpl Kotlin toolkit
│ ├── server-custom-llm.md Custom LLM proxy: RAG, tools, memory
│ └── server-mcp.md MCP memory server: persistent per-user memory
│ ├── server-mcp.md MCP memory server: persistent per-user memory
│ ├── auth-flow.md Three-token flow for direct REST API implementors
│ ├── python-sdk.md agora-agent Python SDK patterns
│ ├── go-sdk.md agora-agent-server-sdk-go patterns
│ └── server-sdks.md TypeScript/Node.js server SDK patterns
├── cloud-recording/ Cloud Recording (REST API)
│ └── README.md acquire/start/query/stop lifecycle, storage config
├── server-gateway/ Server Gateway (Linux SDK)
Expand All @@ -137,7 +150,7 @@ skills/
│ ├── README.md Token types, when tokens are needed
│ └── tokens.md Token generation TOC + links to official docs
└── testing-guidance/ Testing Patterns
└── SKILL.md ConvoAI and RTC test setup, mocking patterns
└── SKILL.md Mocking patterns: Web, React, iOS, Android, RN, Flutter, RTM
```

## Maintaining and Extending
Expand Down
14 changes: 10 additions & 4 deletions skills/agora/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ name: agora
description: Write code using Agora SDKs (agora.io) for real-time communication. Covers RTC (video/voice calling, live streaming, screen sharing), RTM (signaling, messaging, presence), Conversational AI (voice AI agents), Cloud Recording, Server Gateway, and server-side token generation. Use when the user wants to build real-time audio/video applications, integrate Agora SDKs (Web JS/TS, React, iOS Swift, Android Kotlin/Java, Go, Python), manage channels, tracks, tokens, use RTM for messaging/signaling, record RTC sessions, or build Conversational AI with the agent-toolkit. Triggers on mentions of Agora, agora.io, RTC, RTM, video calling, voice calling, real-time communication, screen share, screen sharing, record session, record calls, Cloud Recording, Server Gateway, Linux media SDK, agora-rtc-sdk-ng, agora-rtc-react, agora-rtm, conversational AI with Agora, Agora token generation, Agora authentication, agora-agent-client-toolkit, agora-agent-client-toolkit-react, agora-agent-server-sdk, AgoraVoiceAI, AgoraClient, useConversationalAI, useTranscript, useAgentState, agent transcript, agent state hook.
metadata:
author: agora
version: '1.1.0'
version: '1.2.0'
---

# Agora (agora.io)

Skill version: 1.1.0
Skill version: 1.2.0

Build real-time communication applications using Agora SDKs across Web, iOS, Android, and server-side platforms.

Expand All @@ -28,13 +28,13 @@ Read the README for the product the user needs. Only load what is needed.

Real-time audio and video. Users join channels, publish local tracks, subscribe to remote tracks.

**[references/rtc/README.md](references/rtc/README.md)** — Platforms: Web, React, iOS, Android
**[references/rtc/README.md](references/rtc/README.md)** — Platforms: Web, React, Next.js, iOS, Android, React Native, Flutter. Windows/Electron/Unity exist but require Level 2 fetch — no inline reference files.

### RTM (Signaling / Messaging)

Text messaging, signaling, presence, and metadata. Independent from RTC — channel namespaces are separate.

**[references/rtm/README.md](references/rtm/README.md)** — Platforms: Web
**[references/rtm/README.md](references/rtm/README.md)** — Platforms: Web, iOS, Android (all v2). RTM is a client-side SDK — there is no server-side or desktop (Electron/Windows) RTM variant.

### Conversational AI (Voice AI Agents)

Expand All @@ -60,6 +60,12 @@ Self-hosted Linux SDK for server-side audio/video stream transmission. Use when

**[references/server-gateway/README.md](references/server-gateway/README.md)** — C++, Java, Go, Python

### Multi-Product Integration

Initialization order, UID strategy, channel naming, token matrix, and cleanup sequence when combining RTC + RTM + ConvoAI.

**[references/integration-patterns.md](references/integration-patterns.md)**

### Testing Guidance

Mocking patterns and testing requirements for Agora SDK integration code.
Expand Down
Loading
Loading