Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
220 changes: 161 additions & 59 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,16 @@
# Vectorize Connect SDK

TypeScript/JavaScript SDK for connecting different platforms such as Google Drive and Dropbox to the Vectorize platform.
TypeScript/JavaScript SDK for building multi-user connectors that integrate cloud storage platforms such as Google Drive with the Vectorize platform.

This is a lightweight client that provides functionality for OAuth authentication and Vectorize API integration. The SDK helps you create connectors to various platforms, let users select files, and manage those connections through the Vectorize platform.
## What is this?

The Vectorize Connect SDK enables you to:
- Build applications where multiple users can connect their cloud storage accounts
- Let users select specific files to be processed by Vectorize RAG pipelines
- Manage user authentication and file permissions
- Handle OAuth flows without building the authentication infrastructure yourself

This SDK is designed for developers who want to integrate multiple users' documents into their AI-powered applications using Vectorize's infrastructure.

## SDK Installation

Expand All @@ -21,76 +29,170 @@ yarn add @vectorize-io/vectorize-connect
pnpm add @vectorize-io/vectorize-connect
```

## Documentation

For detailed documentation, please refer to:

- [General Guide](./docs/general-guide.md) - Overview and common concepts
- [Google Drive Integration](./docs/google-drive/) - Google Drive specific integration
- [Dropbox Integration](./docs/dropbox/) - Dropbox specific integration
- [API Reference](./docs/API.md) - Complete API documentation
- [White-Label Integration](./docs/google-drive/white-label-guide.md) - White-label integration
- [Vectorize Integration](./docs/google-drive/vectorize-guide.md) - Vectorize integration
- [Setup Guide](./docs/setup.md) - Setup instructions

## SDK Example Usage

### For Google Drive Integration

For detailed information and examples on Google Drive integration, please refer to the [Google Drive Integration](./docs/google-drive/) guides.

### For Dropbox Integration

For detailed information and examples on Dropbox integration, please refer to the [Dropbox Integration](./docs/dropbox/) guides.
## Quick Start

### 1. Install the SDK
```bash
npm install @vectorize-io/vectorize-connect
```

## API Reference

For detailed API documentation, please refer to the [API Reference](./docs/API.md) which includes:
### 2. Set up your environment
```typescript
import { createGDriveConnector } from '@vectorize-io/vectorize-connect';

- OAuth functions for Google Drive and Dropbox
- File selection functions
- Connector management functions
- User management functions
- Token utilities
- Configuration types
const vectorizeConfig = {
organizationId: 'your-org-id',
authorization: 'your-api-key'
};
```

## Configuration
### 3. Create a connector and add users
```typescript
// Create a connector
const connector = await createGDriveConnector({
connectorName: 'My Google Drive Connector',
config: vectorizeConfig
});

// Add users via OAuth
const authUrl = await getGDriveAuthURL({
connectorId: connector.id,
redirectUri: 'https://your-app.com/callback',
config: vectorizeConfig
});
```

### `VectorizeAPIConfig`
## Documentation

| Property | Type | Description |
|----------|------|-------------|
| `organizationId` | `string` | Your Vectorize organization ID |
| `authorization` | `string` | Your Vectorize API key |
For comprehensive guides and examples:

- [Multi-User Connectors Overview](https://docs.vectorize.io/developer-guides/multi-user-connectors) - Understanding connector types
- [Vectorize-Managed OAuth Guide](https://docs.vectorize.io/developer-guides/multi-user-connectors/vectorize-managed-oauth) - Quick setup using Vectorize's OAuth
- [White-Label OAuth Guide](https://docs.vectorize.io/developer-guides/multi-user-connectors/white-labeled-oauth) - Using your own OAuth apps

### SDK-Specific Documentation

- [API Reference](https://github.com/vectorize-io/vectorize-connect-sdk/blob/main/docs/API.md) - Complete API documentation
- [General Guide](https://github.com/vectorize-io/vectorize-connect-sdk/blob/main/docs/general-guide.md) - SDK concepts and patterns
- [TypeScript Definitions](https://github.com/vectorize-io/vectorize-connect-sdk/blob/main/docs/types.md) - Type definitions

### Implementation Examples

- [Environment Setup](https://github.com/vectorize-io/vectorize-connect-sdk/tree/main/docs/environment-setup)
- [Creating Connectors](https://github.com/vectorize-io/vectorize-connect-sdk/tree/main/docs/creating-connectors)
- [Authentication Flows](https://github.com/vectorize-io/vectorize-connect-sdk/tree/main/docs/authentication)
- [User Management](https://github.com/vectorize-io/vectorize-connect-sdk/tree/main/docs/user-management)
- [Frontend Integration](https://github.com/vectorize-io/vectorize-connect-sdk/tree/main/docs/frontend-implementation)
- [Testing](https://github.com/vectorize-io/vectorize-connect-sdk/tree/main/docs/testing)

## Supported Platforms

### Google Drive
- [Google Drive Overview](https://docs.vectorize.io/integrations/source-connectors/google-drive)
- [Multi-User Setup (Vectorize)](https://docs.vectorize.io/integrations/source-connectors/google-drive/multi-user-vectorize)
- [Multi-User Setup (White-Label)](https://docs.vectorize.io/integrations/source-connectors/google-drive/multi-user-white-label)

### Dropbox
- [Dropbox Overview](https://docs.vectorize.io/integrations/source-connectors/dropbox)
- [Multi-User Setup (Vectorize)](https://docs.vectorize.io/integrations/source-connectors/dropbox/multi-user-vectorize)
- [Multi-User Setup (White-Label)](https://docs.vectorize.io/integrations/source-connectors/dropbox/multi-user-white-label)

### Notion
- [OAuth Setup](https://docs.vectorize.io/integrations/source-connectors/notion)

## Two Approaches: Vectorize-Managed vs White-Label

### Vectorize-Managed OAuth (Recommended for Getting Started)
- Uses Vectorize's pre-configured OAuth apps
- No OAuth app setup required
- Fastest way to get started
- Available on Starter plan and above

### White-Label OAuth (For Production Apps)
- Use your own OAuth applications
- Full control over branding and authentication flow
- Required for production applications with custom branding
- Available on Pro plan and above


## Core Features

### OAuth Authentication
- Handle complex OAuth flows for Google Drive, Dropbox, and Notion
- Support for both Vectorize-managed and white-label OAuth approaches
- Automatic token management and refresh

### File Selection
- Interactive file picker UI components
- Granular file and folder selection
- Support for file type filtering

### User Management
- Add multiple users to a single connector
- Update user file selections
- Remove users and their associated data

### Connector Management
- Create and configure connectors programmatically
- Monitor connector status
- Handle connector lifecycle

## Example: Building a Team Knowledge Base

```typescript
import {
createGDriveConnector,
getGDriveAuthURL,
selectGDriveFiles
} from '@vectorize-io/vectorize-connect';

// 1. Create a connector for your team
const connector = await createGDriveConnector({
connectorName: 'Team Knowledge Base',
config: {
organizationId: process.env.VECTORIZE_ORG_ID,
authorization: process.env.VECTORIZE_API_KEY
}
});

// 2. Each team member authorizes access
app.get('/connect-drive', async (req, res) => {
const authUrl = await getGDriveAuthURL({
connectorId: connector.id,
redirectUri: 'https://your-app.com/oauth/callback',
config: vectorizeConfig
});
res.redirect(authUrl);
});

// 3. Let users select which files to include
app.post('/select-files', async (req, res) => {
await selectGDriveFiles({
connectorId: connector.id,
selectedFiles: req.body.files,
config: vectorizeConfig
});
});
```

## Requirements

- This SDK is compatible with Node.js environments and modern browsers
- TypeScript 4.7+ for type definitions
- Next.js 14.0.0+ for server components (optional)

## Summary

The Vectorize Connect SDK provides:
- Node.js 16+ or modern browsers
- TypeScript 4.7+ (optional, for TypeScript projects)
- A Vectorize account with API credentials

- OAuth authentication for Google Drive and Dropbox
- File selection functionality
- Token management for platform APIs
- Vectorize API integration for connectors
- User management capabilities
## Resources

## Detailed Documentation
### Getting Help
- [Vectorize Documentation](https://docs.vectorize.io)
- [GitHub Issues](https://github.com/vectorize-io/vectorize-connect-sdk/issues)
- [Example Implementation](https://github.com/vectorize-io/test-vectorize-connect-sdk)

For more detailed documentation, please refer to the following guides:
### Related Documentation
- [RAG Pipelines Overview](https://docs.vectorize.io/rag-pipelines/understanding)
- [Source Connectors Guide](https://docs.vectorize.io/integrations/source-connectors)
- [Vectorize Platform Overview](https://docs.vectorize.io/concepts/vectorize-architecture)

- [API Reference](./docs/API.md)
- [Google Drive White-Label Guide](./docs/google-drive/white-label-guide.md)
- [Google Drive Vectorize Guide](./docs/google-drive/vectorize-guide.md)
- [Dropbox White-Label Guide](./docs/dropbox/white-label-guide.md)
- [Dropbox Vectorize Guide](./docs/dropbox/vectorize-guide.md)
- [TypeScript Definitions](./docs/types.md)
- [Setup Guide](./docs/setup.md)

## License

Expand Down
97 changes: 76 additions & 21 deletions docs/general-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Overview

The Vectorize Connect SDK provides a set of tools to integrate with various cloud storage platforms, enabling seamless connection between your application and Vectorize's platform. This guide covers the general concepts and usage patterns for the SDK.
The Vectorize Connect SDK enables you to build multi-user connectors that integrate cloud storage platforms with the Vectorize platform. This SDK allows multiple users to connect their accounts and select files to be processed by Vectorize's RAG pipelines.

## Installation

Expand All @@ -12,36 +12,48 @@ npm install @vectorize-io/vectorize-connect

## Authentication

All interactions with the Vectorize API require authentication using a Vectorize token. You'll need to provide this token in your API requests:
All interactions with the Vectorize API require authentication using your API credentials:

```typescript
const config = {
authorization: 'Bearer your-token', // Use VECTORIZE_API_KEY environment variable
organizationId: 'your-org-id'
const config: VectorizeAPIConfig = {
authorization: process.env.VECTORIZE_API_KEY, // Your API key (without "Bearer" prefix)
organizationId: process.env.VECTORIZE_ORGANIZATION_ID // Your organization ID
};
```

## Connection Types
**Important**: The SDK handles adding the "Bearer" prefix internally, so provide only your API key.

The SDK supports two main connection types:
## Connector Types

1. **White-Label Integration**: Your application handles the OAuth flow and user interface, with Vectorize providing the backend services.
The SDK supports two approaches for implementing multi-user connectors:

2. **Non-White-Label Integration**: Vectorize handles the OAuth flow and user interface, with your application integrating with the Vectorize platform.
### 1. Vectorize-Managed OAuth (Recommended for Getting Started)
- Uses Vectorize's pre-configured OAuth applications
- No need to set up your own OAuth credentials
- Quickest way to get started
- Available on Starter plan and above

### 2. White-Label OAuth (For Production Apps)
- Use your own OAuth applications
- Full control over branding and authentication flow
- Required for production applications with custom branding
- Available on Pro plan and above

## Common Configuration

### VectorizeAPIConfig

Most functions in the SDK require a `VectorizeAPIConfig` object:
All SDK functions require a `VectorizeAPIConfig` object:

```typescript
interface VectorizeAPIConfig {
authorization: string; // Bearer token for authentication - use VECTORIZE_API_KEY env var
organizationId: string; // Your Vectorize organization ID - use VECTORIZE_ORGANIZATION_ID env var
authorization: string; // Your Vectorize API key
organizationId: string; // Your Vectorize organization ID
}
```

Get these values from your Vectorize dashboard under Settings > Access Tokens.

## Error Handling

All SDK functions return Promises that may reject with errors. Always implement proper error handling:
Expand All @@ -55,16 +67,62 @@ try {
}
```

## Connector Management
## Basic Usage Flow

Connectors are the bridge between your application and data sources. The SDK provides functions to create and manage connectors:
### For Vectorize-Managed OAuth (Simple Approach)

With Vectorize-managed OAuth, the authentication flow is handled through the Vectorize platform:

```typescript
// Create a connector
const connectorId = await createConnector(config, connectorName);
import {
// Import the create function for your platform
// e.g., createVectorizeGDriveConnector, createVectorizeDropboxConnector, etc.
createVectorize[Platform]Connector,
getOneTimeConnectorToken
} from '@vectorize-io/vectorize-connect';

// 1. Create a connector
const connectorId = await createVectorize[Platform]Connector(
config,
'Team Knowledge Base'
);

// 2. Generate a one-time token for secure authentication
const tokenData = await getOneTimeConnectorToken(
config,
'user123',
connectorId
);

// 3. Redirect user to Vectorize's OAuth flow
// The user will be redirected to the appropriate platform authentication page

// 4. After user completes OAuth and file selection on Vectorize platform,
// they return to your application
```

// Manage users for a connector
await manageUser(config, connectorId, userId);
### For White-Label OAuth (Advanced)

For white-label implementations, you handle the OAuth flow yourself:

```typescript
import {
// Import the create function and OAuth classes for your platform
createWhiteLabel[Platform]Connector,
[Platform]OAuth,
[Platform]Selection
} from '@vectorize-io/vectorize-connect';

// 1. Create a white-label connector with your own OAuth credentials
const connectorId = await createWhiteLabel[Platform]Connector(
config,
'My Custom Connector',
process.env.PLATFORM_CLIENT_ID!,
process.env.PLATFORM_CLIENT_SECRET!
);

// 2. Use OAuth classes to handle authentication
// See the OAuth Classes section in the API documentation for platform-specific details
```

## Next Steps
Expand All @@ -89,6 +147,3 @@ For specific platform integrations, refer to the step-based documentation struct
- [Frontend Implementation](./frontend-implementation/white-label/README.md)
- [Testing](./testing/white-label/README.md)

### Legacy Platform-Specific Guides (Deprecated)
- [Google Drive Integration](./legacy-docs/google-drive/) - Use step-based guides instead
- [Dropbox Integration](./legacy-docs/dropbox/) - Use step-based guides instead
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@vectorize-io/vectorize-connect",
"version": "0.3.3",
"version": "0.3.6",
"description": "A simple package for Google Drive authorization and file selection",
"main": "dist/index.js",
"module": "dist/index.mjs",
Expand Down
Loading