MCP Server Catalog Improvements

.env.example

@@ -395,6 +395,10 @@ MCPGATEWAY_CATALOG_AUTO_HEALTH_CHECK=true
 # Default: 3600 (1 hour)
 MCPGATEWAY_CATALOG_CACHE_TTL=3600
 
+# Number of catalog servers to display per page
+# Default: 100
+MCPGATEWAY_CATALOG_PAGE_SIZE=100
+
 #####################################
 # Header Passthrough Configuration
 #####################################

README.md

@@ -461,7 +461,7 @@ When using a MCP Client such as Claude with stdio:
 
 ## Quick Start - Containers
 
-Use the official OCI image from GHCR with **Docker** *or* **Podman**. 
+Use the official OCI image from GHCR with **Docker** *or* **Podman**.
 Please note: Currently, arm64 is not supported. If you are e.g. running on MacOS, install via PyPi.
 
 ---
@@ -1307,6 +1307,15 @@ ContextForge implements **OAuth 2.0 Dynamic Client Registration (RFC 7591)** and
 | `MCPGATEWAY_CATALOG_FILE`           | Path to catalog configuration file               | `mcp-catalog.yml`  | string  |
 | `MCPGATEWAY_CATALOG_AUTO_HEALTH_CHECK` | Automatically health check catalog servers    | `true`             | bool    |
 | `MCPGATEWAY_CATALOG_CACHE_TTL`      | Catalog cache TTL in seconds                     | `3600`             | int > 0 |
+| `MCPGATEWAY_CATALOG_PAGE_SIZE`      | Number of catalog servers per page               | `12`               | int > 0 |
+
+**Key Features:**
+- 🔄 Refresh Button - Manually refresh catalog without page reload
+- 🔍 Debounced Search - Optimized search with 300ms debounce
+- 📝 Custom Server Names - Specify custom names when registering
+- 🔌 Transport Detection - Auto-detect SSE, WebSocket, or HTTP transports
+- 🔐 OAuth Support - Register OAuth servers and configure later
+- ⚡ Better Error Messages - User-friendly errors for common issues
 
 **Documentation:**
 - [MCP Server Catalog Guide](https://ibm.github.io/mcp-context-forge/manage/catalog/) - Complete catalog setup and configuration

docs/docs/manage/catalog.md

@@ -34,6 +34,9 @@ MCPGATEWAY_CATALOG_AUTO_HEALTH_CHECK=true
 
 # Catalog cache TTL in seconds (default: 3600)
 MCPGATEWAY_CATALOG_CACHE_TTL=3600
+
+# Number of catalog servers to display per page (default: 12)
+MCPGATEWAY_CATALOG_PAGE_SIZE=12
 ```
 
 ---
@@ -94,6 +97,7 @@ catalog_servers:
     name: "Production Time Server"
     category: "Utilities"
     url: "https://time.api.example.com/sse"
+    transport: "SSE"  # Optional: Explicitly specify transport type
     auth_type: "OAuth2.1"
     provider: "Internal Platform"
     description: "Production time server with geo-replication"
@@ -107,6 +111,21 @@ catalog_servers:
     logo_url: "https://static.example.com/time-server-logo.png"
     documentation_url: "https://docs.example.com/time-server"
 
+  - id: "websocket-server"
+    name: "WebSocket MCP Server"
+    category: "Development Tools"
+    url: "wss://api.example.com/mcp"
+    transport: "WEBSOCKET"  # Specify WebSocket transport
+    auth_type: "API Key"
+    provider: "Internal Platform"
+    description: "Real-time MCP server using WebSocket protocol"
+    requires_api_key: true
+    secure: true
+    tags:
+      - "production"
+      - "websocket"
+      - "real-time"
+
   - id: "database-server"
     name: "Database Server"
     category: "Database"
@@ -177,6 +196,7 @@ Based on the `CatalogServer` schema (schemas.py:5371-5387):
 | `requires_api_key` | boolean | No | Whether API key is required (default: `false`) |
 | `secure` | boolean | No | Whether additional security is required (default: `false`) |
 | `tags` | array | No | Tags for categorization (default: `[]`) |
+| `transport` | string | No | Transport type: `SSE`, `STREAMABLEHTTP`, or `WEBSOCKET` (auto-detected if not specified) |
 | `logo_url` | string | No | URL to server logo/icon |
 | `documentation_url` | string | No | URL to server documentation |
 | `is_registered` | boolean | No | Whether server is already registered (set by system) |
@@ -448,6 +468,57 @@ auth_types:
 
 3. For API Key servers, provide the API key during registration
 
+### Transport Type Issues
+
+**Symptoms:** WebSocket servers fail to connect after registration
+
+**Solutions:**
+
+1. Explicitly specify the `transport` field in your catalog YAML:
+   ```yaml
+   catalog_servers:
+     - id: "websocket-server"
+       url: "wss://api.example.com/mcp"
+       transport: "WEBSOCKET"  # Explicitly set transport
+   ```
+
+2. Verify URL scheme matches transport type:
+   - WebSocket: `ws://` or `wss://`
+   - SSE: `http://` or `https://` with `/sse` path
+   - HTTP: `http://` or `https://` with `/mcp` path
+
+---
+
+## Recent Improvements (v0.7.0)
+
+### Enhanced UI Features
+
+The catalog UI now includes several UX improvements:
+
+- **🔄 Refresh Button**: Manually refresh the catalog without page reload
+- **🔍 Debounced Search**: 300ms debounce on search input for better performance
+- **📝 Custom Server Names**: Ability to specify custom names when registering servers
+- **📄 Pagination with Filters**: Filter parameters preserved when navigating pages
+- **⚡ Better Error Messages**: User-friendly error messages for common issues (connection, auth, SSL, etc.)
+- **🔐 OAuth Support**: OAuth servers can be registered without credentials and configured later
+
+### Transport Type Detection
+
+The catalog now supports:
+
+- **Explicit Transport**: Specify `transport` field in catalog YAML (`SSE`, `WEBSOCKET`, `STREAMABLEHTTP`)
+- **Auto-Detection**: Automatically detects transport from URL if not specified
+  - `ws://` or `wss://` → `WEBSOCKET`
+  - URLs ending in `/sse` → `SSE`
+  - URLs with `/mcp` path → `STREAMABLEHTTP`
+  - Default fallback → `SSE`
+
+### Authentication Improvements
+
+- **Custom Auth Headers**: Properly mapped as list of header key-value pairs
+- **OAuth Registration**: OAuth servers can be registered in "disabled" state until OAuth flow is completed
+- **API Key Modal**: Enhanced modal with custom name field and proper authorization headers
+
 ---
 
 ## See Also

mcpgateway/admin.py

@@ -9801,34 +9801,45 @@ async def catalog_partial(
     root_path = request.scope.get("root_path", "")
 
     # Calculate pagination
-    page_size = 100
+    page_size = settings.mcpgateway_catalog_page_size
     offset = (page - 1) * page_size
 
     catalog_request = CatalogListRequest(category=category, auth_type=auth_type, search=search, show_available_only=False, limit=page_size, offset=offset)
 
     response = await catalog_service.get_catalog_servers(catalog_request, db)
 
+    # Get ALL servers (no filters, no pagination) for counting statistics
+    all_servers_request = CatalogListRequest(show_available_only=False, limit=1000, offset=0)
+    all_servers_response = await catalog_service.get_catalog_servers(all_servers_request, db)
+
+    # Pass filter parameters to template for pagination links
+    filter_params = {
+        "category": category,
+        "auth_type": auth_type,
+        "search": search,
+    }
+
     # Calculate statistics and pagination info
     total_servers = response.total
     registered_count = sum(1 for s in response.servers if s.is_registered)
     total_pages = (total_servers + page_size - 1) // page_size  # Ceiling division
 
-    # Count servers by category, auth type, and provider
+    # Count ALL servers by category, auth type, and provider (not just current page)
     servers_by_category = {}
     servers_by_auth_type = {}
     servers_by_provider = {}
 
-    for server in response.servers:
+    for server in all_servers_response.servers:
         servers_by_category[server.category] = servers_by_category.get(server.category, 0) + 1
         servers_by_auth_type[server.auth_type] = servers_by_auth_type.get(server.auth_type, 0) + 1
         servers_by_provider[server.provider] = servers_by_provider.get(server.provider, 0) + 1
 
     stats = {
-        "total_servers": total_servers,
+        "total_servers": all_servers_response.total,  # Use total from all servers
         "registered_servers": registered_count,
-        "categories": response.categories,
-        "auth_types": response.auth_types,
-        "providers": response.providers,
+        "categories": all_servers_response.categories,
+        "auth_types": all_servers_response.auth_types,
+        "providers": all_servers_response.providers,
         "servers_by_category": servers_by_category,
         "servers_by_auth_type": servers_by_auth_type,
         "servers_by_provider": servers_by_provider,
@@ -9842,6 +9853,7 @@ async def catalog_partial(
         "page": page,
         "total_pages": total_pages,
         "page_size": page_size,
+        "filter_params": filter_params,
     }
 
     return request.app.state.templates.TemplateResponse("mcp_registry_partial.html", context)

mcpgateway/config.py

@@ -311,6 +311,7 @@ class Settings(BaseSettings):
     mcpgateway_catalog_file: str = Field(default="mcp-catalog.yml", description="Path to catalog configuration file")
     mcpgateway_catalog_auto_health_check: bool = Field(default=True, description="Automatically health check catalog servers")
     mcpgateway_catalog_cache_ttl: int = Field(default=3600, description="Catalog cache TTL in seconds")
+    mcpgateway_catalog_page_size: int = Field(default=100, description="Number of catalog servers per page")
 
     # Security
     skip_ssl_verify: bool = False

mcpgateway/schemas.py

@@ -5381,6 +5381,7 @@ class CatalogServer(BaseModel):
     requires_api_key: bool = Field(default=False, description="Whether API key is required")
     secure: bool = Field(default=False, description="Whether additional security is required")
     tags: List[str] = Field(default_factory=list, description="Tags for categorization")
+    transport: Optional[str] = Field(None, description="Transport type: SSE, STREAMABLEHTTP, or WEBSOCKET")
     logo_url: Optional[str] = Field(None, description="URL to server logo/icon")
     documentation_url: Optional[str] = Field(None, description="URL to server documentation")
     is_registered: bool = Field(default=False, description="Whether server is already registered")

mcpgateway/services/catalog_service.py

@@ -31,6 +31,7 @@
     CatalogServerStatusResponse,
 )
 from mcpgateway.services.gateway_service import GatewayService
+from mcpgateway.utils.create_slug import slugify
 
 logger = logging.getLogger(__name__)
 
@@ -120,6 +121,9 @@ async def get_catalog_servers(self, request: CatalogListRequest, db) -> CatalogL
         for server_data in servers:
             server = CatalogServer(**server_data)
             server.is_registered = server.url in registered_urls
+            # Set availability based on registration status (registered servers are assumed available)
+            # Individual health checks can be done via the /status endpoint
+            server.is_available = server.is_registered or server_data.get("is_available", True)
             catalog_servers.append(server)
 
         # Apply filters
@@ -206,18 +210,22 @@ async def register_catalog_server(self, catalog_id: str, request: Optional[Catal
             # First-Party
             from mcpgateway.schemas import GatewayCreate  # pylint: disable=import-outside-toplevel
 
-            # Detect transport type from URL or use SSE as default
-            url = server_data["url"].lower()
-            # Check for SSE patterns (highest priority)
-            if url.endswith("/sse") or "/sse/" in url:
-                transport = "SSE"  # SSE endpoints or paths containing /sse/
-            elif url.startswith("ws://") or url.startswith("wss://"):
-                transport = "SSE"  # WebSocket URLs typically use SSE transport
-            # Then check for HTTP patterns
-            elif "/mcp" in url or url.endswith("/"):
-                transport = "STREAMABLEHTTP"  # Generic MCP endpoints typically use HTTP
-            else:
-                transport = "SSE"  # Default to SSE for most catalog servers
+            # Use explicit transport if provided, otherwise auto-detect from URL
+            transport = server_data.get("transport")
+            if not transport:
+                # Detect transport type from URL or use SSE as default
+                url = server_data["url"].lower()
+                # Check for WebSocket patterns (highest priority)
+                if url.startswith("ws://") or url.startswith("wss://"):
+                    transport = "WEBSOCKET"  # WebSocket transport for ws:// and wss:// URLs
+                # Check for SSE patterns
+                elif url.endswith("/sse") or "/sse/" in url:
+                    transport = "SSE"  # SSE endpoints or paths containing /sse/
+                # Then check for HTTP patterns
+                elif "/mcp" in url or url.endswith("/"):
+                    transport = "STREAMABLEHTTP"  # Generic MCP endpoints typically use HTTP
+                else:
+                    transport = "SSE"  # Default to SSE for most catalog servers
 
             # Check for IPv6 URLs early to provide a clear error message
             url = server_data["url"]
@@ -237,6 +245,8 @@ async def register_catalog_server(self, catalog_id: str, request: Optional[Catal
 
             # Set authentication based on server requirements
             auth_type = server_data.get("auth_type", "Open")
+            skip_initialization = False  # Flag to skip connection test for OAuth servers without creds
+
             if request and request.api_key and auth_type != "Open":
                 # Handle all possible auth types from the catalog
                 if auth_type in ["API Key", "API"]:
@@ -248,10 +258,54 @@ async def register_catalog_server(self, catalog_id: str, request: Optional[Catal
                     gateway_data["auth_type"] = "bearer"
                     gateway_data["auth_token"] = request.api_key
                 else:
-                    # For any other auth types, use custom headers
+                    # For any other auth types, use custom headers (as list of dicts)
                     gateway_data["auth_type"] = "authheaders"
-                    gateway_data["auth_header_key"] = "X-API-Key"
-                    gateway_data["auth_header_value"] = request.api_key
+                    gateway_data["auth_headers"] = [{"key": "X-API-Key", "value": request.api_key}]
+            elif auth_type in ["OAuth2.1", "OAuth"]:
+                # OAuth server without credentials - register but skip initialization
+                # User will need to complete OAuth flow later
+                skip_initialization = True
+                logger.info(f"Registering OAuth server {server_data['name']} without credentials - OAuth flow required later")
+
+            # For OAuth servers without credentials, register directly without connection test
+            if skip_initialization:
+                # Create minimal gateway entry without tool discovery
+                # First-Party
+                from mcpgateway.db import Gateway as DbGateway  # pylint: disable=import-outside-toplevel
+
+                gateway_create = GatewayCreate(**gateway_data)
+                slug_name = slugify(gateway_data["name"])
+
+                db_gateway = DbGateway(
+                    name=gateway_data["name"],
+                    slug=slug_name,
+                    url=gateway_data["url"],
+                    description=gateway_data["description"],
+                    tags=gateway_data.get("tags", []),
+                    transport=gateway_data["transport"],
+                    capabilities={},
+                    auth_type=None,  # Will be set during OAuth configuration
+                    enabled=False,  # Disabled until OAuth is configured
+                    created_via="catalog",
+                    visibility="public",
+                    version=1,
+                )
+
+                db.add(db_gateway)
+                db.commit()
+                db.refresh(db_gateway)
+
+                # First-Party
+                from mcpgateway.schemas import GatewayRead  # pylint: disable=import-outside-toplevel
+
+                gateway_read = GatewayRead.model_validate(db_gateway)
+
+                return CatalogServerRegisterResponse(
+                    success=True,
+                    server_id=str(gateway_read.id),
+                    message=f"Successfully registered {gateway_read.name} - OAuth configuration required before activation",
+                    error=None,
+                )
 
             gateway_create = GatewayCreate(**gateway_data)
 
@@ -284,9 +338,31 @@ async def register_catalog_server(self, catalog_id: str, request: Optional[Catal
 
         except Exception as e:
             logger.error(f"Failed to register catalog server {catalog_id}: {e}")
+
+            # Map common exceptions to user-friendly messages
+            error_str = str(e)
+            user_message = "Registration failed"
+
+            if "Connection refused" in error_str or "connect" in error_str.lower():
+                user_message = "Server is offline or unreachable"
+            elif "SSL" in error_str or "certificate" in error_str.lower():
+                user_message = "SSL certificate verification failed - check server security settings"
+            elif "timeout" in error_str.lower() or "timed out" in error_str.lower():
+                user_message = "Server took too long to respond - it may be slow or unavailable"
+            elif "401" in error_str or "Unauthorized" in error_str:
+                user_message = "Authentication failed - check API key or OAuth credentials"
+            elif "403" in error_str or "Forbidden" in error_str:
+                user_message = "Access forbidden - check permissions and API key"
+            elif "404" in error_str or "Not Found" in error_str:
+                user_message = "Server endpoint not found - check URL is correct"
+            elif "500" in error_str or "Internal Server Error" in error_str:
+                user_message = "Remote server error - the MCP server is experiencing issues"
+            elif "IPv6" in error_str:
+                user_message = "IPv6 URLs are not supported - please use IPv4 or domain names"
+
             # Don't rollback here - let FastAPI handle it
             # db.rollback()
-            return CatalogServerRegisterResponse(success=False, server_id="", message="Registration failed", error=str(e))
+            return CatalogServerRegisterResponse(success=False, server_id="", message=user_message, error=error_str)
 
     async def check_server_availability(self, catalog_id: str) -> CatalogServerStatusResponse:
         """Check if a catalog server is available.

mcpgateway/static/admin.js

@@ -4798,7 +4798,7 @@ function showTab(tabName) {
                 if (tabName === "mcp-registry") {
                     // Load MCP Registry content
                     const registryContent = safeGetElement(
-                        "mcp-registry-content",
+                        "mcp-registry-servers",
                     );
                     if (registryContent) {
                         // Always load on first visit or if showing loading message
@@ -4820,7 +4820,7 @@ function showTab(tabName) {
                                         "GET",
                                         `${rootPath}/admin/mcp-registry/partial`,
                                         {
-                                            target: "#mcp-registry-content",
+                                            target: "#mcp-registry-servers",
                                             swap: "innerHTML",
                                         },
                                     )