Skip to content

Commit 9960f29

Browse files
mbaldessarimbaldessari
authored
Merge pull request #647 from mbaldessari/ingress-docs
Initial pass at ingress-bgp docs
2 parents 61424d1 + 13ed30f commit 9960f29

File tree

8 files changed

+705
-0
lines changed

8 files changed

+705
-0
lines changed

‎content/patterns/ingress-mesh-bgp/_index.adoc‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
---
2+
title: Ingress Mesh BGP
3+
date: 2025-01-26
4+
tier: sandbox
5+
summary: This pattern demonstrates multi-cluster service mesh networking using MetalLB with BGP and Red Hat Service Interconnect (Skupper) for cross-cluster connectivity.
6+
rh_products:
7+
- Red Hat OpenShift Container Platform
8+
- Red Hat Advanced Cluster Management
9+
- Red Hat Service Interconnect
10+
industries:
11+
- General
12+
- Telecommunications
13+
aliases: /ingress-mesh-bgp/
14+
# pattern_logo: ingress-mesh-bgp.png # TODO: Create pattern logo
15+
links:
16+
github: https://github.com/validatedpatterns/ingress-mesh-bgp
17+
install: imbgp-getting-started
18+
bugs: https://github.com/validatedpatterns/ingress-mesh-bgp/issues
19+
ci: ingressmeshbgp
20+
---
21+
:toc:
22+
:imagesdir: /images
23+
:_content-type: ASSEMBLY
24+
include::modules/comm-attributes.adoc[]
25+
26+
include::modules/imbgp-about.adoc[leveloffset=+1]
27+
28+
include::modules/imbgp-architecture.adoc[leveloffset=+1]
29+
30+
[id="next-steps_imbgp-index"]
31+
== Next steps
32+
33+
* link:imbgp-getting-started[Deploy the Ingress Mesh BGP pattern].

‎content/patterns/ingress-mesh-bgp/imbgp-getting-started.adoc‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
title: Getting started
3+
weight: 10
4+
aliases: /ingress-mesh-bgp/imbgp-getting-started/
5+
---
6+
7+
:toc:
8+
:imagesdir: /images
9+
:_content-type: ASSEMBLY
10+
include::modules/comm-attributes.adoc[]
11+
12+
include::modules/imbgp-deploying.adoc[leveloffset=1]
13+
14+
[id="next-steps_imbgp-getting-started"]
15+
== Next steps
16+
17+
After the pattern is deployed and working correctly, you can:
18+
19+
* Verify the BGP routing is functioning correctly by checking the routing table on the core router
20+
* Test the anycast service by accessing it from the client VM
21+
* Explore the Red Hat Service Interconnect (Skupper) console to see the cross-cluster connectivity

‎modules/imbgp-about.adoc‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
:_content-type: CONCEPT
2+
:imagesdir: ../../images
3+
4+
[id="about-ingress-mesh-bgp-pattern"]
5+
= About the Ingress Mesh BGP pattern
6+
7+
Use case::
8+
9+
* Deploy multi-cluster applications with unified ingress using BGP-based load balancing.
10+
* Enable anycast IP addressing for seamless failover between OpenShift clusters.
11+
* Connect services across clusters using Red Hat Service Interconnect (Skupper) for secure east-west traffic.
12+
* Leverage Kubernetes Gateway API to give application developers self-service control over service routing without depending on network infrastructure teams.
13+
* Demonstrate enterprise-grade BGP routing integration with Kubernetes/OpenShift environments.
14+
+
15+
[NOTE]
16+
====
17+
This pattern is designed for AWS environments and simulates a datacenter-like BGP network topology using EC2 instances as routing infrastructure.
18+
====
19+
20+
Background::
21+
Modern distributed applications often span multiple clusters for high availability, geographic distribution, or workload isolation. Traditional ingress solutions provide per-cluster access, but organizations need unified entry points that can intelligently route traffic across clusters.
22+
23+
This pattern addresses these requirements by combining:
24+
25+
* *MetalLB with BGP mode* - Provides load balancer services that advertise routes via BGP, enabling anycast addressing where the same IP is reachable through multiple clusters.
26+
* *Gateway API* - Delivers L4 and L7 service routing via next generation Ingress, Load Balancing, and Service Mesh APIs. Provides GatewayClasses (infrastructure) and Gateways (operations) so application developers can create routes (HTTPRoute, GRPCRoute, etc.) for their services.
27+
* *Red Hat Service Interconnect (Skupper)* - Creates a secure application-layer virtual network connecting services across clusters without requiring VPN or special network configurations.
28+
* *FRRouting (FRR)* - Industry-standard routing software running on EC2 instances that acts as the BGP peering infrastructure, simulating top-of-rack (TOR) switches and core routers.
29+
30+
Gateway API plays a central role in this architecture. It is the intermediary layer between BGP and Skupper: traffic arriving at a cluster via BGP-advertised anycast IPs is routed through Gateway API down to the appropriate Skupper site, where Skupper handles inter-cluster routing for sparse deployments or services not locally available. Gateway API also separates concerns across organizational roles — infrastructure teams define GatewayClasses, operations teams create Gateways, and application developers independently manage their own Routes (HTTPRoute, GRPCRoute, etc.) for both simple routing and more advanced mesh-like routing scenarios.
31+
32+
[id="about-solution-imbgp"]
33+
== About the solution
34+
35+
This pattern deploys a complete multi-cluster networking demonstration on AWS that includes:
36+
37+
* Two OpenShift clusters, designated as "west" (ACM hub) and "east" (spoke)
38+
* A simulated routing infrastructure with FRR-based routers
39+
* MetalLB configured in BGP mode on both clusters
40+
* Red Hat Service Interconnect linking services between clusters
41+
* A hello-world application deployed across both clusters demonstrating the connectivity
42+
43+
The solution uses ECMP (Equal-Cost Multi-Path) routing to distribute traffic across both clusters when accessing the anycast IP address.
44+
45+
.Benefits of the Ingress Mesh BGP pattern:
46+
47+
* *Unified service access* - Single IP address reaches services on multiple clusters
48+
* *Automatic failover* - BGP route withdrawal provides fast failover when a cluster becomes unavailable
49+
* *Secure cross-cluster communication* - Skupper encrypts all inter-cluster traffic using mutual TLS
50+
* *No network infrastructure changes* - Skupper works over existing networks without VPNs or firewall changes
51+
* *GitOps-driven deployment* - All components are deployed and managed through ArgoCD
52+
* *Application owner autonomy* - App owners can describe their own routes on approved gateways and gatewayclasses without relying on network infrastructure teams
53+
54+
[id="about-technology-imbgp"]
55+
== About the technology
56+
57+
The following technologies are used in this solution:
58+
59+
https://www.redhat.com/en/technologies/cloud-computing/openshift/try-it[Red Hat OpenShift Platform]::
60+
An enterprise-ready Kubernetes container platform built for an open hybrid cloud strategy. It provides a consistent application platform to manage hybrid cloud, public cloud, and edge deployments.
61+
62+
https://www.redhat.com/en/technologies/management/advanced-cluster-management[Red Hat Advanced Cluster Management for Kubernetes]::
63+
Controls clusters and applications from a single console, with built-in security policies. Extends the value of Red Hat OpenShift by deploying apps, managing multiple clusters, and enforcing policies across multiple clusters at scale.
64+
65+
https://www.redhat.com/en/technologies/cloud-computing/service-interconnect[Red Hat Service Interconnect]::
66+
Based on the open source Skupper project, Red Hat Service Interconnect enables secure communication between services across different environments without requiring VPN infrastructure or special firewall rules. It creates a virtual application network that works at Layer 7.
67+
68+
https://metallb.universe.tf/[MetalLB]::
69+
A load-balancer implementation for bare metal Kubernetes clusters, using standard routing protocols. In this pattern, MetalLB operates in BGP mode to advertise service IPs as routes to the upstream network infrastructure.
70+
71+
https://frrouting.org/[FRRouting (FRR)]::
72+
A free and open source Internet routing protocol suite for Linux and Unix platforms. It implements BGP, OSPF, RIP, and other protocols. In this pattern, FRR runs on EC2 instances to simulate datacenter routing infrastructure.
73+
74+
https://gateway-api.sigs.k8s.io/[Kubernetes Gateway API]::
75+
The next generation of Kubernetes Ingress, providing a more expressive and extensible API for managing traffic into and within a cluster. Gateway API is the intermediary layer between BGP and Skupper — traffic arriving via BGP passes through Gateway API for routing before reaching Skupper for inter-cluster communication. It provides GatewayClasses for infrastructure providers, Gateways for operations teams, and Routes (HTTPRoute, GRPCRoute, etc.) for application developers, enabling self-service routing without relying on network infrastructure teams.

‎modules/imbgp-architecture.adoc‎

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
:_content-type: CONCEPT
2+
:imagesdir: ../../images
3+
4+
[id="architecture-ingress-mesh-bgp"]
5+
= Architecture
6+
7+
The Ingress Mesh BGP pattern demonstrates a multi-cluster networking architecture that combines BGP-based anycast ingress with service mesh connectivity. The architecture simulates an enterprise network topology on AWS.
8+
9+
[id="network-topology-imbgp"]
10+
== Network topology
11+
12+
The pattern creates the following network components on AWS:
13+
14+
.Network topology diagram
15+
image::ingress-mesh-bgp/network.svg[Network Topology,700]
16+
17+
=== VPCs and subnets
18+
19+
The pattern provisions several VPCs to simulate separate network segments:
20+
21+
* *Client-Core VPC (192.168.8.0/24)* - Contains the client VM and core router
22+
* *Core-West TOR VPC (192.168.12.0/24)* - Connects the core router to the west cluster's top-of-rack router
23+
* *Core-East TOR VPC (192.168.16.0/24)* - Connects the core router to the east cluster's top-of-rack router
24+
* *West Workers VPC (10.0.0.0/16)* - The west OpenShift cluster's VPC
25+
* *East Workers VPC (10.1.0.0/16)* - The east OpenShift cluster's VPC
26+
27+
=== Routing infrastructure
28+
29+
The pattern deploys EC2 instances running FRRouting to create a simulated datacenter network:
30+
31+
[cols="1,1,2"]
32+
|===
33+
|Component |ASN |Description
34+
35+
|Core Router
36+
|64666
37+
|Central router that peers with both TOR routers and advertises client network routes
38+
39+
|West TOR
40+
|64001
41+
|Top-of-rack router for the west cluster, peers with core and west OpenShift workers
42+
43+
|East TOR
44+
|64002
45+
|Top-of-rack router for the east cluster, peers with core and east OpenShift workers
46+
47+
|West OpenShift (MetalLB)
48+
|65001
49+
|MetalLB speakers on west cluster workers, peer with west TOR
50+
51+
|East OpenShift (MetalLB)
52+
|65002
53+
|MetalLB speakers on east cluster workers, peer with east TOR
54+
|===
55+
56+
=== Anycast addressing
57+
58+
Both clusters advertise the same anycast IP range (192.168.155.0/24) via BGP. When a client accesses an anycast IP:
59+
60+
.Client path to services via anycast and BGP
61+
image::ingress-mesh-bgp/ingress-path.svg[Client Ingress Path,700]
62+
63+
. The core router receives BGP advertisements from both TOR routers for the anycast range
64+
. ECMP routing distributes traffic across both paths
65+
. Requests reach either the west or east cluster based on the routing decision
66+
. If one cluster becomes unavailable, BGP route withdrawal automatically redirects traffic to the remaining cluster
67+
68+
[id="cluster-components-imbgp"]
69+
== Cluster components
70+
71+
=== West cluster (Hub)
72+
73+
The west cluster acts as the management hub and includes:
74+
75+
* *Red Hat Advanced Cluster Management* - Manages the east cluster as a spoke
76+
* *HashiCorp Vault* - Centralized secrets management
77+
* *External Secrets Operator* - Synchronizes secrets from Vault to Kubernetes
78+
* *MetalLB* - Provides BGP-advertised load balancer services (ASN 65001)
79+
* *Gateway API* - Routes incoming traffic to appropriate services, providing the intermediary layer between BGP ingress and Skupper
80+
* *Red Hat Service Interconnect (Skupper)* - Hosts the Skupper site with link access enabled
81+
* *Hello-world application* - Frontend component of the demo application
82+
83+
=== East cluster (Spoke)
84+
85+
The east cluster is a managed spoke that includes:
86+
87+
* *External Secrets Operator* - Retrieves secrets from the hub's Vault
88+
* *MetalLB* - Provides BGP-advertised load balancer services (ASN 65002)
89+
* *Gateway API* - Routes incoming traffic to appropriate services, providing the intermediary layer between BGP ingress and Skupper
90+
* *Red Hat Service Interconnect (Skupper)* - Connects back to the west cluster's Skupper site
91+
* *Hello-world application* - Backend component of the demo application
92+
93+
[id="metallb-imbgp"]
94+
== MetalLB
95+
96+
MetalLB provides load balancer services on bare metal and cloud environments where cloud-native load balancers are not available or not suitable:
97+
98+
* Each cluster runs MetalLB in BGP mode with a unique ASN (65001 for west, 65002 for east)
99+
* MetalLB speakers on worker nodes peer with the local TOR router and advertise service IPs via BGP
100+
* Both clusters advertise the same anycast IP range (192.168.155.0/24), enabling ECMP routing from the core
101+
* When a cluster becomes unavailable, its BGP routes are withdrawn and traffic is automatically redirected to the remaining cluster
102+
103+
[id="gateway-api-imbgp"]
104+
== Gateway API
105+
106+
Gateway API provides the L4/L7 routing layer between BGP ingress and application services:
107+
108+
* *GatewayClass* - Defined by infrastructure providers to describe the type of gateway infrastructure available
109+
* *Gateway* - Created by operations teams to instantiate a gateway from a GatewayClass, defining listeners and allowed routes
110+
* *HTTPRoute / GRPCRoute* - Created by application developers to describe how traffic should be routed to their services
111+
112+
Gateway API is the intermediary step between BGP and Skupper. Traffic arriving at a cluster via BGP-advertised anycast IPs passes through Gateway API for service routing before reaching Skupper for inter-cluster communication. This separation of concerns allows application developers to define their own routing rules on approved gateways without relying on network infrastructure teams.
113+
114+
[id="service-interconnect-imbgp"]
115+
== Red Hat Service Interconnect
116+
117+
Red Hat Service Interconnect (based on Skupper) creates a virtual application network between the clusters:
118+
119+
* The west cluster hosts a Skupper site with `linkAccess: default`, allowing other sites to connect
120+
* The east cluster establishes a link to the west cluster using a pre-shared access token
121+
* Services exposed through Skupper listeners become accessible across both clusters
122+
* All traffic between sites is encrypted using mutual TLS
123+
124+
The pattern uses the Skupper v2 API with the following components:
125+
126+
* *Site* - Defines the Skupper installation in each namespace
127+
* *Listener* - Exposes a service to the Skupper network
128+
* *Connector* - Connects a local workload to a Skupper-exposed service
129+
* *AccessGrant/AccessToken* - Manages secure connection between sites
130+
131+
[id="gitops-structure-imbgp"]
132+
== GitOps structure
133+
134+
The pattern follows the Validated Patterns framework:
135+
136+
----
137+
ingress-mesh-bgp/
138+
├── values-global.yaml # Global configuration
139+
├── values-west.yaml # West (hub) cluster configuration
140+
├── values-east.yaml # East (spoke) cluster configuration
141+
├── charts/
142+
│ ├── all/
143+
│ │ ├── hello-world/ # Demo application
144+
│ │ ├── metallb/ # MetalLB configuration
145+
│ │ └── rhsi/ # Skupper configuration for west
146+
│ └── east-site/
147+
│ └── rhsi-east/ # Skupper configuration for east
148+
└── ansible/
149+
└── playbooks/ # Infrastructure automation
150+
----
151+
152+
ArgoCD manages the deployment of all components, with Red Hat Advanced Cluster Management distributing configurations to the appropriate clusters.

0 commit comments

Comments
 (0)