Changes to make tests work with docker, and to REVERSE_CONNECTION cluster #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open

basundhara-c wants to merge 513 commits into agrawroh:feat-rev-conns-new from basundhara-c:feat-rev-conns-new

.bazelrc

-Original file line number
+Diff line change
@@ Expand Up @@
     build:macos --action_env=PATH=/opt/homebrew/bin:/opt/local/bin:/usr/local/bin:/usr/bin:/bin
     build:macos --host_action_env=PATH=/opt/homebrew/bin:/opt/local/bin:/usr/local/bin:/usr/bin:/bin
     build:macos --define tcmalloc=disabled
+    build:macos --cxxopt=-Wno-nullability-completeness
     # macOS ASAN/UBSAN
     build:macos-asan --config=asan
@@ Expand Down Expand Up / @@ -275,7 +276,7 @@ build:fuzz-coverage --test_tag_filters=-nocoverage @@
     # resources required to build and run the tests.
     build:fuzz-coverage --define=wasm=disabled
     build:fuzz-coverage --config=fuzz-coverage-config
-    build:fuzz-coverage-config --//tools/coverage:config=//test:fuzz_coverage_config
+    build:fuzz-coverage-config --//tools/coverage:config=@envoy//test:fuzz_coverage_config
     build:cache-local --remote_cache=grpc://localhost:9092
@@ Expand Down Expand Up @@
     common:common-envoy-engflow --credential_helper=*.engflow.com=%workspace%/bazel/engflow-bazel-credential-helper.sh
     common:common-envoy-engflow --grpc_keepalive_time=60s
     common:common-envoy-engflow --grpc_keepalive_timeout=30s
+    common:common-envoy-engflow --remote_cache_compression
     common:cache-envoy-engflow --remote_cache=grpcs://mordenite.cluster.engflow.com
     common:cache-envoy-engflow --remote_timeout=3600s
@@ Expand All @@
     common:remote-cache-envoy-engflow --config=cache-envoy-engflow
     # Specifies the rustfmt.toml for all rustfmt_test targets.
-    build --@rules_rust//rust/settings:rustfmt.toml=//:rustfmt.toml
+    build --@rules_rust//rust/settings:rustfmt.toml=@envoy//:rustfmt.toml
     #############################################################################
     # debug: Various Bazel debugging flags
@@ Expand Down @@

.bazelversion

Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		7.6.0
		7.6.1

.clang-tidy

-Original file line number
+Diff line change
@@ Expand Up / @@ -71,8 +71,7 @@ CheckOptions: @@
         |^value_or$|
         |^Ip6(ntohl|htonl)$|
         |^get_$|
-        |^HeaderHasValue(Ref)?$|
-        |^HeaderValueOf$|
+        |^ContainsHeader$|
         |^Is(Superset|Subset)OfHeaders$|
         |^LLVMFuzzerInitialize$|
         |^LLVMFuzzerTestOneInput$|
@@ Expand Down @@

.devcontainer/README.md

-Original file line number
+Diff line change
@@ Expand Up @@
     Docker for Mac/Windows is known to have disk performance issue, this makes formatting all files in the container very slow.
     [Update the mount consistency to 'delegated'](https://code.visualstudio.com/docs/remote/containers-advanced#_update-the-mount-consistency-to-delegated-for-macos) is recommended.
+    ## Detailed Setup Guide for Local Development
+    This section provides an exhaustive walkthrough for setting up Envoy source code locally for development and debugging. This guide has been tested on both Apple M1 Max and Intel Core i9 machines running macOS Sequoia (version 15.5).
+    For developers who are not C++ experts or are more familiar with CMake/Make build systems, Envoy's Bazel-based build system and dev container support makes local development much more accessible.
+    ### Prerequisites
+    - macOS Sequoia (version 15.5) or compatible system
+    - VSCode or Cursor IDE
+    - Docker Desktop with sufficient resources
+    - Dev Containers extension
+    ### Step-by-Step Setup Process
+    #### 1. Repository Setup
+. Clone the Envoy repository from GitHub or create a fork:
+       ```bash
+       git clone https://github.com/envoyproxy/envoy.git
+       # OR
+       git clone https://github.com/YOUR_USERNAME/envoy.git
+       ```
+. Open the downloaded folder in VSCode or Cursor
+. The IDE will detect the `.devcontainer` folder and prompt to reopen in Dev Container mode. Accept this prompt.
+       You should see a blue label at the bottom-left of the window indicating "Container" or "Dev Container" status, confirming you're running in container mode.
+       ![Container Status](./images/container-status.png)
+    #### 2. Initialize Development Environment
+. Once in dev container mode, open a new terminal (**Terminal** → **New Terminal**)
+. Run the compilation database refresh script:
+       ```bash
+       ci/do_ci.sh refresh_compdb
+       ```
+       This script refreshes the compilation database and generates dependencies for code completion, including protobuf generated codes and external dependencies. This process may take 30-60 minutes depending on your machine performance.
+       > **Note**: Re-run this script whenever you change proto definitions or modify bazel structure to maintain proper code completion.
+    #### 3. Troubleshooting Build Issues
+    If you encounter "no space left on device" errors during the compilation process, you'll need to adjust Docker Desktop settings:
+    ![Docker Space Error](./images/docker-space-error.png)
+    **Resolution Steps:**
+. Open Docker Desktop
+. Navigate to **Settings** → **Resources**
+. Increase **Memory Limit** to at least 8GB (recommended: 16GB+)
+. Increase **Disk usage limit** to at least 100GB
+       ![Docker Settings](./images/docker-settings.png)
+. If settings are grayed out, perform a factory reset:
+       - Go to **Troubleshoot** → **Reset to factory defaults**
+       - Restart Docker Desktop
+       - Reconfigure the resource limits
+       - Click **Apply and restart**
+    **Successful Build Verification:**
+    Look for a completion message similar to (the total actions might vary):
+    ```
+    Build completed successfully, 11415 total actions
+    ```
+    ![Build Success](./images/build-success.png)
+    #### 4. Debug Configuration Setup
+    ##### Create Envoy Configuration File
+    Before generating debug configuration, create an `envoy.yaml` file in the repository root (you can choose any name of your liking).This is a standard envoy configuration, you should be able to change it according to your need. Here's a sample configuration for debugging OpenTelemetry tracing:
+    ```yaml
+    admin:
+      address:
+        socket_address:
+          protocol: TCP
+          address: 0.0.0.0
+          port_value: 9902
+    static_resources:
+      listeners:
+      - name: listener_0
+        address:
+          socket_address:
+            protocol: TCP
+            address: 0.0.0.0
+            port_value: 10000
+        filter_chains:
+        - filters:
+          - name: envoy.filters.network.http_connection_manager
+            typed_config:
+              "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
+              stat_prefix: ingress_http
+              # Enable tracing
+              tracing:
+                provider:
+                  name: envoy.tracers.opentelemetry
+                  typed_config:
+                    "@type": type.googleapis.com/envoy.config.trace.v3.OpenTelemetryConfig
+                    grpc_service:
+                      envoy_grpc:
+                        cluster_name: opentelemetry_collector
+                      timeout: 100s
+                    service_name: envoy-proxy
+                random_sampling:
+                  value: 100
+              access_log:
+              - name: envoy.access_loggers.stdout
+                typed_config:
+                  "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
+              http_filters:
+              - name: envoy.filters.http.router
+                typed_config:
+                  "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
+              route_config:
+                name: local_route
+                virtual_hosts:
+                - name: local_service
+                  domains: ["*"]
+                  routes:
+                  - match:
+                      prefix: "/"
+                    route:
+                      host_rewrite_literal: www.envoyproxy.io
+                      cluster: service_envoyproxy_io
+      clusters:
+      - name: service_envoyproxy_io
+        type: LOGICAL_DNS
+        # Comment out the following line to test on v6 networks
+        dns_lookup_family: V4_ONLY
+        lb_policy: ROUND_ROBIN
+        load_assignment:
+          cluster_name: service_envoyproxy_io
+          endpoints:
+          - lb_endpoints:
+            - endpoint:
+                address:
+                  socket_address:
+                    address: www.envoyproxy.io
+                    port_value: 443
+        transport_socket:
+          name: envoy.transport_sockets.tls
+          typed_config:
+            "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
+            sni: www.envoyproxy.io
+      # OpenTelemetry Collector cluster
+      - name: opentelemetry_collector
+        type: STRICT_DNS
+        dns_lookup_family: V4_ONLY
+        lb_policy: ROUND_ROBIN
+        typed_extension_protocol_options:
+          envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
+            "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
+            explicit_http_config:
+              http2_protocol_options: {}
+        load_assignment:
+          cluster_name: opentelemetry_collector
+          endpoints:
+          - lb_endpoints:
+            - endpoint:
+                address:
+                  socket_address:
+                    address: localhost
+                    port_value: 5002
+    ```
+    ##### Generate Debug Configuration
+    Run the debug configuration generator:
+    ```bash
+    tools/vscode/generate_debug_config.py //source/exe:envoy-static --args "-c envoy.yaml"
+    ```
+    **Advanced Configuration Options:**
+    The script now supports automatic compiler configuration detection and manual override options:
+    - **Auto-detection**: The script automatically detects the optimal compiler configuration based on your platform:
+      - On ARM64/aarch64 architectures (Apple Silicon Macs): Automatically uses `clang` for better C++20 concepts compatibility
+      - On other architectures: Uses Bazel's default configuration
+    - **Manual override**: You can explicitly specify a build configuration using the `--config` flag:
+      ```bash
+      # Force use of clang configuration
+      tools/vscode/generate_debug_config.py //source/exe:envoy-static --args "-c envoy.yaml" --config clang
+      # Force use of gcc configuration
+      tools/vscode/generate_debug_config.py //source/exe:envoy-static --args "-c envoy.yaml" --config gcc
+      ```
+    - **Additional options**:
+      ```bash
+      # Use LLDB debugger instead of GDB (recommended for macOS)
+      tools/vscode/generate_debug_config.py //source/exe:envoy-static --args "-c envoy.yaml" --debugger lldb
+      # Overwrite existing configuration completely
+      tools/vscode/generate_debug_config.py //source/exe:envoy-static --args "-c envoy.yaml" --overwrite
+      ```
+    ![Debug Configuration Run](./images/debug-configuration-run.png)
+    > **Important**: This command may take over an hour to complete. You'll need to re-run this script whenever you make code changes.
+    > **ARM64/Apple Silicon Note**: If you're using an ARM64 system (like Apple Silicon Macs) and encounter GCC-related compilation errors with C++20 concepts compatibility, the script will automatically use clang configuration to resolve these issues. This addresses protobuf dependency compatibility problems that can occur with GCC on ARM64 architectures.
+    There are two types of debuggers envoy community recommends to use, GDB (GNU Debugger) , a widely used and powerful debugger for various languages, including C++ and  LLDB,  the LLVM project's debugger, often used with Clang-compiled code (Clang is one of the many compilers for C++).
+    Now click F5 to start debugging, this should have generated the `launch.json` as shown below. The generated `launch.json` will use GDB by default.
+    ![GDB Launch JSON](./images/GDB-launch-json.png)
+    However, if you encounter an error with GDB (which can happen with some Mac Machines), try using the LLDB debugger as shown in the next section.
+    ![GDB Error](./images/gdb-error.png)
+    ##### Configure LLDB Debugger (Recommended for macOS)
+    For better compatibility on macOS, modify the generated `launch.json` to use LLDB:
+. Open `launch.json` (Cmd+P → type "launch.json")
+       ![Launch JSON](./images/GDB-launch-json.png)
+. Replace the GDB configuration with LLDB:
+    ```json
+    {
+        "version": "0.2.0",
+        "configurations": [
+          {
+            "name": "LLDB //source/exe:envoy-static",
+            "type": "lldb",
+            "request": "launch",
+            "program": "/build/.cache/bazel/_bazel_vscode/2d35de14639eaad1ac7060a4dd7e3351/execroot/envoy/bazel-out/k8-dbg/bin/source/exe/envoy-static",
+            "args": ["-c", "envoy.yaml"],
+            "cwd": "${workspaceFolder}",
+            "stopOnEntry": false,
+            "sourceMap": {
+              "/proc/self/cwd": "${workspaceFolder}"
+            }
+          }
+        ]
+    }
+    ```
+    > **Note**: Update the `program` path with the actual hash from your generated GDB configuration. Ensure the CodeLLDB extension is installed in your dev container.
+    ![CodeLLDB Extension](./images/code-lldb-extension.png)
+    #### 5. Setting Up Breakpoints
+    ##### Main Entry Point
+    Add a breakpoint in the main function located at `source/exe/main.cc`:
+    ![Main Breakpoint](./images/main-breakpoint.png)
+    ##### Custom Debugging Points
+    You can add additional debug points, For example, for tracing debugging, add breakpoints in specific areas like `source/extensions/tracers/opentelemetry/opentelemetry_tracer_impl.cc`:
+    ![Tracer Breakpoint](./images/tracer-breakpoint.png)
+    #### 6. Starting Debug Session
+. Start debugging by pressing **F5** or going to **Run** → **Start Debugging**
+. Allow 1-2 minutes for the debugger to attach to the process (Envoy is a large codebase)
+. Most of the Envoy source code is located in the `source` folder at the root of the repository.The breakpoint in `source/exe/main.cc` should activate
+       ![Debug Active](./images/debug-active.png)
+. Continue execution (**Run** → **Continue**) to see Envoy start up and begin emitting logs
+    #### 7. Verifying Setup
+    Once Envoy starts successfully, verify the setup by accessing. The ports are based on the configuration we have used earlier:
+    - **Admin Interface**: http://localhost:9902
+      ![Admin Interface](./images/admin-interface.png)
+    - **Proxy Endpoint**: Try reaching to the URL http://localhost:10000 and you should be redirected to envoyproxy.io based on the configuration file we gave above.
+      ![Proxy Endpoint](./images/proxy-endpoint.png)
+    - **Admin Endpoints**:
+    Try accessing different admin endpoints from your local machine.
+      - Clusters: http://localhost:9902/clusters
+      - Config Dump: http://localhost:9902/config_dump
+      - Stats: http://localhost:9902/stats
+      ![Cluster Endpoints](./images/cluster-endpoints.png)
+      ![Config Dump](./images/config-dump.png)
+      ![Stats Endpoints](./images/stats-endpoints.png)

.devcontainer/devcontainer.json

-Original file line number
+Diff line change
@@ Expand Up / @@ -15,6 +15,9 @@ @@
       ],
       "containerEnv": {
         "ENVOY_SRCDIR": "${containerWorkspaceFolder}",
+        "HTTPS_PROXY": "${env:VSCODE_CONTAINER_HTTPS_PROXY}",
+        "HTTP_PROXY": "${env:VSCODE_CONTAINER_HTTP_PROXY}",
+        "NO_PROXY": "${env:VSCODE_CONTAINER_NO_PROXY}"
       },
       "remoteUser": "vscode",
       "containerUser": "vscode",
@@ Expand Down @@

.devcontainer/images/GDB-launch-json.png

Sorry, something went wrong. Reload?