+ ## Contributing Please see the [contributing guide](https://github.com/nginxinc/nginx-azure-workshops/blob/main/CONTRIBUTING.md) for guidelines on how to best contribute to this project. +
+ ## Commercial Support Commercial support for this project may be available. Please get in touch with [NGINX sales](https://www.nginx.com/contact-sales/) or check your contract details for more info! + +
+ +### Community Slack + +We have a community [Slack](https://nginxcommunity.slack.com/)! + +If you are not a member, click [here](https://community.nginx.org/joinslack) to sign up (and let us know if the link does not seem to be working!) + +Once you join, check out the `#beginner-questions` and `nginx-users` channels :) diff --git a/ca-notes/LabOutline.md b/ca-notes/LabOutline.md deleted file mode 100644 index 7543cce..0000000 --- a/ca-notes/LabOutline.md +++ /dev/null @@ -1,184 +0,0 @@ -# Nginx for Azure Workshop Outline / Summary - -## Lab 0 - Prequesites - Subscription / Resources -## Lab 1 - Azure VNet/Subnet / Network Security Group / Nginx for Azure Overview -## Lab 2 - UbuntuVM/Docker / Windows VM / Cafe Demo Deployment -## Lab 3 - AKS / ACR / Nginx Ingress Controller Deployment -## Lab 4 - NIC Dashboard / Cafe Demo / Redis Deployment -## Lab 5 - Nginx for Azure Load Balancing / Reverse Proxy -## Lab 6 - Azure Key Vault / TLS Essentials -## Lab 7 - Azure Monitoring / Logging Analytics -## Lab 8 - Nginx Garage or Azure Petshop -## Lab 9 - Nginx Caching / Rate Limits / Juiceshop -## Lab 10 - Grafana for Azure -## Lab 11 - Optional Exercises - Windows VM -## Summary and Wrap-up - -
- -### Lab 0 - Prequesites - Subscription / Resources - -- Overview -In this Lab, the Prerequisite Requirements for both the Student and the Azure environment will be detailed. It is imperative that you have the appropriate computer, tools, skills, and Azure access to successfully complete the workshop. The Lab exercises must be done sequentially to build the environment as described. This is an intermediate level class, you must be proficient in several areas to successfully complete the workshop. Beginner level workshops are available from Nginx, to help prepare you for this workshop - see the References section below. - -- Learning Objectives -Verify you have the proper computer requirements - hardware and software. -- Hardware: Laptop, Admin rights, Internet connection -- Software: Visual Studio, Terminal, Chrome, Docker, AKS and AZ CLI, Redis-CLI. -Verify you have proper computer skills. -- Computer skills: Linux CLI, file mgmt, SSH/Terminal, Docker/Compose, Azure Portal, HTTP/S, Kubernetes Nodes/Pods/Services skills, Load Balancing concepts -- Optional: TLS, DNS, HTTP caching, Prometheus, Grafana, Redis -Verify you have the proper access to Azure resources. -- Azure subscription, list of Azure Roles/permissions here - -- Nginx for Azure Workshop has the following REQUIRED Nginx Skills -Students must be familiar with Nginx basic operations, configurations, and concepts for HTTP traffic. --- The Nginx Basics Workshop is HIGHLY recommended, students should have taken this workshop prior. --- The Nginx Plus Ingress Controller workshop is also HIGHLY recommended, students should have taken this workshop prior. --- Previous training on Azure Resource Groups, VMs, Azure Networking, AKS, ACR, and NSG is HIGHLY recommended. - -
- -### Lab 1 - Azure VNet/Subnet / Network Security Group / Nginx for Azure Overview - -- Overview -In this lab, you will be adding and configuring the Azure Networking components needed for this workshop. This will require a few network resources, and a Network Security Group to allow incoming traffic to your Nginx for Azure workshop resources. Then you will explore the Nginx for Azure product, as a quick Overview of what it is and how to deploy it. - -- Learning Objectives -Setup your Azure Vnet and Subnets -Setup your Azure Network Security Group for inbound traffic -Explore Nginx for Azure -Deploy an Nginx for Azure instance / enable logging -Test Nginx for Azure welcome page - -
- -### Lab 2 - Ubuntu VM/Docker / Windows VM / Cafe Demo Deployment - -- Overview -In this lab, you will deploy an Ubuntu VM, and configure it for a Legacy web application. You will deploy a Windows VM. You will configure Nginx for Azure to proxy and load balance these backends. - -- Learning Objectives -Deploy Ubuntu VM -Install Docker and Docker-compose -Run Legacy docker container apps - Cafe Demo -Optional Exercise: Deploy Windows VM -Configure Nginx Load Balancing for these apps - -
- -### Lab 3 - AKS / ACR / Nginx Ingress Controller Deployment - -- Overview -In this lab, you will deploy 2 AKS clusters, with Nginx Ingress Controllers. You will also deploy a private Container Registry. - -- Learning Objectives -Deploy 1 AKS cluster using the Azure AZ CLI. -Deploy 2nd AKS cluster with a bash script. -Deploy Nginx Plus Ingress Controller with F5 Private Registry, to both the Clusters. -Configure Nginx Plus Ingress Controller Dashboards. -Expose the NIC Plus Dashboards externally for Live Monitoring. - -
- -### 4 - Cafe Demo / Redis Deployment / Plus Dashboard - -- Overview -In this lab, you will deploy 2 AKS clusters, with Nginx Ingress Controllers, a Redis cluster, and a Modern Web Application. - -- Learning Objectives -Deploy a demo web application in the clusters. -Deploy and test a Redis In Memory Cache to the AKS cluster. -Configure Nginx Ingress for Cafe Demo. -Configure Nginx Ingress for Redis Leader. -Configure Nginx for Azure for Cafe and Redis applications. - -
- -### Lab 5 - Nginx Load Balancing / Reverse Proxy - -- Overview -In this lab, you will configure Nginx for Azure to Load Balance various workloads running in Azure. After successful configuration and adding Best Practice Nginx parameters, you will Load Test these applications, and test multiple load balancing and request routing parameters to suit different use cases. - -- Learning Objectives -Configure Nginx for Azure, to Load Balance traffic to both AKS Nginx Ingress Controllers. -Configure HTTP Split Clients, and route traffic to all 3 backend systems. -Load test the Legacy and Modern web applications. - -
- -### Lab 6 - Azure Key Vault / TLS Essentials - -- Overview -In this lab, you use Azure Key Vault for TLS certificates and keys. You will configure Nginx for Azure to use these Azure resources to terminate TLS. - -- Learning Objectives -Create a sample Azure Key Vault -Create a TLS cert/key -Configure and test Nginx for Azure to use the Azure Keys -Update the previous Nginx configurations to use TLS for apps -Update NSGs for TLS inbound traffic - -
- -### Lab 7 - Azure Montoring / Logging Analytics - -- Overview -Enable and configure Azure Monitoring for Nginx for Azure. Create custom Azure Dashboards for your applications. Gain experience using Azure Logs and logging tools. - -- Learning Objectives -Enable, configure, and test Azure Monitoring for Nginx for Azure. -Create a couple custom dashboards for your load balanced applications. -Explore the Azure logging and Analytics tools available. - -
- -### Lab 8 - Nginx Garage or Azure Petshop - -- Overview -In this lab, you will deploy a modern application in your AKS cluster. You will expose it with Nginx Ingress Controller and Nginx for Azure. - -- Learning Objectives -Deploy the modern app in AKS -Test and Verify the app is working correctly -Expose this application outside the cluster with Nginx Ingress Controller -Configure Nginx for Azure for this new application - -
- -### Lab 9 - Nginx Caching / Rate Limits / Juiceshop - -- Overview -In this lab, you will deploy an image rich application, and use Nginx Caching to cache images to improve performance. - -- Learning Objectives -Deploy JuiceShop in AKS cluster. -Expose JuiceShop with Nginx Ingress Controller. -Configure Nginx for Azure for load balancing JuiceShop. -Add Nginx Caching to improve delivery of images. - -
- -### Lab 10 - Grafana for Azure - -- Overview -In this lab, you will explore the Nginx and Grafana for Azure integration. - -- Learning Objectives -Deploy Grafana for Azure. -Configure the Datasource -Explore a sample Grafana Dashboard for Nginx for Azure - - -
- -### Lab 11 - Optional Exercises - - - - -
- -### Summary and Wrap-up - -- Summary and Wrap-up comments here \ No newline at end of file diff --git a/ca-notes/N4A Reference Arch.md b/ca-notes/N4A Reference Arch.md deleted file mode 100644 index ee85d0d..0000000 --- a/ca-notes/N4A Reference Arch.md +++ /dev/null @@ -1,44 +0,0 @@ -N4A Reference Arch - -Plus features: - -Least time algo -Active HC - not working -Dashboard - not available -metrics -Prometheus -KV store - no API access -Zone synch -Active Active nodes - -OIDC-jwt - no -Split clients -NTLM - no app -Caching -FIPS - AKS/NIC only - -NIC - Deep Insight -NLK -NAP WAF - not available - -Azure Integrations -Azure Console -AzureAD - not available -Azure Mon -Azure DNS -Azure Log Analisys -Azure Key Vault - certs/keys -Azure HSM - not possible - - -Infrastructure -FQDN Reg -Public IPs - -2 Demo Apps, -One for VMs with NTLM -One for AKS with NIC -And cafe to start - -**** - -N4A Feedback / Issues, feedback, suggestions diff --git a/ca-notes/N4A-feedback.md b/ca-notes/N4A-feedback.md deleted file mode 100644 index 4f73adf..0000000 --- a/ca-notes/N4A-feedback.md +++ /dev/null @@ -1,154 +0,0 @@ -# N4A Feedback / Issues, feedback, suggestions - - - -## Azure Metrics are blank, text box is malformed, see screenshot - -Nginx Default config `nginx.conf` should have two status_zone directives included. One for server{} block, one for / location block. **This will allow metrics to show up immediately, without the user having to find, understand, and configure status_zones in their nginx.conf file, or included files in /etc/nginx/conf.d.** - -```nginx - -user nginx; -worker_processes auto; -worker_rlimit_nofile 8192; -pid /run/nginx/nginx.pid; - -events { - worker_connections 4000; -} - -error_log /var/log/nginx/error.log error; - -http { - access_log off; - server_tokens ""; - server { - listen 80 default_server; - status_zone default; # Add something here - server_name localhost; - location / { - status_zone /; # Add something here - # Points to a directory with a basic html index file with - # a "Welcome to NGINX as a Service for Azure!" page - root /var/www; - index index.html; - } - } -} - -``` - -There should be a step by step config guide for getting the Metrics to show up, and create a basic Dashboard for Nginx, including the Prerequisites. - -## Nginx Standards and Best Practice Violations/Issues - -The Nginx default HTML folder/files are missing. This should be included, `/usr/share/nginx/html`, with all the Nginx Error Pages, and other Nginx primitives. Consult a new installation of NginxPlus-R3x to match files. - -The usage of the `/var/www` folder is an Apache/Microsoft standard, not an Nginx standard. It should be replaced with `/usr/share/nginx/html` for Nginx users. - -The usage of the `/var/cache` folder for caching content is inconsistent with Nginx standards and docs. Most Nginx documentation for caching refers to the `/data/nginx/cache` folder location, and should be changed for Nginx users. - -Missing standard nginx.conf `include` directive, for including files in /etc/nginx/conf.d folder. - -## NGINX configuration issues - -Upload Config Package overwrites the existing nginx.conf, this is a terrible idea. Config package upload should be modified to only allow uploads to the /etc/nginx/conf.d folder, the Nginx standard location for http config files. Perhaps also allow uploads to `/etc/nginx/stream`, the Nginx standard for L4 config files. Even better, allow uploads to a dedicated folder that won't overwrite standard Nginx folders/files, but the user would have to manually copy/paste to move them into the correct folder. Lots of room for discussion and improvement here. - -## Caching - -From the docs: NGINXaaS for Azure only supports caching to /var/cache/nginx. This is because data at /var/cache/nginx will be stored in a separate Temporary Disk. The size of the temporary disk is 4GB. - -This is too small, and there should be an option to use other Azure storage options besides a Temporary disk. - -Caching Configuration example is incomplete. It only set up the cache_path location: - -http { - # ... - proxy_cache_path /var/cache/nginx keys_zone=mycache:10m; -} - -It is `missing` all the other parameters needed for caching to work. A link to Nginx Content Caching is provided, but that is not very helpful. - -A complete Caching config example should be provided, perhaps with an include file, pre-configured ? - -```nginx - -http { - ... - - proxy_cache_path /data/nginx/cache levels=1:2 keys_zone=mycache:10m max_size=500m use_temp_path=off; - - ... - - server { - ... - server_name localhost; - location /images { - ... - proxy_cache mycache; # Use the cache - proxy_cache_key "$host$request_uri$cookie_user"; # Cache Key - proxy_cache_min_uses 2; # Cache after 2 reqs - proxy_cache_valid 200 30m; # Cache for 30m - proxy_cache_valid 404 1m; - - # Required caching headers - proxy_ignore_headers X-Accel-Expires Expires Cache-Control Set-Cookie; - add_header Cache-Control "public"; - add_header X-Cache-Status $upstream_cache_status; # Add Cache status header - - - } - } -} - -``` - -## Default 'includes' Directive is missing - -## Can't see the Nginx Upstreams - -Without the Plus realtime dash board, there is no way to know if the Upstreams defined are correct or working, because - -## No access to Nginx Access or Error logs - -There is no realtime access to either the Error or Access Logs from Nginx. It makes it virtually impossible to "see what's going on" with Nginx without these logs. - -Using the Azure Logging services is complicated, you can't see the original Access or Error logs. An Azure Logging Workspace that shows the error and Access log should be included when the user Deploys N4A. It should be there by default. The lack of this feature will frustrate a large number of Nginx users, especially if they are new to Azure Monitoring / Logging Workspaces. - - -## Nginx Keepalive for HTTP1.1 settings should be included, missing Best Practice. - -```nginx -# Default is HTTP/1, keepalive is only enabled in HTTP/1.1 -proxy_http_version 1.1; - -# Remove the Connection header if the client sends it, -# it could be "close" to close a keepalive connection -proxy_set_header Connection ""; - -# Host request header field, or the server name matching a request -proxy_set_header Host $host; - -``` - -## Nginx Azure Monitor - can't used Saved Dashboards. - -If you create and save a dashboard, it does not work. Looks like you have Share a Dashboard. - -I can see the name in the list, but Azure Monitor does not let me "load it and use it". It starts with a new, blank dashboard instead. If you refresh the browser page, all customizations are lost and you start at the beginning. - -nginx requests and responses, plus.http.status.4xx are reporting incorrectly. looks like 2xx and 4xx metrics are swapped! - -Unique Server, Location, and Upstream block metrics are not available, everything is aggregated in to a Total, no metrics with fine grain resolution. - -Very difficult to even see the Upstream IP addresses, this is critical for a Proxy configuration. - - -*********** - -## Engineering Areas to investigate - -Adam - AzureAD/DNS/Grafana - ingress-demo container update -Chris - Plus LB, AZcompute, Cafe Demo, aks/nic/cafe -Shouvik - AZ monitor, KeyVault, new repo -Steve - Demo app, Redis \ No newline at end of file diff --git a/ca-notes/R30Plus-dashboard.html b/ca-notes/R30Plus-dashboard.html deleted file mode 100644 index c80db68..0000000 --- a/ca-notes/R30Plus-dashboard.html +++ /dev/null @@ -1,1928 +0,0 @@ -
-- Introduction to `xx` -- Build an `yyy` Nginx configuration -- Test access to your lab enviroment with Curl and Chrome -- Investigate `zzz` +## Prerequisites +In this Lab0, the requirements for both the Student and the Azure environment will be described. *It is imperative that you have the appropriate computer, tools, and Azure access to successfully complete the Workshop.* -## Pre-Requisites + -- You must have `aaaa` installed and running -- You must have `bbbbb` installed -- See `Lab0` for instructions on setting up your system for this Workshop -- Familiarity with basic Linux commands and commandline tools -- Familiarity with basic Docker concepts and commands -- Familiarity with basic HTTP protocol +NGINXaaS for Azure | NGINX Plus | Kubernetes | Docker | Redis +:-------------------------:|:-------------------------:|:-------------------------:|:-------------------------:|:-------------------------: + |  |  |  | 
-### Lab exercise 1 +### Student Hardware/Software/Azure Requirements -
+ +## Required Skills -### Lab exercise 3 +- Nginx for Azure NGINXperts Workshop has minimum REQUIRED Nginx Skills: Students must be familiar with Nginx operation, configurations, and concepts for HTTP traffic. +- The NGINXperts Basics Workshop is HIGHLY recommended, students should have taken this workshop prior. +- The NGINXperts Plus Ingress Controller workshop is also HIGHLY recommended, students should have taken this workshop prior. +- Azure admin skills, previous training from Microsoft Learn is HIGHLY recommended. +- Recommended: TLS, DNS, HTTP caching, Grafana, Redis -
-### << more exercises/steps>> +[NGINXperts Basics Workshop](https://github.com/nginxinc/nginx-basics-workshops) -
@@ -71,7 +73,8 @@ By the end of the lab you will be able to: - Chris Akker - Solutions Architect - Community and Alliances @ F5, Inc. - Shouvik Dutta - Solutions Architect - Community and Alliances @ F5, Inc. - Adam Currier - Solutions Architect - Community and Alliances @ F5, Inc. +- Steve Wagner - Solutions Architect - Community and Alliances @ F5, Inc. ------------- -Navigate to ([Lab1](../lab1/readme.md) | [LabX](../labX/readme.md)) +Navigate to ([Lab1](../lab1/readme.md) | [LabGuide](../readme.md)) diff --git a/labs/lab1/logs.json b/labs/lab1/logs.json deleted file mode 100644 index b4dffc0..0000000 --- a/labs/lab1/logs.json +++ /dev/null @@ -1 +0,0 @@ -[{category:NginxLogs,enabled:true,retention-policy:{enabled:false,days:0}}] \ No newline at end of file diff --git a/labs/lab1/media/enable_system_identity.png b/labs/lab1/media/enable_system_identity.png deleted file mode 100644 index 066d873..0000000 Binary files a/labs/lab1/media/enable_system_identity.png and /dev/null differ diff --git a/labs/lab1/media/enable_system_identity_success.png b/labs/lab1/media/enable_system_identity_success.png deleted file mode 100644 index 3c8184a..0000000 Binary files a/labs/lab1/media/enable_system_identity_success.png and /dev/null differ diff --git a/labs/lab1/media/lab1_azure-network.png b/labs/lab1/media/lab1_azure-network.png new file mode 100644 index 0000000..4f44498 Binary files /dev/null and b/labs/lab1/media/lab1_azure-network.png differ diff --git a/labs/lab1/media/lab1_azure-subnets.png b/labs/lab1/media/lab1_azure-subnets.png new file mode 100644 index 0000000..44add21 Binary files /dev/null and b/labs/lab1/media/lab1_azure-subnets.png differ diff --git a/labs/lab1/media/lab1_copy_ip_address.png b/labs/lab1/media/lab1_copy_ip_address.png new file mode 100644 index 0000000..37d32a2 Binary files /dev/null and b/labs/lab1/media/lab1_copy_ip_address.png differ diff --git a/labs/lab1/media/lab1_diagram.png b/labs/lab1/media/lab1_diagram.png new file mode 100644 index 0000000..80ad694 Binary files /dev/null and b/labs/lab1/media/lab1_diagram.png differ diff --git a/labs/lab1/media/n4a_index_page.png b/labs/lab1/media/lab1_n4a_index_page.png similarity index 100% rename from labs/lab1/media/n4a_index_page.png rename to labs/lab1/media/lab1_n4a_index_page.png diff --git a/labs/lab1/media/nginx_conf_editor.png b/labs/lab1/media/lab1_nginx_conf_editor.png similarity index 100% rename from labs/lab1/media/nginx_conf_editor.png rename to labs/lab1/media/lab1_nginx_conf_editor.png diff --git a/labs/lab1/media/nginx_conf_populate.png b/labs/lab1/media/lab1_nginx_conf_populate.png similarity index 100% rename from labs/lab1/media/nginx_conf_populate.png rename to labs/lab1/media/lab1_nginx_conf_populate.png diff --git a/labs/lab1/media/nginx_conf_submit_success.png b/labs/lab1/media/lab1_nginx_conf_submit_success.png similarity index 100% rename from labs/lab1/media/nginx_conf_submit_success.png rename to labs/lab1/media/lab1_nginx_conf_submit_success.png diff --git a/labs/lab1/media/portal_n4a_home.png b/labs/lab1/media/lab1_portal_n4a_home.png similarity index 100% rename from labs/lab1/media/portal_n4a_home.png rename to labs/lab1/media/lab1_portal_n4a_home.png diff --git a/labs/lab1/media/portal_rg_home.png b/labs/lab1/media/lab1_portal_rg_home.png similarity index 100% rename from labs/lab1/media/portal_rg_home.png rename to labs/lab1/media/lab1_portal_rg_home.png diff --git a/labs/lab1/media/nginx-azure-icon.png b/labs/lab1/media/nginx-azure-icon.png new file mode 100644 index 0000000..70ab132 Binary files /dev/null and b/labs/lab1/media/nginx-azure-icon.png differ diff --git a/labs/lab1/readme.md b/labs/lab1/readme.md index 345820c..3488ab2 100644 --- a/labs/lab1/readme.md +++ b/labs/lab1/readme.md @@ -1,28 +1,30 @@ -# Azure Network Build and Nginx for Azure Overview +# Azure Network Build and NGINX for Azure Overview ## Introduction -In this lab, you will be adding and configuring the Azure Networking components needed for this workshop. This will require a few network resources, and a Network Security Group to allow incoming traffic to your Nginx for Azure workshop resources. Then you will explore the Nginx for Azure product, as a quick Overview of what it is and how to deploy it. +In this lab, you will be adding and configuring the Azure components needed for this workshop. This will require a few network resources, a Network Security Group and a Public IP to allow incoming traffic to your NGINX for Azure workshop resource. You will also deploy NGINX for Azure resource. Then you will explore the Nginx for Azure product, as a quick Overview of what it is and how to deploy it. -< Lab specific Images here, in the /media sub-folder > +
-NGINX aaS | Docker -:-------------------------:|:-------------------------: - | +NGINX aaS for Azure | +:-------------------------:| +| +
+ ## Learning Objectives By the end of the lab you will be able to: -- Setup your Azure resource group for this workshop +- Setup your Azure Resource Group for this workshop - Setup your Azure Virtual Network, Subnets and Network Security Group for inbound traffic -- Create Public IP and user assigned managed identity to access NGINX for Azure +- Create a Public IP and user assigned managed identity to access NGINX for Azure - Deploy an Nginx for Azure resource - Explore Nginx for Azure - Create an initial Nginx configuration for testing -- Create Log Analytics workspace to collect NGINX error and access logs from NGINX for azure +- Create Log Analytics workspace to collect NGINX error and access logs from Nginx for Azure -## Pre-Requisites +## Prerequisites - You must have an Azure account - You must have the Azure CLI software installed on your local system @@ -33,7 +35,11 @@ By the end of the lab you will be able to:
-### Setup your Azure resource group for this workshop + + +
+ +### Setup your Azure Resource group for this workshop 1. In your local machine open terminal and make sure you have Azure Command Line Interface (CLI) installed by running below command. @@ -45,7 +51,9 @@ By the end of the lab you will be able to: 1. Create a new Azure Resource Group called `
+ ### Setup your Azure Virtual Network, Subnets and Network Security Group + + +You will create an Azure Vnet for this Workshop. Inside of this Vnet are 4 different subnets, representing various backend networks for Azure resources like Nginx for Azure, VMs, and Kubernetes clusters. + +Name | Subnet | Assignment +:---:|:----:|:---: +n4a-subnet | 172.16.1.0/24 | Nginx for Azure +vm-subnet | 172.16.2.0/24 | Virtual Machines +aks1-subnet | 172.16.10.0/23 | AKS Cluster #1 +aks2-subnet | 172.16.20.0/23 | AKS Cluster #2 + +
+ 1. Create a virtual network (vnet) named `n4a-vnet` using below command. ```bash ## Set environment variables - MY_RESOURCEGROUP=s.dutta-workshop - MY_PUBLICIP=$(curl -4 ifconfig.co) + export MY_RESOURCEGROUP=s.dutta-workshop + export MY_PUBLICIP=$(curl ipinfo.io/ip) ``` ```bash @@ -103,7 +126,9 @@ By the end of the lab you will be able to: } ``` -1. Create a network security group(NSG) named `n4a-nsg` using below command. + > **NOTE:** Within the output json you should have a `"provisioningState": "Succeeded"` field which validates the command successfully provisioned the resource. + +2. Create a network security group (NSG) named `n4a-nsg` using below command. ```bash az network nsg create \ @@ -111,7 +136,7 @@ By the end of the lab you will be able to: --name n4a-nsg ``` -1. Add two NSG rules to allow any traffic on port 80 and 443 from your system's public IP. Run below command to create the two rules. +3. Add two NSG rules to allow any traffic on port 80 and 443 from your system's public IP. Run below command to create the two rules. ```bash ## Rule 1 for HTTP traffic @@ -199,7 +224,9 @@ By the end of the lab you will be able to: } ``` -1. Create a subnet that you will use with NGINX for Azure resource. You will also attach the NSG that you just created to this subnet. + > **NOTE:** Within the output json you should have a `"provisioningState": "Succeeded"` field which validates the command successfully provisioned the resource. + +4. Create a subnet that you will use with NGINX for Azure resource. You will also attach the NSG that you just created to this subnet. ```bash az network vnet subnet create \ @@ -244,9 +271,21 @@ By the end of the lab you will be able to: } ``` -1. In similar fashion create two more subnets that would be used with AKS cluster in later labs. + > **NOTE:** Within the output json you should have a `"provisioningState": "Succeeded"` field which validates the command successfully provisioned the resource. + +5. In similar fashion create three more subnets that would be used with docker virtual machines and AKS clusters in later labs. + + ```bash + # VM Subnet + az network vnet subnet create \ + --resource-group $MY_RESOURCEGROUP \ + --name vm-subnet \ + --vnet-name n4a-vnet \ + --address-prefixes 172.16.2.0/24 + ``` ```bash + # AKS1 Subnet az network vnet subnet create \ --resource-group $MY_RESOURCEGROUP \ --name aks1-subnet \ @@ -255,6 +294,7 @@ By the end of the lab you will be able to: ``` ```bash + # AKS2 Subnet az network vnet subnet create \ --resource-group $MY_RESOURCEGROUP \ --name aks2-subnet \ @@ -262,6 +302,12 @@ By the end of the lab you will be able to: --address-prefixes 172.16.20.0/23 ``` +Your completed Vnet/Subnets should look similar to this: + + + +
+ ### Create Public IP and user assigned managed identity to access NGINX for Azure 1. Create a Public IP that you will attach to NGINX for Azure. You will use this public IP to access NGINX for Azure from outside the Azure network. Use below command to create a new Public IP. @@ -302,7 +348,9 @@ By the end of the lab you will be able to: } ``` -1. Create a user assigned managed identity that would be tied to the NGINX for Azure resource. This managed identity would be used to read certificates and keys from Azure keyvault in later labs. + > **NOTE:** Within the output json you should have a `"provisioningState": "Succeeded"` field which validates the command successfully provisioned the resource. + +1. Create a user assigned managed identity that would be tied to the NGINX for Azure resource. This managed identity would be used to read certificates and keys from Azure key vault in later labs. ```bash az identity create \ @@ -326,24 +374,27 @@ By the end of the lab you will be able to: } ``` -### Deploy an Nginx for Azure resource +
-1. Once all the previous Azure resources have been created, you will then create the NGINX for Azure resource using below commands (This would take couple of minutes to finish) +## Deploy an Nginx for Azure Resource + +
+ +1. Once all the previous Azure resources have been created, you will then create the NGINX for Azure resource using below commands (This will take couple of minutes to finish): ```bash ## Set environment variables - MY_RESOURCEGROUP=s.dutta-workshop - MY_SUBSCRIPTIONID=$(az account show --query id -o tsv) + export MY_RESOURCEGROUP=s.dutta-workshop + export MY_SUBSCRIPTIONID=$(az account show --query id -o tsv) ``` ```bash az nginx deployment create \ --resource-group $MY_RESOURCEGROUP \ --name nginx4a \ - --location centralus \ --sku name="standard_Monthly" \ --network-profile front-end-ip-configuration="{public-ip-addresses:[{id:/subscriptions/$MY_SUBSCRIPTIONID/resourceGroups/$MY_RESOURCEGROUP/providers/Microsoft.Network/publicIPAddresses/n4a-publicIP}]}" network-interface-configuration="{subnet-id:/subscriptions/$MY_SUBSCRIPTIONID/resourceGroups/$MY_RESOURCEGROUP/providers/Microsoft.Network/virtualNetworks/n4a-vnet/subnets/n4a-subnet}" \ - --identity="{type:UserAssigned,userAssignedIdentities:{/subscriptions/$MY_SUBSCRIPTIONID/resourceGroups/$MY_RESOURCEGROUP/providers/Microsoft.ManagedIdentity/userAssignedIdentities/n4a-useridentity:{}}}" + --identity="{type:'SystemAssigned, UserAssigned',userAssignedIdentities:{/subscriptions/$MY_SUBSCRIPTIONID/resourceGroups/$MY_RESOURCEGROUP/providers/Microsoft.ManagedIdentity/userAssignedIdentities/n4a-useridentity:{}}}" ``` ```bash @@ -351,7 +402,9 @@ By the end of the lab you will be able to: { "id": "/subscriptions/
### Create Log Analytics workspace to collect NGINX error and Access logs from NGINX for azure In this section you will create a Log Analytics resource that would collect Nginx logs from your Nginx for Azure resource. As this resource takes time to get provisioned and attached to NGINX for Azure resource, you are building it up here. -1. Within the NGINX for Azure resource (nginx4a), navigate to managed identity section by clicking on `Identity` from the left menu. Within this section, inside the `System assigned` tab, enable system managed identity by changing the status to `on`. Click on `Save` to save your changes. Press `Yes` for the "Enable system assigned managed identity" prompt. -  - -2. If you open up the Notifications pane, you should see a success status as shown below. -  - -3. Now open a terminal and create a Log Analytics workspace resource that you will attach to NGINX for Azure using Azure CLI. This resource would be used to capture and store NGINX error and access logs. Use below command to create this resource. - - ```bash - ## Set environment variables - MY_RESOURCEGROUP=s.dutta-workshop - ``` +1. Create a Log Analytics workspace resource that you will attach to NGINX for Azure using Azure CLI. This resource would be used to capture and store NGINX error and access logs. Use below command to create this resource. ```bash az monitor log-analytics workspace create \ @@ -453,7 +473,39 @@ In this section you will create a Log Analytics resource that would collect Ngin --name n4a-loganalytics ``` -4. Next you will update your NGINX for Azure resource to enable sending metrics to Azure monitor by setting the `--enable-diagnostics` flag to `true` using below command. + ```bash + { + "createdDate": "2024-04-17T20:42:48.2028783Z", + "customerId": "xxxx-xxxx-xxxx-xxxx-xxxx", + "etag": "\"98028759-0000-0500-0000-662fc5330000\"", + "features": { + "enableLogAccessUsingOnlyResourcePermissions": true + }, + "id": "/subscriptions/
+ +### Explore Nginx for Azure + +
+ +NGINX as a Service for Azure is a service offering that is tightly integrated into Microsoft Azure public cloud and its ecosystem, making applications fast, efficient, and reliable with full lifecycle management of advanced NGINX traffic services. NGINXaaS for Azure is available in the Azure Marketplace. + +NGINXaaS for Azure is powered by NGINX Plus, which extends NGINX Open Source with advanced functionality and provides customers with a complete application delivery solution. Initial use cases covered by NGINXaaS include L4 TCP and L7 HTTP load balancing and reverse proxy which can be managed through various Azure management tools. NGINXaaS allows you to provision distinct deployments as per your business or technical requirements. + +In this section you will be looking at NGINX for Azure resource that you created within Azure portal. + +1. Open Azure portal within your browser and then open your resource group. + +  + +2. Click on your NGINX for Azure resource (nginx4a) which should open the Overview section of your resource. You can see useful information like Status, NGINX for Azure resource's public IP, which Nginx version is running, which vnet/subnet it is using, etc. + +  + +3. From the left pane click on `Settings > NGINX Configuration`. As you are opening this resource for first time and you do not have any configuration present, Azure will prompt you to "Get started with a Configuration example". Click on `Populate now` button to start with a sample configuration example. + +  + +4. Once you click on the `Populate now` button you will see the configuration editor section has been populated with `nginx.conf` and an `index.html` page. Click on the `Submit` button to deploy this sample config files to the NGINX for Azure resource. + +  + +5. Once you have submitted the configuration, you can watch its progress in the notification tab present in right top corner. Intially status would be "Updating NGINX configuration" which would change to "Updated NGINX configuration successfully". + +  + +6. Navigate back to Overview section and copy the public IP address of NGINX for Azure resource. + +  + +7. In a new browser window, paste the public IP into the address bar. You will notice the sample index page gets rendered into your browser. + +  + +8. This completes the validation of all the resources that you created using Azure CLI. In the upcoming labs you would be modifying the configuration files and exploring various features of NGINX for Azure resources.
@@ -522,4 +660,4 @@ In this section you will create a Log Analytics resource that would collect Ngin ------------- -Navigate to ([Lab2](../lab2/readme.md) | [LabX](../labX/readme.md)) +Navigate to ([Lab2](../lab2/readme.md) | [LabGuide](../readme.md)) diff --git a/labs/lab10/N4A-Dashboard.json b/labs/lab10/N4A-Dashboard.json new file mode 100644 index 0000000..0cf0a46 --- /dev/null +++ b/labs/lab10/N4A-Dashboard.json @@ -0,0 +1,3511 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "grafana", + "uid": "-- Grafana --" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 19, + "links": [], + "liveNow": false, + "panels": [ + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 0 + }, + "id": 10, + "panels": [], + "title": "NGINXaaS", + "type": "row" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 0, + "y": 1 + }, + "id": 13, + "options": { + "minVizHeight": 75, + "minVizWidth": 75, + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showThresholdLabels": false, + "showThresholdMarkers": true, + "sizing": "auto" + }, + "pluginVersion": "10.4.1", + "targets": [ + { + "azureMonitor": { + "aggregation": "Count", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "nginxaas statistics", + "dimensionFilters": [], + "metricName": "ncu.provisioned", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + }, + { + "azureMonitor": { + "aggregation": "Count", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "nginxaas statistics", + "dimensionFilters": [], + "metricName": "ncu.requested", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "B" + }, + { + "azureMonitor": { + "aggregation": "Count", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINXaaS Statistics", + "dimensionFilters": [], + "metricName": "ncu.consumed", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "C" + } + ], + "title": "NCU Stats", + "type": "gauge" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 5, + "w": 12, + "x": 12, + "y": 1 + }, + "id": 9, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "10.4.1", + "targets": [ + { + "azureMonitor": { + "aggregation": "Count", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Connections Statistics", + "dimensionFilters": [], + "metricName": "nginx.conn.accepted", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "C" + }, + { + "azureMonitor": { + "aggregation": "Count", + "alias": "", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Connections Statistics", + "dimensionFilters": [], + "metricName": "nginx.conn.dropped", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto", + "top": "" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "B" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Connections Statistics", + "dimensionFilters": [], + "metricName": "nginx.conn.active", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Connections Statistics", + "dimensionFilters": [], + "metricName": "nginx.conn.idle", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "D" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Connections Statistics", + "dimensionFilters": [], + "metricName": "nginx.conn.current", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "E" + } + ], + "title": "NGINX connections statistics", + "type": "stat" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 4, + "w": 12, + "x": 12, + "y": 6 + }, + "id": 14, + "options": { + "colorMode": "background", + "graphMode": "none", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "10.4.1", + "targets": [ + { + "azureMonitor": { + "aggregation": "Count", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Worker Statistics", + "dimensionFilters": [], + "metricName": "plus.worker.conn.accepted", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + }, + { + "azureMonitor": { + "aggregation": "Count", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Worker Statistics", + "dimensionFilters": [], + "metricName": "plus.worker.conn.dropped", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "B" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Worker Statistics", + "dimensionFilters": [], + "metricName": "plus.worker.conn.active", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "C" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Worker Statistics", + "dimensionFilters": [], + "metricName": "plus.worker.conn.idle", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "D" + }, + { + "azureMonitor": { + "aggregation": "Count", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Worker Statistics", + "dimensionFilters": [], + "metricName": "plus.worker.http.request.total", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "E" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Worker Statistics", + "dimensionFilters": [], + "metricName": "plus.worker.http.request.current", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "F" + } + ], + "title": "NGINX worker statistics", + "type": "stat" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 3, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 10, + "w": 12, + "x": 0, + "y": 10 + }, + "id": 5, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Upstream Statistics", + "dimensionFilters": [ + { + "dimension": "upstream", + "filters": [], + "operator": "eq" + } + ], + "metricName": "plus.http.upstream.peers.response.time", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "NGINX.NGINXPLUS/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + } + ], + "title": "Upstream Response Time", + "type": "timeseries" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 10, + "w": 12, + "x": 12, + "y": 10 + }, + "id": 2, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "pluginVersion": "10.4.1", + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Requests and Response Statistics", + "dimensionFilters": [], + "metricName": "plus.http.status.2xx", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "NGINX.NGINXPLUS/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Requests and Response Statistics", + "dimensionFilters": [], + "metricName": "plus.http.status.3xx", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "NGINX.NGINXPLUS/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "C" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Requests and Response Statistics", + "dimensionFilters": [], + "metricName": "plus.http.status.4xx", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "NGINX.NGINXPLUS/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "B" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Requests and Response Statistics", + "dimensionFilters": [], + "metricName": "plus.http.status.5xx", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "NGINX.NGINXPLUS/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "D" + } + ], + "title": "HTTP Metrics", + "type": "timeseries" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 20 + }, + "id": 15, + "options": { + "colorMode": "value", + "graphMode": "none", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "10.4.1", + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "nginx system statistics", + "dimensionFilters": [], + "metricName": "system.cpu", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "nginx system statistics", + "dimensionFilters": [], + "metricName": "system.interface.bytes_rcvd", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "B" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "nginx system statistics", + "dimensionFilters": [], + "metricName": "system.interface.bytes_sent", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "C" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "nginx system statistics", + "dimensionFilters": [], + "metricName": "system.interface.packets_rcvd", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "D" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "nginx system statistics", + "dimensionFilters": [], + "metricName": "system.interface.packets_sent", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "E" + }, + { + "azureMonitor": { + "aggregation": "Total", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "nginx system statistics", + "dimensionFilters": [], + "metricName": "system.interface.total_bytes", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "F" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "nginx system statistics", + "dimensionFilters": [], + "metricName": "system.interface.egress_throughput", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Nginx.NginxPlus/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "G" + } + ], + "title": "NGINX system statistics", + "type": "stat" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 20 + }, + "id": 4, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "azureMonitor": { + "aggregation": "Count", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX SSL Statistics", + "dimensionFilters": [ + { + "dimension": "server_zone", + "filters": [], + "operator": "eq" + } + ], + "metricName": "plus.http.ssl.handshakes", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "NGINX.NGINXPLUS/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + }, + { + "azureMonitor": { + "aggregation": "Count", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX SSL Statistics", + "dimensionFilters": [ + { + "dimension": "server_zone", + "filters": [], + "operator": "eq" + } + ], + "metricName": "plus.http.ssl.session.reuses", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "NGINX.NGINXPLUS/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "B" + }, + { + "azureMonitor": { + "aggregation": "Count", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX SSL Statistics", + "dimensionFilters": [ + { + "dimension": "server_zone", + "filters": [], + "operator": "eq" + } + ], + "metricName": "plus.http.ssl.handshakes.failed", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "NGINX.NGINXPLUS/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "C" + } + ], + "title": "SSL Metrics", + "type": "timeseries" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 3, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 10, + "w": 12, + "x": 0, + "y": 28 + }, + "id": 1, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Cache Statistics", + "dimensionFilters": [ + { + "dimension": "cache_zone", + "filters": [ + "image_cache" + ], + "operator": "eq" + } + ], + "metricName": "plus.cache.hit.ratio", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "NGINX.NGINXPLUS/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + } + ], + "title": "Cache Hit Ratio", + "type": "timeseries" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 3, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 10, + "w": 12, + "x": 12, + "y": 28 + }, + "id": 6, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "azureMonitor": { + "aggregation": "Count", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Upstream Statistics", + "dimensionFilters": [ + { + "dimension": "upstream", + "filters": [], + "operator": "eq" + } + ], + "metricName": "plus.http.upstream.peers.health_checks.checks", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "NGINX.NGINXPLUS/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + }, + { + "azureMonitor": { + "aggregation": "Count", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "customNamespace": "NGINX Upstream Statistics", + "dimensionFilters": [ + { + "dimension": "upstream", + "filters": [], + "operator": "eq" + } + ], + "metricName": "plus.http.upstream.peers.health_checks.unhealthy", + "metricNamespace": "nginx.nginxplus/nginxdeployments", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "NGINX.NGINXPLUS/nginxDeployments", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$MY_RESOURCENAME" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "B" + } + ], + "title": "Health Checks", + "type": "timeseries" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 38 + }, + "id": 11, + "panels": [ + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 2 + }, + "id": 7, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "10.4.1", + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000 + ], + "dimensionFilters": [], + "metricName": "apiserver_current_inflight_requests", + "metricNamespace": "microsoft.containerservice/managedclusters", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.ContainerService/managedClusters", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$AKSCluster1" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + } + ], + "title": "AKS1 Inflight Requests", + "type": "stat" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 2 + }, + "id": 8, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "10.4.1", + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000 + ], + "dimensionFilters": [], + "metricName": "apiserver_current_inflight_requests", + "metricNamespace": "microsoft.containerservice/managedclusters", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.ContainerService/managedClusters", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$AKSCluster2" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + } + ], + "title": "AKS2 Inflight Requests", + "type": "stat" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 4, + "w": 12, + "x": 0, + "y": 10 + }, + "id": 16, + "options": { + "colorMode": "background", + "graphMode": "none", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "10.4.1", + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000 + ], + "dimensionFilters": [], + "metricName": "node_network_in_bytes", + "metricNamespace": "microsoft.containerservice/managedclusters", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.ContainerService/managedClusters", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$AKSCluster1" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "B" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000 + ], + "dimensionFilters": [], + "metricName": "node_network_out_bytes", + "metricNamespace": "microsoft.containerservice/managedclusters", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.ContainerService/managedClusters", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$AKSCluster1" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + } + ], + "title": "AKS 1 Network Stats", + "type": "stat" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 4, + "w": 12, + "x": 12, + "y": 10 + }, + "id": 17, + "options": { + "colorMode": "background", + "graphMode": "none", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "10.4.1", + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000 + ], + "dimensionFilters": [], + "metricName": "node_network_in_bytes", + "metricNamespace": "microsoft.containerservice/managedclusters", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.ContainerService/managedClusters", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$AKSCluster2" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000 + ], + "dimensionFilters": [], + "metricName": "node_network_out_bytes", + "metricNamespace": "microsoft.containerservice/managedclusters", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.ContainerService/managedClusters", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$AKSCluster2" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "B" + } + ], + "title": "AKS 2 Network Stats", + "type": "stat" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "continuous-GrYlRd" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "scheme", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "smooth", + "lineWidth": 3, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 14 + }, + "id": 18, + "options": { + "legend": { + "calcs": [], + "displayMode": "hidden", + "placement": "right", + "showLegend": false + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000 + ], + "dimensionFilters": [], + "metricName": "node_cpu_usage_percentage", + "metricNamespace": "microsoft.containerservice/managedclusters", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.ContainerService/managedClusters", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$AKSCluster1" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + } + ], + "title": "AKS1 CPU Usage", + "type": "timeseries" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "continuous-GrYlRd" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "scheme", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "smooth", + "lineWidth": 3, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 14 + }, + "id": 19, + "options": { + "legend": { + "calcs": [], + "displayMode": "hidden", + "placement": "right", + "showLegend": false + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000 + ], + "dimensionFilters": [], + "metricName": "node_cpu_usage_percentage", + "metricNamespace": "microsoft.containerservice/managedclusters", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.ContainerService/managedClusters", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$AKSCluster2" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + } + ], + "title": "AKS1 CPU Usage", + "type": "timeseries" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "continuous-GrYlRd" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "bars", + "fillOpacity": 90, + "gradientMode": "scheme", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 22 + }, + "id": 20, + "options": { + "legend": { + "calcs": [], + "displayMode": "hidden", + "placement": "right", + "showLegend": false + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000 + ], + "dimensionFilters": [], + "metricName": "node_memory_working_set_percentage", + "metricNamespace": "microsoft.containerservice/managedclusters", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.ContainerService/managedClusters", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$AKSCluster1" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + } + ], + "title": "AKS1 Memory Working Set %", + "type": "timeseries" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "continuous-GrYlRd" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "bars", + "fillOpacity": 90, + "gradientMode": "scheme", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 22 + }, + "id": 21, + "options": { + "legend": { + "calcs": [], + "displayMode": "hidden", + "placement": "right", + "showLegend": false + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000 + ], + "dimensionFilters": [], + "metricName": "node_memory_working_set_percentage", + "metricNamespace": "microsoft.containerservice/managedclusters", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.ContainerService/managedClusters", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$AKSCluster1" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + } + ], + "title": "AKS2 Memory Working Set %", + "type": "timeseries" + } + ], + "title": "K8s Clusters", + "type": "row" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 39 + }, + "id": 22, + "panels": [ + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "continuous-GrYlRd" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "scheme", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "smooth", + "lineWidth": 3, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 3 + }, + "id": 23, + "options": { + "legend": { + "calcs": [], + "displayMode": "hidden", + "placement": "right", + "showLegend": false + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "dimensionFilters": [], + "metricName": "Inbound Flows", + "metricNamespace": "microsoft.compute/virtualmachines", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.Compute/virtualMachines", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$ubuntuvm" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + } + ], + "title": "Ubuntu VM Inbound Flow", + "type": "timeseries" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "continuous-GrYlRd" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "scheme", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "smooth", + "lineWidth": 3, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 3 + }, + "id": 24, + "options": { + "legend": { + "calcs": [], + "displayMode": "hidden", + "placement": "right", + "showLegend": false + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "dimensionFilters": [], + "metricName": "Inbound Flows", + "metricNamespace": "microsoft.compute/virtualmachines", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.Compute/virtualMachines", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$windowsvm" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + } + ], + "title": "Windows VM Inbound Flow", + "type": "timeseries" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "mappings": [], + "thresholds": { + "mode": "percentage", + "steps": [ + { + "color": "green" + }, + { + "color": "orange", + "value": 70 + }, + { + "color": "red", + "value": 85 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 11 + }, + "id": 25, + "options": { + "minVizHeight": 75, + "minVizWidth": 75, + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showThresholdLabels": false, + "showThresholdMarkers": true, + "sizing": "auto" + }, + "pluginVersion": "10.4.1", + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "dimensionFilters": [], + "metricName": "Disk Read Bytes", + "metricNamespace": "microsoft.compute/virtualmachines", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.Compute/virtualMachines", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$ubuntuvm" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + }, + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "dimensionFilters": [], + "metricName": "Disk Write Bytes", + "metricNamespace": "microsoft.compute/virtualmachines", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.Compute/virtualMachines", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$ubuntuvm" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "B" + }, + { + "azureMonitor": { + "aggregation": "Count", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "dimensionFilters": [], + "metricName": "Percentage CPU", + "metricNamespace": "microsoft.compute/virtualmachines", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.Compute/virtualMachines", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$ubuntuvm" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "C" + }, + { + "azureMonitor": { + "aggregation": "Total", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "dimensionFilters": [], + "metricName": "Available Memory Bytes", + "metricNamespace": "microsoft.compute/virtualmachines", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.Compute/virtualMachines", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$ubuntuvm" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "D" + } + ], + "title": "Ubuntu System Stats", + "type": "gauge" + }, + { + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "fieldConfig": { + "defaults": { + "mappings": [], + "thresholds": { + "mode": "percentage", + "steps": [ + { + "color": "green" + }, + { + "color": "orange", + "value": 70 + }, + { + "color": "red", + "value": 85 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 11 + }, + "id": 26, + "options": { + "minVizHeight": 75, + "minVizWidth": 75, + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showThresholdLabels": false, + "showThresholdMarkers": true, + "sizing": "auto" + }, + "pluginVersion": "10.4.1", + "targets": [ + { + "azureMonitor": { + "aggregation": "Average", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "dimensionFilters": [], + "metricName": "Disk Read Bytes", + "metricNamespace": "microsoft.compute/virtualmachines", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.Compute/virtualMachines", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$windowsvm" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "queryType": "Azure Monitor", + "refId": "A" + }, + { + "azureMonitor": { + "aggregation": "Total", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "dimensionFilters": [], + "metricName": "Disk Write Bytes", + "metricNamespace": "microsoft.compute/virtualmachines", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.Compute/virtualMachines", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$windowsvm" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "B" + }, + { + "azureMonitor": { + "aggregation": "Count", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "dimensionFilters": [], + "metricName": "Percentage CPU", + "metricNamespace": "microsoft.compute/virtualmachines", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.Compute/virtualMachines", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$windowsvm" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "C" + }, + { + "azureMonitor": { + "aggregation": "Total", + "allowedTimeGrainsMs": [ + 60000, + 300000, + 900000, + 1800000, + 3600000, + 21600000, + 43200000, + 86400000 + ], + "dimensionFilters": [], + "metricName": "Available Memory Bytes", + "metricNamespace": "microsoft.compute/virtualmachines", + "region": "$MY_LOCATION", + "resources": [ + { + "metricNamespace": "Microsoft.Compute/virtualMachines", + "region": "$MY_LOCATION", + "resourceGroup": "$MY_RESOURCEGROUP", + "resourceName": "$windowsvm" + } + ], + "timeGrain": "auto" + }, + "datasource": { + "type": "grafana-azure-monitor-datasource", + "uid": "azure-monitor-oob" + }, + "hide": false, + "queryType": "Azure Monitor", + "refId": "D" + } + ], + "title": "Windows System Stats", + "type": "gauge" + } + ], + "title": "Virtual Machines", + "type": "row" + } + ], + "refresh": "5s", + "schemaVersion": 39, + "tags": [], + "templating": { + "list": [ + { + "current": { + "selected": false, + "text": "eastus", + "value": "eastus" + }, + "hide": 0, + "name": "MY_LOCATION", + "options": [], + "query": "eastus", + "skipUrlSync": false, + "type": "textbox" + }, + { + "current": { + "selected": false, + "text": "a.currier-workshop", + "value": "a.currier-workshop" + }, + "hide": 0, + "name": "MY_RESOURCEGROUP", + "options": [], + "query": "a.currier-workshop", + "skipUrlSync": false, + "type": "textbox" + }, + { + "current": { + "selected": false, + "text": "nginx4a", + "value": "nginx4a" + }, + "hide": 0, + "name": "MY_RESOURCENAME", + "options": [], + "query": "nginx4a", + "skipUrlSync": false, + "type": "textbox" + }, + { + "current": { + "selected": false, + "text": "n4a-aks1", + "value": "n4a-aks1" + }, + "hide": 0, + "label": "AKS Cluster 1", + "name": "AKSCluster1", + "options": [], + "query": "n4a-aks1", + "skipUrlSync": false, + "type": "textbox" + }, + { + "current": { + "selected": false, + "text": "n4a-aks2", + "value": "n4a-aks2" + }, + "hide": 0, + "label": "AKS Cluster 2", + "name": "AKSCluster2", + "options": [], + "query": "n4a-aks2", + "skipUrlSync": false, + "type": "textbox" + }, + { + "current": { + "selected": false, + "text": "windowsvm", + "value": "windowsvm" + }, + "hide": 0, + "label": "Windows VM", + "name": "windowsvm", + "options": [ + { + "selected": true, + "text": "windowsvm", + "value": "windowsvm" + } + ], + "query": "windowsvm", + "skipUrlSync": false, + "type": "textbox" + }, + { + "current": { + "selected": false, + "text": "ubuntuvm", + "value": "ubuntuvm" + }, + "hide": 0, + "label": "Ubuntu VM", + "name": "ubuntuvm", + "options": [], + "query": "ubuntuvm", + "skipUrlSync": false, + "type": "textbox" + } + ] + }, + "time": { + "from": "now-2d", + "to": "now" + }, + "timepicker": { + "hidden": false + }, + "timezone": "", + "title": "Nginx 4 Azure Workshop", + "uid": "b70b2c11-a815-4506-a10a-3f15b6970797", + "version": 20, + "weekStart": "" +} + diff --git a/labs/lab10/media/EntraID-sign_in.png b/labs/lab10/media/EntraID-sign_in.png new file mode 100644 index 0000000..690776e Binary files /dev/null and b/labs/lab10/media/EntraID-sign_in.png differ diff --git a/labs/lab10/media/NGINXaaS-icon.png b/labs/lab10/media/NGINXaaS-icon.png new file mode 100644 index 0000000..1998d39 Binary files /dev/null and b/labs/lab10/media/NGINXaaS-icon.png differ diff --git a/labs/lab10/media/grafana-dashboards-import.png b/labs/lab10/media/grafana-dashboards-import.png new file mode 100644 index 0000000..f9e60f3 Binary files /dev/null and b/labs/lab10/media/grafana-dashboards-import.png differ diff --git a/labs/lab10/media/grafana-dashboards-import2.png b/labs/lab10/media/grafana-dashboards-import2.png new file mode 100644 index 0000000..0272b64 Binary files /dev/null and b/labs/lab10/media/grafana-dashboards-import2.png differ diff --git a/labs/lab10/media/grafana-dashboards-json.png b/labs/lab10/media/grafana-dashboards-json.png new file mode 100644 index 0000000..994ba83 Binary files /dev/null and b/labs/lab10/media/grafana-dashboards-json.png differ diff --git a/labs/lab10/media/grafana-dashboards-k8s-vm.png b/labs/lab10/media/grafana-dashboards-k8s-vm.png new file mode 100644 index 0000000..cde2cb0 Binary files /dev/null and b/labs/lab10/media/grafana-dashboards-k8s-vm.png differ diff --git a/labs/lab10/media/grafana-dashboards-n4a.png b/labs/lab10/media/grafana-dashboards-n4a.png new file mode 100644 index 0000000..ea01b2a Binary files /dev/null and b/labs/lab10/media/grafana-dashboards-n4a.png differ diff --git a/labs/lab10/media/grafana-dashboards-new.png b/labs/lab10/media/grafana-dashboards-new.png new file mode 100644 index 0000000..8f03448 Binary files /dev/null and b/labs/lab10/media/grafana-dashboards-new.png differ diff --git a/labs/lab10/media/grafana-dashboards.png b/labs/lab10/media/grafana-dashboards.png new file mode 100644 index 0000000..4e1f63e Binary files /dev/null and b/labs/lab10/media/grafana-dashboards.png differ diff --git a/labs/lab10/media/grafana-icon.png b/labs/lab10/media/grafana-icon.png new file mode 100644 index 0000000..49a13fb Binary files /dev/null and b/labs/lab10/media/grafana-icon.png differ diff --git a/labs/lab10/media/grafana-landing-page.png b/labs/lab10/media/grafana-landing-page.png new file mode 100644 index 0000000..3ac7bb8 Binary files /dev/null and b/labs/lab10/media/grafana-landing-page.png differ diff --git a/labs/lab10/media/grafana-variables.png b/labs/lab10/media/grafana-variables.png new file mode 100644 index 0000000..8a3c81d Binary files /dev/null and b/labs/lab10/media/grafana-variables.png differ diff --git a/labs/lab10/media/managed-grafana.png b/labs/lab10/media/managed-grafana.png new file mode 100644 index 0000000..4cd2781 Binary files /dev/null and b/labs/lab10/media/managed-grafana.png differ diff --git a/labs/lab10/readme.md b/labs/lab10/readme.md index 5063452..f51c685 100644 --- a/labs/lab10/readme.md +++ b/labs/lab10/readme.md @@ -1,68 +1,165 @@ -# NGINX Caching / Juiceshop or Garage +# Monitoring NGINXaaS for Azure with Grafana ## Introduction -In this lab, you will build ( x,y,x ). +In this lab, you will be exploring the integration between NGINXaaS for Azure and Grafana for monitoring of the service. NGINX as a Service for Azure is a service offering that is tightly integrated into Microsoft Azure public cloud and its ecosystem, making applications fast, efficient, and reliable with full lifecycle management of advanced NGINX traffic services. -< Lab specific Images here, in the /media sub-folder > +NGINXaaS is powered by NGINX Plus, so much of the configuration is similar to what you are already used to. We will use Grafana to create a dashboard in which we will monitor: +- HTTP requests +- HTTP Metrics +- Cache Hit Ratio +- SSL Metrics +- Upstream Response Time +- Health checks -NGINX aaS | Docker +The data for these will be based on the work done in previous labs. + + + +NGINXaaS for Azure | Grafana :-------------------------:|:-------------------------: - | + |  ## Learning Objectives By the end of the lab you will be able to: -- Introduction to `xx` -- Build an `yyy` Nginx configuration -- Test access to your lab enviroment with Curl and Chrome -- Investigate `zzz` - +- Create a Grafana managed instance in Azure +- Create a Dashboard to monitor metrics in NGINXaaS for Azure +- Test the Grafana Server +- View the Grafana Dashboard ## Pre-Requisites -- You must have `aaaa` installed and running -- You must have `bbbbb` installed +- You must be using NGINXaaS for Azure - See `Lab0` for instructions on setting up your system for this Workshop +- Have Docker installed to run workloads (for graph data) - Familiarity with basic Linux commands and commandline tools -- Familiarity with basic Docker concepts and commands - Familiarity with basic HTTP protocol +- Familiarity with Grafana + + +1. Ensure you are in the `lab10` folder. We will set two environment variables and then use these to create the Grafana Instance via the Azure CLI. + +> Please Note there is a charge associated with standing up a Managed Grafana instance in Azure and you should be sure to delete the resources when you are finished exploring the lab. + +The resource group should be the same as the one you have been using for the whole workshop. If it is not set, do it here. The MY_GRAFANA variable is what the resource will be called when you are looking for it in Azure. + +```bash +export MY_RESOURCEGROUP=a.currier-workshop +export MY_GRAFANA=grafanaworkshop + +az grafana create --name $MY_GRAFANA --resource-group $MY_RESOURCEGROUP +``` + +2. In the output of the above command, take note of the endpoint that has been created for you. It should be found in a key labelled *endpoint*. + + + +Using the endpoint URL you can log into the Managed Grafana instance using your Microsoft Entra ID (that you have been using for these labs). If you forgot to grab the endpoint URL, you can retrieve it via the Azure CLI tool: +```bash +az grafana show --name $MY_GRAFANA --resource-group $MY_RESOURCEGROUP --query "properties.endpoint" --output tsv +``` + +Open a web browser and go to the endpoint address listed. You should see an Entra ID login which may or may not have your credentials pre-populated: + + + + +Once signed in, you will be taken to the default Grafana landing page. + + + +From here we will want to click on the Dashboards Menu on the left hand side. + + + +In the upper right of the page is a blue drop down button. We will select *Import*: + + + +In Visual Studio Code, navigate to the Lab10 folder. Open the N4A-Dashboard.json file and inspect it. + +This template file makes use of Grafana Variables to make it easier to customize to your environment. Let's retrieve the values we will need for the dashboard in the VS terminal by running the following commands: + +```bash +export MY_RESOURCEGROUP=$(az resource list --resource-group a.currier-workshop --resource-type Nginx.NginxPlus/nginxDeployments --query "[].resourceGroup" -o tsv) +export MY_RESOURCENAME=$(az resource list --resource-group a.currier-workshop --resource-type Nginx.NginxPlus/nginxDeployments --query "[].name" -o tsv) +export MY_LOCATION=$(az resource list --resource-group a.currier-workshop --resource-type Nginx.NginxPlus/nginxDeployments --query "[].location" -o tsv) +export MY_AKSCluster1=n4a-aks1 +export MY_AKSCluster2=n4a-aks2 +export MY_WindowsVM=windowsvm +export MY_UbuntuVM=ubuntuvm +``` + +Confirm the values were set: +```bash +set | grep MY +MY_AKSCluster1=n4a-aks1 +MY_AKSCluster2=n4a-aks2 +MY_LOCATION=eastus +MY_RESOURCEGROUP=a.currier-workshop +MY_RESOURCENAME=nginx4a +MY_UbuntuVM=ubuntuvm +MY_WindowsVM=windowsvm +``` + +Now that we have these 7 values we can use them in the Dashboard template. + +Copy the code from the N4A-Dashboard.json file. In the grafana import window, paste the code into the import field and then click the blue load button. + + + +To get the Dashboards to load. Replace each variable field at the top (see image) with the values you retrieved for your lab: + + + +### Generate a workload + +1. Start the WRK load generation tool. This will provide some traffic to the NGINXaaS for Azure instance, so that the statistics will be increasing. + +```bash +docker run --rm williamyeh/wrk -t20 -d600s -c1000 https://cafe.example.com/ +```
-### Lab exercise 1 -
-**This completes Lab10.** +**This completes Lab 10.**
## References: -- [NGINX As A Service for Azure](https://docs.nginx.com/nginxaas/azure/) -- [NGINX Plus Product Page](https://docs.nginx.com/nginx/) -- [NGINX Ingress Controller](https://docs.nginx.com//nginx-ingress-controller/) -- [NGINX on Docker](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-docker/) -- [NGINX Directives Index](https://nginx.org/en/docs/dirindex.html) -- [NGINX Variables Index](https://nginx.org/en/docs/varindex.html) +- [NGINX For Azure Metrics Catalog](https://docs.nginx.com/nginxaas/azure/monitoring/metrics-catalog/) +- [Azure Managed Grafana Docs](https://learn.microsoft.com/en-us/azure/managed-grafana/) +- [Build a Grafana Dashboard](https://grafana.com/docs/grafana/latest/getting-started/build-first-dashboard/) +- [NGINX Admin Guide](https://docs.nginx.com/nginx/admin-guide/) - [NGINX Technical Specs](https://docs.nginx.com/nginx/technical-specs/) -- [NGINX - Join Community Slack](https://community.nginx.org/joinslack) + +
@@ -74,4 +171,6 @@ By the end of the lab you will be able to: ------------- -Navigate to ([LabX](../labX/readme.md) | [LabX](../labX/readme.md)) +Navigate to ([Lab Guide](../readme.md)) + + diff --git a/labs/lab2/cafe-docker-upstreams.conf b/labs/lab2/cafe-docker-upstreams.conf index 5f55ac9..dd8d2b2 100644 --- a/labs/lab2/cafe-docker-upstreams.conf +++ b/labs/lab2/cafe-docker-upstreams.conf @@ -7,9 +7,9 @@ upstream cafe_nginx { zone cafe_nginx 256k; # from docker compose - server ubuntuvm:81; - server ubuntuvm:82; - server ubuntuvm:83; + server n4a-ubuntuvm:81; + server n4a-ubuntuvm:82; + server n4a-ubuntuvm:83; keepalive 32; diff --git a/labs/lab2/docker-compose.yml b/labs/lab2/docker-compose.yml index 5222e37..9286d15 100644 --- a/labs/lab2/docker-compose.yml +++ b/labs/lab2/docker-compose.yml @@ -2,7 +2,6 @@ # NGINX for Azure, Mar 2024 # Chris Akker, Shouvik Dutta, Adam Currier # -version: '3.3' services: web1: hostname: docker-web1 diff --git a/labs/lab2/init.sh b/labs/lab2/init.sh new file mode 100644 index 0000000..30700f7 --- /dev/null +++ b/labs/lab2/init.sh @@ -0,0 +1,13 @@ +#!/bin/bash + +sudo apt-get update + +# Install Linux Networking Tools +sudo apt install -y net-tools + +# Install Docker +sudo apt-get install -y docker.io + +# Install Docker Compose +sudo curl -L "https://github.com/docker/compose/releases/download/v2.27.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose +sudo chmod +x /usr/local/bin/docker-compose \ No newline at end of file diff --git a/labs/lab2/media/cafe-icon.png b/labs/lab2/media/cafe-icon.png new file mode 100644 index 0000000..bd93a41 Binary files /dev/null and b/labs/lab2/media/cafe-icon.png differ diff --git a/labs/lab2/media/docker-icon.png b/labs/lab2/media/docker-icon.png new file mode 100644 index 0000000..02ee3f1 Binary files /dev/null and b/labs/lab2/media/docker-icon.png differ diff --git a/labs/lab2/media/lab2-cloudshell.png b/labs/lab2/media/lab2-cloudshell.png new file mode 100644 index 0000000..4458504 Binary files /dev/null and b/labs/lab2/media/lab2-cloudshell.png differ diff --git a/labs/lab2/media/lab2_cafe-diagram.png b/labs/lab2/media/lab2_cafe-diagram.png new file mode 100644 index 0000000..44a730f Binary files /dev/null and b/labs/lab2/media/lab2_cafe-diagram.png differ diff --git a/labs/lab2/media/lab2_cafe-docker-upstreams.png b/labs/lab2/media/lab2_cafe-docker-upstreams.png new file mode 100644 index 0000000..0ca49f6 Binary files /dev/null and b/labs/lab2/media/lab2_cafe-docker-upstreams.png differ diff --git a/labs/lab2/media/lab2_cafe-example-com-conf.png b/labs/lab2/media/lab2_cafe-example-com-conf.png new file mode 100644 index 0000000..813ec4e Binary files /dev/null and b/labs/lab2/media/lab2_cafe-example-com-conf.png differ diff --git a/labs/lab2/media/lab2_cafe-inspect.png b/labs/lab2/media/lab2_cafe-inspect.png new file mode 100644 index 0000000..6352302 Binary files /dev/null and b/labs/lab2/media/lab2_cafe-inspect.png differ diff --git a/labs/lab2/media/lab2_cafe-out-of-stock.png b/labs/lab2/media/lab2_cafe-out-of-stock.png new file mode 100644 index 0000000..1b6aaa5 Binary files /dev/null and b/labs/lab2/media/lab2_cafe-out-of-stock.png differ diff --git a/labs/lab2/media/lab2_cafe-windows-iis.png b/labs/lab2/media/lab2_cafe-windows-iis.png new file mode 100644 index 0000000..5a9aee0 Binary files /dev/null and b/labs/lab2/media/lab2_cafe-windows-iis.png differ diff --git a/labs/lab2/media/lab2_diagram.png b/labs/lab2/media/lab2_diagram.png new file mode 100644 index 0000000..79ae0d2 Binary files /dev/null and b/labs/lab2/media/lab2_diagram.png differ diff --git a/labs/lab2/media/lab2_windows-upstreams.png b/labs/lab2/media/lab2_windows-upstreams.png new file mode 100644 index 0000000..ca137e6 Binary files /dev/null and b/labs/lab2/media/lab2_windows-upstreams.png differ diff --git a/labs/lab2/media/nginx-azure-icon.png b/labs/lab2/media/nginx-azure-icon.png new file mode 100644 index 0000000..70ab132 Binary files /dev/null and b/labs/lab2/media/nginx-azure-icon.png differ diff --git a/labs/lab2/media/ubuntu-icon.png b/labs/lab2/media/ubuntu-icon.png new file mode 100644 index 0000000..cd4fa72 Binary files /dev/null and b/labs/lab2/media/ubuntu-icon.png differ diff --git a/labs/lab2/media/windows-icon.png b/labs/lab2/media/windows-icon.png new file mode 100644 index 0000000..05ac958 Binary files /dev/null and b/labs/lab2/media/windows-icon.png differ diff --git a/labs/lab2/nginx.conf b/labs/lab2/nginx.conf new file mode 100644 index 0000000..29eeb45 --- /dev/null +++ b/labs/lab2/nginx.conf @@ -0,0 +1,42 @@ +# Nginx 4 Azure - Default - Updated Nginx.conf +# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 +# +user nginx; +worker_processes auto; +worker_rlimit_nofile 8192; +pid /run/nginx/nginx.pid; + +events { + worker_connections 4000; +} + +error_log /var/log/nginx/error.log error; + +http { + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log off; + server_tokens ""; + server { + listen 80 default_server; + server_name localhost; + location / { + # Points to a directory with a basic html index file with + # a "Welcome to NGINX as a Service for Azure!" page + root /var/www; + index index.html; + } + } + + include /etc/nginx/conf.d/*.conf; + # include /etc/nginx/includes/*.conf; # shared files + +} + +# stream { + +# include /etc/nginx/stream/*.conf; # Stream TCP nginx files + +# } diff --git a/labs/lab2/readme.md b/labs/lab2/readme.md index 8370f2a..f5516a7 100644 --- a/labs/lab2/readme.md +++ b/labs/lab2/readme.md @@ -1,4 +1,4 @@ -# UbuntuVM/Docker / Windows VM / Cafe Demo Deployment +# Ubuntu VM / Docker / Windows VM / Cafe Demo Deployment ## Introduction @@ -6,30 +6,32 @@ In this lab, you will be creating various application backend resources. You wi Your completed Ubuntu and Windows VM deployment will look like this: -< Lab specific Images here, in the /media sub-folder > +
NGINX aaS | Ubuntu | Docker | Windows -:---------------------:|:---------------------:|:---------------------: - | | | - +:---------------------:|:---------------------:|:---------------------:|:---------------------: + | | | + +
+ ## Learning Objectives By the end of the lab you will be able to: -- Deploy Ubuntu VM with Azure CLI -- Install Docker and Docker Compose +- Deploy Ubuntu VM with Docker and Docker-Compose preinstalled using Azure CLI - Run Nginx demo application containers -- Deploy Windows VM with Azure CLI +- Configure Nginx for Azure to Load Balance Docker containers - Test and validate your lab environment -- Configure Nginx for Azure to load balance these resources +- Deploy Windows VM with Azure CLI +- Configure Nginx for Azure to proxy to the Windows VM - Test your Nginx for Azure configs ## Pre-Requisites - You must have Azure Networking configured for this Workshop -- You must have access to Azure VMs +- You must have proper access to create Azure VMs - You must have Azure CLI tool installed on your local system - You must have an SSH client software installed on your local system - You must have your Nginx for Azure instance deployed and running @@ -40,624 +42,628 @@ By the end of the lab you will be able to:
-## Deploy UbuntuVM with Azure CLI - -After logging onto your Azure tenant, set the following Environment variables needed for this lab: - -```bash -export MY_RESOURCEGROUP=myResourceGroup -export REGION=CentralUS -export MY_VM_NAME=ubuntuvm -export MY_USERNAME=azureuser -export MY_VM_IMAGE="Canonical:0001-com-ubuntu-server-jammy:server-22_04-lts-gen2:latest" - -``` - -To see a list of UbuntuVMs available to you, try: +## Deploy Ubuntu VM with Docker and Docker-Compose preinstalled using Azure CLI + +1. In your local machine open terminal and make sure you are logged onto your Azure tenant. Set the following Environment variable which points to your Resource Group: + + ```bash + ## Set environment variables + export MY_RESOURCEGROUP=s.dutta-workshop + ``` + + >*Make sure your Terminal is the `nginx-azure-workshops/labs` directory for all commands during this Workshop.* + +1. Create the Ubuntu VM that would be acting as your backend application server using below command: + + ```bash + az vm create \ + --resource-group $MY_RESOURCEGROUP \ + --name n4a-ubuntuvm \ + --image Ubuntu2204 \ + --admin-username azureuser \ + --vnet-name n4a-vnet \ + --subnet vm-subnet \ + --assign-identity \ + --generate-ssh-keys \ + --public-ip-sku Standard \ + --custom-data lab2/init.sh + ``` + + ```bash + ##Sample Output## + { + "fqdns": "", + "id": "/subscriptions/
-Copy the contents from the `lab3/docker-compose.yml` file, into the same filename on the Ubuntu VM. Save the file and exit VI. - -Start up the three Nginx demo containers. This tells Docker to read the compose file and start the three containers: - -```bash -sudo docker-compose up -d - -``` - -Check the containers are running: - -```bash -sudo docker ps -a - -``` +### Deploy Nginx Demo containers -It should look similar to this. Notice that each container is listening on a unique TCP port on the Docker host - Ports 81, 82, and 83 for web1, web2 and web3, respectively. +You will now use Docker Compose to create and deploy three Nginx `ingress-demo` containers. These will be your first group of `backends` that will be used for load balancing with Nginx for Azure. -```bash -#Sample output -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -33ca8329cece nginxinc/ingress-demo "/docker-entrypoint.…" 2 minutes ago Up 2 minutes 0.0.0.0:82->80/tcp, :::82->80/tcp, 0.0.0.0:4432->443/tcp, :::4432->443/tcp docker-web2 -d3bf38f7b575 nginxinc/ingress-demo "/docker-entrypoint.…" 2 minutes ago Up 2 minutes 0.0.0.0:83->80/tcp, :::83->80/tcp, 0.0.0.0:4433->443/tcp, :::4433->443/tcp docker-web3 -1982b1a4356d nginxinc/ingress-demo "/docker-entrypoint.…" 2 minutes ago Up 2 minutes 0.0.0.0:81->80/tcp, :::81->80/tcp, 0.0.0.0:4431->443/tcp, :::4431->443/tcp docker-web1 +1. Inspect the `lab2/docker-compose.yml` file. Notice you are pulling the `nginxinc/ingress-demo` image, and starting three containers. The three containers are configured as follows: -``` + Container Name | Name:port + :-------------:|:------------: + docker-web1 | ubuntuvm:81 + docker-web2 | ubuntuvm:82 + docker-web3 | ubuntuvm:83 + +1. On the Ubuntu VM, create a new sub-directory in the `/home/azureuser` directory, call it `cafe`. + + ```bash + cd $HOME + mkdir cafe + ``` + +1. Within the `cafe` sub-directory, you will now add `docker-compose.yml`. + You can do this in two ways. You can create a new `docker-compose.yml` file and copy the contents from the `lab2/docker-compose.yml` file, into this new file on the Ubuntu VM using editor of your choice (Below example uses vi tool). + + ```bash + cd cafe + vi docker-compose.yml + ``` -Verify that all THREE containers have their TCP ports exposed on the Ubuntu VM host: + Alternatively, you can get this file by running the `wget` command as shown below: -```bash -azureuser@ubuntuvm:~/cafe$ netstat -tnl + ```bash + cd cafe + wget https://raw.githubusercontent.com/nginxinc/nginx-azure-workshops/main/labs/lab2/docker-compose.yml + ``` -``` +1. Start up the three Nginx demo containers using below command. This instructs Docker to read the compose file and start the three containers: -```bash -#Sample output -Active Internet connections (only servers) -Proto Recv-Q Send-Q Local Address Foreign Address State -tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN -tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -tcp 0 0 0.0.0.0:81 0.0.0.0:* LISTEN -tcp 0 0 0.0.0.0:83 0.0.0.0:* LISTEN -tcp 0 0 0.0.0.0:82 0.0.0.0:* LISTEN -tcp 0 0 0.0.0.0:4433 0.0.0.0:* LISTEN -tcp 0 0 0.0.0.0:4432 0.0.0.0:* LISTEN -tcp 0 0 0.0.0.0:4431 0.0.0.0:* LISTEN -tcp6 0 0 :::22 :::* LISTEN -tcp6 0 0 :::81 :::* LISTEN -tcp6 0 0 :::83 :::* LISTEN -tcp6 0 0 :::82 :::* LISTEN -tcp6 0 0 :::4433 :::* LISTEN -tcp6 0 0 :::4432 :::* LISTEN -tcp6 0 0 :::4431 :::* LISTEN + ```bash + sudo docker-compose up -d + ``` -``` +1. Check the containers are running: -Yes, looks like ports 81, 82, and 83 are Listening. Note: If you used a different VM, you may need some host Firewall rules to allow traffic to the containers. + ```bash + sudo docker ps + ``` -Test all three containers with curl: + ```bash + ##Sample Output## + CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + 33ca8329cece nginxinc/ingress-demo "/docker-entrypoint.…" 2 minutes ago Up 2 minutes 0.0.0.0:82->80/tcp, :::82->80/tcp, 0.0.0.0:4432->443/tcp, :::4432->443/tcp docker-web2 + d3bf38f7b575 nginxinc/ingress-demo "/docker-entrypoint.…" 2 minutes ago Up 2 minutes 0.0.0.0:83->80/tcp, :::83->80/tcp, 0.0.0.0:4433->443/tcp, :::4433->443/tcp docker-web3 + 1982b1a4356d nginxinc/ingress-demo "/docker-entrypoint.…" 2 minutes ago Up 2 minutes 0.0.0.0:81->80/tcp, :::81->80/tcp, 0.0.0.0:4431->443/tcp, :::4431->443/tcp docker-web1 + ``` -```bash -azureuser@ubuntuvm:~/cafe$ curl -s localhost:81 |grep Server + Notice that each container is listening on a unique TCP port on the Docker host - Ports 81, 82, and 83 for docker-web1, docker-web2 and docker-web3, respectively. -``` +1. Verify that all THREE containers have their TCP ports exposed on the Ubuntu VM host: -Gives you the 1st Container Name as `Server Name`, and Container's IP address as `Server Address`: + ```bash + netstat -tnl + ``` -```bash -#Sample output -
Server Name: docker-web1
-Server Address: 172.18.0.2:80
+ ```bash + #Sample output + Active Internet connections (only servers) + Proto Recv-Q Send-Q Local Address Foreign Address State + tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN + tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN + tcp 0 0 0.0.0.0:81 0.0.0.0:* LISTEN + tcp 0 0 0.0.0.0:83 0.0.0.0:* LISTEN + tcp 0 0 0.0.0.0:82 0.0.0.0:* LISTEN + tcp 0 0 0.0.0.0:4433 0.0.0.0:* LISTEN + tcp 0 0 0.0.0.0:4432 0.0.0.0:* LISTEN + tcp 0 0 0.0.0.0:4431 0.0.0.0:* LISTEN + tcp6 0 0 :::22 :::* LISTEN + tcp6 0 0 :::81 :::* LISTEN + tcp6 0 0 :::83 :::* LISTEN + tcp6 0 0 :::82 :::* LISTEN + tcp6 0 0 :::4433 :::* LISTEN + tcp6 0 0 :::4432 :::* LISTEN + tcp6 0 0 :::4431 :::* LISTEN + ``` -``` + Yes, looks like ports 81, 82, and 83 are Listening. Note: If you used a different VM, you may need to update the VM Host Firewall rules to allow traffic to the containers. -```bash -azureuser@ubuntuvm:~/cafe$ curl -s localhost:82 |grep Server +1. Test all three containers by running curl command within the Ubuntu VM: -``` + ```bash + curl -s localhost:81 |grep Server + ``` -Gives you the 2nd Container Name as `Server Name`, and Container's IP address as `Server Address`: + Gives you the 1st Container Name as `Server Name`, and Container's IP address as `Server Address`: -```bash -#Sample output -Server Name: docker-web2
-Server Address: 172.18.0.3:80
+ ```bash + ##Sample Output## +Server Name: docker-web1
+Server Address: 172.18.0.2:80
+ ``` -``` + ```bash + curl -s localhost:82 |grep Server + ``` -```bash -azureuser@ubuntuvm:~/cafe$ curl -s localhost:83 |grep Server + Gives you the 2nd Container Name as `Server Name`, and Container's IP address as `Server Address`: + + ```bash + ##Sample Output## +Server Name: docker-web2
+Server Address: 172.18.0.3:80
+ ``` -``` + ```bash + curl -s localhost:83 |grep Server + ``` + + Gives you the 3rd Container Name as `Server Name`, and Container's IP address as `Server Address`: + + ```bash + ##Sample Output## +Server Name: docker-web3
+Server Address: 172.18.0.4:80
+ ``` + + If you able to see Responses from all THREE containers, you can continue. -Gives you the 3rd Container Name as `Server Name`, and Container's IP address as `Server Address`: +-```bash -#Sample output -
Server Name: docker-web3
-Server Address: 172.18.0.4:80
+## Configure Nginx for Azure to Load Balance Docker containers -``` +-If you able to see Responses from all THREE containers, you can continue. +In this exercise, you will create your first Nginx config files, for the Nginx Server, Location, and Upstream blocks, to load balance your three Docker containers running on the Ubuntu VM. -## Configure Nginx for Azure to Load Balance Docker containers -In this exercise, you will create your first Nginx config files, for the Nginx Server, Location, and Upstream blocks, to load balance your three Docker containers running on the Ubuntu VM. + -< diagram here > +
NGINX aaS | Docker | Cafe Demo :-------------------------:|:-------------------------:|:-------------------------:  | | -Open the Azure Portal, your Resource Group, then Nginx for Azure, Settings, and then the NGINX Configuration panel. +1. Open Azure portal within your browser and then open your Resource Group. Click on your NGINX for Azure resource (nginx4a) which should open the Overview section of your resource. From the left pane click on `NGINX Configuration` under Settings. -Click on `+ New File`, to create a new Nginx config file. +1. Click on `+ New File`, to create a new Nginx config file. Name the new file `/etc/nginx/conf.d/cafe-docker-upstreams.conf`. -Name the new file `/etc/nginx/conf.d/cafe-docker-upstreams.conf`. + **Important:** You must use the full Linux /directory/filename path for every Nginx config file, for it to be properly created and placed in the correct directory. If you forget, you can delete it and must re-create it. The Azure Portal Text Edit panels do not let you move, or drag-n-drop files or directories. You can `rename` a file by clicking the Pencil icon, and `delete` a file by clicking the Trashcan icon at the top. -**Important:** You must use the full Linux /folder/filename path for every Nginx config file, for it to be properly created and placed in the correct folder. If you forget, you can delete it and must re-create it. The Azure Portal Text Edit panels do not let you move, or drag-n-drop files or folders. You can `rename` a file by clicking the Pencil icon, and `delete` a file by clicking the Trashcan icon at the top. +1. Copy and paste the contents from the matching file present in `lab2` directory from Github, into the Configuration Edit window, shown here: -Copy and paste the contents from the matching file from Github, into the Configuration Edit window, shown here: + ```nginx + # Nginx 4 Azure, Cafe Nginx Demo Upstreams + # Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 + # + # cafe-nginx servers + # + upstream cafe_nginx { + zone cafe_nginx 256k; + + # from docker compose + server n4a-ubuntuvm:81; + server n4a-ubuntuvm:82; + server n4a-ubuntuvm:83; -```nginx -# Nginx 4 Azure, Cafe Nginx Demo Upstreams -# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 -# -# cafe-nginx servers -# -upstream cafe_nginx { - zone cafe_nginx 256k; - - # from docker compose - server ubuntuvm:81; - server ubuntuvm:82; - server ubuntuvm:83; + keepalive 32; - keepalive 32; + } + ``` -} +  -``` + This creates an Nginx Upstream Block, which defines the backend server group that Nginx will load balance traffic to. -<< ss here >> + Click `Submit` to save your Nginx configuration. -This creates an Nginx Upstream Block, which defines the backend server group that Nginx will load balance traffic to. +1. Click the ` + New File` again, and create a second Nginx config file, using the same Nginx for Azure Configuration editor tool. Name the second file `/etc/nginx/conf.d/cafe.example.com.conf`. -Click the ` + New File` again, and create a second Nginx config file, using the same Nginx for Azure Configuration editor tool. +1. Copy and paste the contents of the matching file present in `lab2` directory from Github, into the Configuration Edit window, shown here: -Name the second file `/etc/nginx/conf.d/cafe.example.com.conf`. + ```nginx + # Nginx 4 Azure - Cafe Nginx HTTP + # Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 + # + server { + + listen 80; # Listening on port 80 on all IP addresses on this machine -Copy, then paste the contents of the matching file from Github, into the Configuration Edit window, shown here: + server_name cafe.example.com; # Set hostname to match in request + status_zone cafe.example.com; # Metrics zone name -```nginx -# Nginx 4 Azure - Cafe Nginx HTTP -# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 -# -server { - - listen 80; # Listening on port 80 on all IP addresses on this machine + access_log /var/log/nginx/cafe.example.com.log main; + error_log /var/log/nginx/cafe.example.com_error.log info; - server_name cafe.example.com; # Set hostname to match in request - status_zone cafe.example.com; # Metrics zone name + location / { + # + # return 200 "You have reached cafe.example.com, location /\n"; + + proxy_pass http://cafe_nginx; # Proxy AND load balance to a list of servers + add_header X-Proxy-Pass cafe_nginx; # Custom Header - access_log /var/log/nginx/cafe.example.com.log main; - error_log /var/log/nginx/cafe.example.com_error.log info; + # proxy_pass http://windowsvm; # Proxy AND load balance to a list of servers + # add_header X-Proxy-Pass windowsvm; # Custom Header - location / { - # - # return 200 "You have reached cafe.example.com, location /\n"; - - proxy_pass http://cafe_nginx; # Proxy AND load balance to a list of servers - add_header X-Proxy-Pass cafe_nginx; # Custom Header + } } -} - -``` + ``` -Click the `Submit` Button above the Editor. Nginx will validate your configuration, and if successfull, will reload Nginx with your new configuration. If you receive an error, you will need to fix it before you proceed. + Click `Submit` to save your Nginx configuration. -### Update your local system's DNS /etc/host file +1. Now you need to include these new files into your main `nginx.conf` file within your `nginx4a` resource. Copy and paste the contents of the `nginx.conf` file present in `lab2` directory from Github, into the `nginx.conf` file using Configuration Edit window, shown here: -For easy access your new website, you will need to add the hostname `cafe.example.com` and the Nginx4Azure Public IP address, to your local system DNS hosts file for name resolution. Your N4A Public IP address can be found in your Azure Portal, under `nginx1-ip`. Use VI or other text editor to add the entry to `/etc/hosts`: + ```nginx + # Nginx 4 Azure - Default - Updated Nginx.conf + # Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 + # + user nginx; + worker_processes auto; + worker_rlimit_nofile 8192; + pid /run/nginx/nginx.pid; -```bash -cat /etc/hosts + events { + worker_connections 4000; + } -127.0.0.1 localhost -... -# Nginx for Azure testing -20.3.16.67 cafe.example.com -... + error_log /var/log/nginx/error.log error; + + http { + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log off; + server_tokens ""; + server { + listen 80 default_server; + server_name localhost; + location / { + # Points to a directory with a basic html index file with + # a "Welcome to NGINX as a Service for Azure!" page + root /var/www; + index index.html; + } + } + + include /etc/nginx/conf.d/*.conf; + # include /etc/nginx/includes/*.conf; # shared files + + } + ``` -``` + Notice that the Nginx standard / Best Practice of placing the HTTP Context config files in the `/etc/nginx/conf.d` folder is being followed, and the `include` directive is being used to read these files at Nginx configuration load time. -Save your /etc/hosts file, and quit VI. +1. Click the `Submit` Button above the Editor. Nginx will validate your configurations, and if successful, will reload Nginx with your new configurations. If you receive an error, you will need to fix it before you proceed. -### Update your Azure Network Security Group +
-You likely have one, or more, Azure Network Security Groups that need to updated to allow port 80 HTTP traffic inbound to your Resources. Check and verify that your Source IP is allowed access to both your VNet, and your `nginx1` instance. +### Test your Nginx for Azure configuration -### Test your Nginx4Azure configuration +1. For easy access your new website, update your local system's DNS `/etc/hosts` file. You will add the hostname `cafe.example.com` and the Nginx for Azure Public IP address, to your local system DNS hosts file for name resolution. Your Nginx for Azure Public IP address can be found in your Azure Portal, under `n4a-publicIP`. Use vi tool or any other text editor to add an entry to `/etc/hosts` as shown below: -Using a new Terminal, send a curl command to `http://cafe.example.com`, what do you see ? + ```bash + cat /etc/hosts -```bash -curl -I http://cafe.example.com -``` + 127.0.0.1 localhost + ... -```bash -#Sample output -HTTP/1.1 200 OK -Server: N4A-1.25.1 -Date: Thu, 04 Apr 2024 21:36:30 GMT -Content-Type: text/html; charset=utf-8 -Connection: keep-alive -Expires: Thu, 04 Apr 2024 21:36:29 GMT -Cache-Control: no-cache -X-Proxy-Pass: cafe-nginx + # Nginx for Azure testing + 11.22.33.44 cafe.example.com -``` + ... + ``` -Try the coffee and tea URLs, at http://cafe.example.com/coffee and /tea. + where + - `11.22.33.44` replace with your `n4a-publicIP` resource IP address. -You should see a 200 OK Response. Did you see the `X-Proxy-Pass` header - set to the Upstream block name. +1. Once you have updated the host your /etc/hosts file, save it and quit vi tool. -Did you notice the `Server` header? This is the Nginx Server Token. Optional - Change the Server token to your name, and Submit your configuration. The server_tokens directive is found in the `nginx.conf` file. Change it from `N4A-$nginx_version`, to `N4A-$nginx_version-myname`, and click Submit. +1. Using a new Terminal, send a curl command to `http://cafe.example.com`, what do you see ? -Try the curl again. See the change ? Set it back if you like, the Server token is usually hidden for Security reasons, but you can use it as a quick identity tool temporarily. (Which server did I hit?) + ```bash + curl -I http://cafe.example.com + ``` -```bash -#Sample output -HTTP/1.1 200 OK -Server: N4A-1.25.1-cakker # appended a name -Date: Thu, 04 Apr 2024 21:41:04 GMT -Content-Type: text/html; charset=utf-8 -Connection: keep-alive -Expires: Thu, 04 Apr 2024 21:41:03 GMT -Cache-Control: no-cache -X-Proxy-Pass: cafe-nginx + ```bash + ##Sample Output## + HTTP/1.1 200 OK + Date: Thu, 04 Apr 2024 21:36:30 GMT + Content-Type: text/html; charset=utf-8 + Connection: keep-alive + Expires: Thu, 04 Apr 2024 21:36:29 GMT + Cache-Control: no-cache + X-Proxy-Pass: cafe-nginx + ``` -``` + Try the coffee and tea URLs, at http://cafe.example.com/coffee and http://cafe.example.com/tea. -### Test Nginx 4 Azure to Docker + You should see a 200 OK Response. Did you see the `X-Proxy-Pass` header - set to the Upstream block name? -Try access to your website with a Browser. Open Chrome, and nagivate to `http://cafe.example.com`. You should see an `Out of Stock` image, with a gray metadata panel, filled with names, IP addresses, URLs, etc. This panel comes from the Docker container, using Nginx $variables to populate the gray panel fields. If you open Chrome Developer Tools, and look at the Response Headers, you should be able to see the Server and X-Proxy-Pass Headers set respectively. +1. Now try access to your cafe application with a Browser. Open Chrome, and nagivate to `http://cafe.example.com`. You should see an `Out of Stock` image, with a gray metadata panel, filled with names, IP addresses, URLs, etc. This panel comes from the Docker container, using Nginx $variables to populate the gray panel fields. If you Right+Click, and Inspect to open Chrome Developer Tools, and look at the Response Headers, you should be able to see the `Server and X-Proxy-Pass Headers` set respectively. -<< out of stock ss here >> + Click Refresh serveral times. You will notice the `Server Name` and `Server Ip` fields changing, as N4A is round-robin load balancing the three Docker containers - docker-web1, 2, and 3 respectively. If you open Chrome Developer Tools, and look at the Response Headers, you should be able to see the Server and X-Proxy-Pass Headers set respectively. -Congratulations!! You have just completed launching a simple web application with Nginx for Azure, running on the Internet, with just a VM, Docker, and 2 config files for Nginx for Azure. That wasn't so hard now, was it? - -
+ -## Deploy Windows VM with Azure CLI +Try http://cafe.example.com/coffee and http://cafe.example.com/tea in Chrome, refreshing several times. You should find Nginx for Azure is load balancing these Docker web containers as expected. -After logging onto your Azure tenant, set the following Environment variables needed for this lab: +>**Congratulations!!** You have just completed launching a simple web application with Nginx for Azure, running on the Internet, with just a VM, Docker, and 2 config files for Nginx for Azure. That pretty easy, not so hard now, was it? -```bash -export MY_RESOURCEGROUP=myResourceGroup -export REGION=CentralUS -export MY_VM_NAME=windowsvm -export MY_USERNAME=azureuser -export MY_VM_IMAGE="Windows Server 20xx" +
-``` +### Deploy Windows VM with Azure CLI + +Similar to how you deployed an Ubuntu VM, you will now deploy a Windows VM. + +1. In your local machine open terminal and make sure you are logged onto your Azure tenant. Set the following Environment variables: + + ```bash + export MY_RESOURCEGROUP=s.dutta-workshop + export MY_VM_IMAGE=cognosys:iis-on-windows-server-2016:iis-on-windows-server-2016:1.2019.1009 + ``` + +1. Create the Windows VM (This will take some time to deploy): + + ```bash + az vm create \ + --resource-group $MY_RESOURCEGROUP \ + --name n4a-windowsvm \ + --image $MY_VM_IMAGE \ + --vnet-name n4a-vnet \ + --subnet vm-subnet \ + --admin-username azureuser \ + --public-ip-sku Standard + ``` + + ```bash + ##Sample Output## + Admin Password: + Confirm Admin Password: + Consider upgrading security for your workloads using Azure Trusted Launch VMs. To know more about Trusted Launch, please visit https://aka.ms/TrustedLaunch. + { + "fqdns": "", + "id": "/subscriptions/
## Configure Nginx for Azure to proxy the Windows VM -In this exercise, you will create your second Nginx config file, for the Nginx Server, Location, and Upstream blocks, to proxy your IIS Server running on the Windows VM. +In this exercise, you will create another Nginx config file, for the Windows VM Upstream block, to proxy your IIS Server running on the Windows VM. -< diagram here > + -NGINX aaS | Windows | ? Which Demo Pages -:-------------------------:|:-------------------------:|:-------------------------: - | | +NGINX aaS | Windows VM / IIS +:-------------------------:|:-------------------------: + |  -Open the Azure Portal, your Resource Group, then Nginx for Azure, Settings, and then the NGINX Configuration panel. +
-Click on `+ New File`, to create a new Nginx config file. +1. Open Azure portal within your browser and then open your Resource Group. Click on your NGINX for Azure resource (nginx4a) which should open the Overview section of your resource. From the left pane click on `NGINX Configuration` under settings. -Name the new file `/etc/nginx/conf.d/windows-upstreams.conf`. +1. Click on `+ New File`, to create a new Nginx config file. Name the new file `/etc/nginx/conf.d/windows-upstreams.conf`. -**Important:** You must use the full Linux /folder/filename path for every Nginx config file, for it to be properly created and placed in the correct folder. If you forget, you can delete it and must re-create it. The Azure Portal Text Edit panels do not let you move, or drag-n-drop files or folders. You can `rename` a file by clicking the Pencil icon, and `delete` a file by clicking the Trashcan icon at the top. + **Important:** You must use the full Linux /folder/filename path for every Nginx config file, for it to be properly created and placed in the correct folder. If you forget, you can delete it and must re-create it. The Azure Portal Text Edit panels do not let you move, or drag-n-drop files or folders. You can `rename` a file by clicking the Pencil icon, and `delete` a file by clicking the Trashcan icon at the top. -Copy and paste the contents from the matching file from Github, into the Configuration Edit window, shown here: +1. Copy and paste the contents from the matching file present in `lab2` directory from Github, into the Configuration Edit window, shown here: -```nginx -# Nginx 4 Azure, Windows IIS Upstreams -# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 -# -# windows IIS server -# -upstream windowsvm { - zone windowsvm 256k; - - server windowsvm:80; # IIS Server + ```nginx + # Nginx 4 Azure, Windows IIS Upstreams + # Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 + # + # windows IIS server + # + upstream windowsvm { + zone windowsvm 256k; + + server n4a-windowsvm:80; # IIS Server - keepalive 32; + keepalive 32; -} + } + ``` -``` +  -<< ss here >> + Click `Submit` to save your Nginx configuration. + + This creates a new Nginx Upstream Block, which defines the Windows IIS backend server group that Nginx will load balance traffic to. -This creates a new Nginx Upstream Block, which defines the Windows IIS backend server group that Nginx will load balance traffic to. +1. Edit the comment characters in `/etc/nginx/conf.d/cafe.example.com.conf`, to enable the `proxy_pass` to the `windowsvm`, and disable it for the `cafe-nginx`, as follows: -Edit the comment characters in `/etc/nginx/conf.d/cafe.example.com.conf`, to enable the `proxy_pass` to the `windowsvm`, and disable it for the `cafe-nginx`, as follows: + ```nginx + # Nginx 4 Azure - Cafe Nginx and Windows IIS HTTP + # Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 + # + server { + + listen 80; # Listening on port 80 on all IP addresses on this machine -```nginx -# Nginx 4 Azure - Cafe Nginx and Windows IIS HTTP -# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 -# -server { - - listen 80; # Listening on port 80 on all IP addresses on this machine + server_name cafe.example.com; # Set hostname to match in request + status_zone cafe.example.com; # Metrics zone name - server_name cafe.example.com; # Set hostname to match in request - status_zone cafe.example.com; # Metrics zone name + access_log /var/log/nginx/cafe.example.com.log main; + error_log /var/log/nginx/cafe.example.com_error.log info; - access_log /var/log/nginx/cafe.example.com.log main; - error_log /var/log/nginx/cafe.example.com_error.log info; + location / { + # + # return 200 "You have reached cafe.example.com, location /\n"; + + # proxy_pass http://cafe_nginx; # Proxy AND load balance to a list of servers + # add_header X-Proxy-Pass cafe_nginx; # Custom Header - location / { - # - # return 200 "You have reached cafe.example.com, location /\n"; - - # proxy_pass http://cafe_nginx; # Proxy AND load balance to a list of servers - # add_header X-Proxy-Pass cafe_nginx; # Custom Header + proxy_pass http://windowsvm; # Proxy AND load balance to a list of servers + add_header X-Proxy-Pass windowsvm; # Custom Header - proxy_pass http://windowsvm; # Proxy AND load balance to a list of servers - add_header X-Proxy-Pass windowsvm; # Custom Header + } } + ``` -} +1. Click the `Submit` Button above the Editor. Nginx will validate your configuration, and if successfull, will reload Nginx with your new configuration. If you receive an error, you will need to fix it before you proceed. + +
-``` +### Test your Nginx for Azure configs -Click the `Submit` Button above the Editor. Nginx will validate your configuration, and if successfull, will reload Nginx with your new configuration. If you receive an error, you will need to fix it before you proceed. +1. Test access again to http://cafe.example.com. You will now see the IIS default server page, instead of the Cafe Out of Stock page. If you check Chrome Dev Tools, the X-Proxy-Pass Header should now show `windowsvm`. -Test access again to http://cafe.example.com. You will now see the IIS default server page, instead of the Out of Stock page. If you check Chrome Dev Tools, the X-Proxy-Pass header should now show `windowsvm`. +  -Notice how easy it was, to create a new backend server, and then tell Nginx to proxy_pass to a different Upstream ? You used the same Hostname, DNS record, and Nginx Server block, but you just told Nginx to switch backends. + >Notice how easy it was, to create a new backend server, and then tell Nginx to `proxy_pass` to a different Upstream. You used the same Hostname, DNS record, and Nginx Server block, but you just told Nginx to switch backends with a different `proxy_pass` directive. -Edit the `cafe.example.com.conf` file again, and change the comments to enable the `proxy_pass` for `cafe_nginx`, as you will use it again in a future lab exercise. +1. Edit the `cafe.example.com.conf` file again, and change the comments to disable `windowsvm`, and re-enable the `proxy_pass` for `cafe_nginx`, as you will use it again in a future lab exercise. -Submit your changes, and re-test to verify that http://cafe.example.com works again for Cafe Nginx. Don't forget to change the custom header as well. +1. Submit your Nginx changes, and re-test to verify that http://cafe.example.com works again for Cafe Nginx. Don't forget to change the custom Header as well. + +
**This completes Lab2.** @@ -666,8 +672,9 @@ Submit your changes, and re-test to verify that http://cafe.example.com works ag ## References: - [NGINX As A Service for Azure](https://docs.nginx.com/nginxaas/azure/) + - [NGINX Plus Product Page](https://docs.nginx.com/nginx/) -- [NGINX on Docker](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-docker/) + - [NGINX Directives Index](https://nginx.org/en/docs/dirindex.html) - [NGINX Variables Index](https://nginx.org/en/docs/varindex.html) - [NGINX Technical Specs](https://docs.nginx.com/nginx/technical-specs/) @@ -683,4 +690,4 @@ Submit your changes, and re-test to verify that http://cafe.example.com works ag ------------- -Navigate to ([Lab3](../lab3/readme.md) | [LabX](../labX/readme.md)) +Navigate to ([Lab3](../lab3/readme.md) | [LabGuide](../readme.md)) diff --git a/labs/lab2/windows-upstreams.conf b/labs/lab2/windows-upstreams.conf index 8478bfe..0ffb181 100644 --- a/labs/lab2/windows-upstreams.conf +++ b/labs/lab2/windows-upstreams.conf @@ -6,7 +6,7 @@ upstream windowsvm { zone windowsvm 256k; - server windowsvm:80; # IIS Server + server n4a-windowsvm:80; # IIS Server keepalive 32; diff --git a/labs/lab3/PUT-NGINXplus-REPO-JWT-HERE b/labs/lab3/PUT-NGINXplus-REPO-JWT-HERE new file mode 100644 index 0000000..e69de29 diff --git a/ca-notes/aks/nic/dashboard-vs.yaml b/labs/lab3/dashboard-vs.yaml similarity index 89% rename from ca-notes/aks/nic/dashboard-vs.yaml rename to labs/lab3/dashboard-vs.yaml index 24151a0..040e92a 100644 --- a/ca-notes/aks/nic/dashboard-vs.yaml +++ b/labs/lab3/dashboard-vs.yaml @@ -18,7 +18,7 @@ metadata: name: dashboard-vs namespace: nginx-ingress spec: - host: dashboard.nginxazure.build + host: dashboard.example.com upstreams: - name: dashboard service: dashboard-svc @@ -29,4 +29,5 @@ spec: pass: dashboard - path: /api action: - pass: dashboard \ No newline at end of file + pass: dashboard + \ No newline at end of file diff --git a/labs/lab3/media/aks-icon.png b/labs/lab3/media/aks-icon.png new file mode 100644 index 0000000..0d4ff61 Binary files /dev/null and b/labs/lab3/media/aks-icon.png differ diff --git a/labs/lab3/media/aks-icon2.png b/labs/lab3/media/aks-icon2.png new file mode 100644 index 0000000..37de29c Binary files /dev/null and b/labs/lab3/media/aks-icon2.png differ diff --git a/labs/lab3/media/lab3_diagram.png b/labs/lab3/media/lab3_diagram.png new file mode 100644 index 0000000..30eeac8 Binary files /dev/null and b/labs/lab3/media/lab3_diagram.png differ diff --git a/labs/lab3/media/lab3_nic-dashboards-diagram.png b/labs/lab3/media/lab3_nic-dashboards-diagram.png new file mode 100644 index 0000000..83111a4 Binary files /dev/null and b/labs/lab3/media/lab3_nic-dashboards-diagram.png differ diff --git a/labs/lab3/media/nginx-azure-icon.png b/labs/lab3/media/nginx-azure-icon.png new file mode 100644 index 0000000..70ab132 Binary files /dev/null and b/labs/lab3/media/nginx-azure-icon.png differ diff --git a/labs/lab3/media/nginx-ingress-icon.png b/labs/lab3/media/nginx-ingress-icon.png new file mode 100644 index 0000000..0196a5a Binary files /dev/null and b/labs/lab3/media/nginx-ingress-icon.png differ diff --git a/ca-notes/aks/nic/nginx-plus-ingress.yaml b/labs/lab3/nginx-plus-ingress.yaml similarity index 96% rename from ca-notes/aks/nic/nginx-plus-ingress.yaml rename to labs/lab3/nginx-plus-ingress.yaml index a3cde18..51a0990 100644 --- a/ca-notes/aks/nic/nginx-plus-ingress.yaml +++ b/labs/lab3/nginx-plus-ingress.yaml @@ -19,6 +19,8 @@ spec: prometheus.io/scheme: http spec: serviceAccountName: nginx-ingress + imagePullSecrets: + - name: regcred automountServiceAccountToken: true securityContext: seccompProfile: @@ -33,7 +35,7 @@ spec: # - name: nginx-log # emptyDir: {} containers: - - image: acrakker.azurecr.io/nginx-plus-ingress:3.2.1-alpine-fips + - image: private-registry.nginx.com/nginx-ic/nginx-plus-ingress:3.3.2 imagePullPolicy: IfNotPresent name: nginx-plus-ingress ports: diff --git a/labs/lab3/nic1-dashboard-upstreams.conf b/labs/lab3/nic1-dashboard-upstreams.conf new file mode 100644 index 0000000..0018d19 --- /dev/null +++ b/labs/lab3/nic1-dashboard-upstreams.conf @@ -0,0 +1,16 @@ +# Nginx 4 Azure to NIC, AKS Node for Upstreams +# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 +# +# nginx ingress dashboard +# +upstream nic1_dashboard { + zone nic1_dashboard 256k; + + # from nginx-ingress NodePort Service / aks1 Node IPs + server aks-nodepool1-19055428-vmss000003:32090; #aks1 node1 + server aks-nodepool1-19055428-vmss000004:32090; #aks1 node2 + server aks-nodepool1-19055428-vmss000005:32090; #aks1 node3 + + keepalive 8; + +} diff --git a/ca-notes/n4a-configs/nic1-dashboard.conf b/labs/lab3/nic1-dashboard.conf similarity index 77% rename from ca-notes/n4a-configs/nic1-dashboard.conf rename to labs/lab3/nic1-dashboard.conf index a3a1dd0..8393b0e 100644 --- a/ca-notes/n4a-configs/nic1-dashboard.conf +++ b/labs/lab3/nic1-dashboard.conf @@ -1,8 +1,8 @@ -# N4A NIC Dashboard config +# N4A NIC Dashboard config for AKS1 # server { listen 9001; - server_name dashboard.nginxazure.build; + server_name dashboard.example.com; access_log off; location = /dashboard.html { diff --git a/ca-notes/n4a-configs/aks2-dashboard-upstreams.conf b/labs/lab3/nic2-dashboard-upstreams.conf similarity index 50% rename from ca-notes/n4a-configs/aks2-dashboard-upstreams.conf rename to labs/lab3/nic2-dashboard-upstreams.conf index 99370d6..263deca 100644 --- a/ca-notes/n4a-configs/aks2-dashboard-upstreams.conf +++ b/labs/lab3/nic2-dashboard-upstreams.conf @@ -7,10 +7,10 @@ upstream nic2_dashboard { zone nic2_dashboard 256k; # from nginx-ingress NodePort Service / aks Node IPs - #server aks-nodepool1-19485366:32090; #aks2 cluster nodes: - server aks-nodepool1-19485366-vmss00000h:32090; #aks node1: - server aks-nodepool1-19485366-vmss00000i:32090; #aks node2: - server aks-nodepool1-19485366-vmss00000j:32090; #aks node3: + server aks-nodepool1-29147198-vmss000000:32090; #aks2 node1 + server aks-nodepool1-29147198-vmss000001:32090; #aks2 node2 + server aks-nodepool1-29147198-vmss000002:32090; #aks2 node3 + server aks-nodepool1-29147198-vmss000003:32090; #aks2 node4 keepalive 8; diff --git a/ca-notes/n4a-configs/nic2-dashboard.conf b/labs/lab3/nic2-dashboard.conf similarity index 77% rename from ca-notes/n4a-configs/nic2-dashboard.conf rename to labs/lab3/nic2-dashboard.conf index e07551b..76cc874 100644 --- a/ca-notes/n4a-configs/nic2-dashboard.conf +++ b/labs/lab3/nic2-dashboard.conf @@ -1,8 +1,8 @@ -# N4A NIC Dashboard config +# N4A NIC Dashboard config for AKS2 # server { listen 9002; - server_name dashboard.nginxazure.build; + server_name dashboard.example.com; access_log off; location = /dashboard.html { diff --git a/labs/lab3/nodeport-static.yaml b/labs/lab3/nodeport-static.yaml new file mode 100644 index 0000000..3c47ee4 --- /dev/null +++ b/labs/lab3/nodeport-static.yaml @@ -0,0 +1,23 @@ +apiVersion: v1 +kind: Service +metadata: + name: nginx-ingress + namespace: nginx-ingress +spec: + type: NodePort + ports: + - port: 80 + nodePort: 32080 + protocol: TCP + name: http + - port: 443 + nodePort: 32443 + protocol: TCP + name: https + - port: 9000 + nodePort: 32090 + protocol: TCP + name: dashboard + selector: + app: nginx-ingress + \ No newline at end of file diff --git a/labs/lab3/readme.md b/labs/lab3/readme.md index 963ceff..b447c1a 100644 --- a/labs/lab3/readme.md +++ b/labs/lab3/readme.md @@ -1,725 +1,996 @@ -# AKS / NGINX Ingress Controller / Cafe or Garage Demo Deployment +# AKS / Nginx Ingress Controller Deployment ## Introduction -In this lab, you will explore how Nginx for Azure can route and load balance traffic to backend Kubernetes applications, pods, and services. You will create 2 AKS Kubernetes clusters, install NGINX Plus Ingress Controllers, and several demo applications. This will be your testing platform for Nginx for Azure with AKS - deploying and managing applications, networking, and using both NGINX for Azure and NGINX Ingress features to control traffic to your Modern Apps running in the clusters. Then you will pull an NGINX Plus Ingress Controller Image from the F5 NGINX Private Registry. Then you will deploy the NGINX Ingress Controllers, and configure it to route traffic to the demo app. +In this lab, you will expand your test environment by adding AKS, Azure Kubernetes Resources. You will create 2 new AKS Kubernetes clusters, and deploy NGINX Plus Ingress Controller. This will be your testing platform for Nginx for Azure with AKS - deploying and managing applications, networking, and using both NGINX for Azure and NGINX Plus Ingress features to control traffic to your Modern Apps running in the clusters. You will use a Kubernetes Service to access the Nginx Plus Ingress Dashboard, and expose it with Nginx for Azure, so you can see in real time what is happening inside both AKS clusters. + +
+ +NGINX aaS | AKS | Nginx Plus Ingress +:---------------------:|:---------------------:|:---------------------: + | |
## Learning Objectives +- Understand what is Azure AKS? - Deploy 2 Kubernetes clusters using Azure CLI. -- Pulling and deploying the NGINX Plus Ingress Controller image. -- Deploying the Nginx Ingress Dashboard -- Deploying the Cafe Demo application -- Test and verify proper operation on both AKS clusters. -- Expose the Cafe Demo app and Nginx Ingress Dashboards +- Test and verify proper operation of both AKS clusters. +- Deploy the NGINX Plus Ingress Controller image. +- Deploy the Nginx Plus Ingress Dashboard. +- Expose the Nginx Ingress Dashboards with Nginx for Azure. + +## Prerequisites + +- You must have Azure Networking configured for this Workshop +- You must have Azure CLI tool installed on your local system +- You must have Kubectl installed on your local system +- You must have Git installed on your local system +- You must Docker Desktop or Docker client tools installed on your local system +- You must have your Nginx for Azure instance deployed and running +- Familiarity with Azure Resource types - Resource Groups, VMs, NSG, AKS, etc +- Familiarity with basic Linux commands and commandline tools +- Familiarity with Kubernetes / AKS concepts and commands +- Familiarity with basic HTTP protocol +- Familiarity with Ingress Controller concepts +- See `Lab0` for instructions on setting up your system for this Workshop -## What is Azure AKS? +
-Azure Kubernetes Service is a service provided for Kubernetes on Azure -infrastructure. The Kubernetes resources will be fully managed by Microsoft Azure, which offloads the burden of maintaining the infrastructure, and makes sure these resources are highly available and reliable at all times. This is often a good choice for Modern Applications running as containers, and using Kubernetes Services to control them. +Your new Lab Diagram will look similar to this: -## Azure Regions and naming convention suggestions + -1. Check out the available [Azure Regions](https://learn.microsoft.com/en-us/azure/reliability/availability-zones-overview).
-Decide on a [Datacenter region](https://azure.microsoft.com/en-us/explore/global-infrastructure/geographies/#geographies) that is closest to you and meets your needs.
-Check out the [Azure latency test](https://www.azurespeed.com/Azure/Latency)! We will need to choose one and input a region name in the following steps. +
-2. Consider a naming and tagging convention to organize your cloud assets to support user identification of shared subscriptions. +### What is Azure AKS? -**Example:** +Azure Kubernetes Service is a service provided for Kubernetes on Azure infrastructure. The Kubernetes resources will be fully managed by Microsoft Azure, which offloads the burden of maintaining the infrastructure, and makes sure these resources are highly available and reliable at all times. This is often a good choice for Modern Applications running as containers, and using Kubernetes Services to control them. -You are located in Chicago, Illinois. You choose the Datacenter region -`Central US`. These labs will use the following naming convention: +### Deploy first Kubernetes Cluster with Azure CLI -```bash -
-To do so create a `private-registry.nginx.com` directory under below paths based on your operating system. (See [references](#references) section for more details) - - **linux** : `/etc/docker/certs.d` - - **mac** : `~/.docker/certs.d` - - **windows** : `~/.docker/certs.d` - -3. Copy your `nginx-repo.crt` and `nginx-repo.key` file in the newly created directory. - - Below are the commands for mac/windows based systems - ```bash - mkdir -p ~/.docker/certs.d/private-registry.nginx.com - cp nginx-repo.crt ~/.docker/certs.d/private-registry.nginx.com/client.cert - cp nginx-repo.key ~/.docker/certs.d/private-registry.nginx.com/client.key - ``` - -4. ***Optional** Step only for Mac and Windows system - - Restart Docker Desktop so that it copies the `~/.docker/certs.d` directory from your Mac or Windows system to the `/etc/docker/certs.d` directory on **Moby** (the Docker Desktop `xhyve` virtual machine). - -5. Once Docker Desktop has restarted, run below command to pull the NGINX Plus Ingress Controller image from F5 private container registry. - ```bash - docker pull private-registry.nginx.com/nginx-ic/nginx-plus-ingress:3.2.1-alpine - ``` - >**Note**: At the time of this writing `3.2.1-alpine` is the latest NGINX Plus Ingress version that is available. Please feel free to use the latest version of NGINX Plus Ingress Controller. Look into [references](#references) for the latest Ingress images. + ``` -6. Set below variables to tag and push image to Azure ACR - ```bash - MY_ACR=acrshouvik - MY_REPO=nginxinc/nginx-plus-ingress - MY_TAG=3.2.1-alpine - MY_IMAGE_ID=$(docker images private-registry.nginx.com/nginx-ic/nginx-plus-ingress:$MY_TAG --format "{{.ID}}") - ``` - Check all variables have been set properly by running below command: - ```bash - set | grep MY_ - ``` +1. Open your browser to http://localhost:9000/dashboard.html. + + You should see the Nginx Plus Dashboard. This dashboard would provide more metrics as you progress through the workshop. + + Type `Ctrl+C` within your terminal to stop the Port-Forward when you are finished. + +### Expose NGINX Plus Ingress Controller Dashboard using Virtual Server + +In this section, you are going to expose the NGINX Plus Dashboard to monitor both NGINX Ingress Controller as well as your backend applications as Upstreams. This is a great Plus feature to allow you to watch and triage any potential issues with NGINX Plus Ingress controller as well as any issues with your backend applications in real time. + +You will deploy a `Service` and a `VirtualServer` resource to provide access to the NGINX Plus Dashboard for live monitoring. NGINX Ingress [`VirtualServer`](https://docs.nginx.com/nginx-ingress-controller/configuration/virtualserver-and-virtualserverroute-resources/) is a [Custom Resource Definition (CRD)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) used by NGINX to configure NGINX Server and Location blocks for NGINX configurations. + +1. Switch your context to point to first AKS cluster using below command: -7. After setting the variables, tag the pulled NGINX Plus Ingress image using below command - ```bash - docker tag $MY_IMAGE_ID $MY_ACR.azurecr.io/$MY_REPO:$MY_TAG - ``` -8. Login to the ACR registry using below command. ```bash - az acr login --name $MY_ACR + # Set context to 1st cluster(n4a-aks1) + kubectl config use-context n4a-aks1 ``` -9. Push your tagged image to ACR registry ```bash - docker push $MY_ACR.azurecr.io/$MY_REPO:$MY_TAG + ##Sample Output## + Switched to context "n4a-aks1". ``` -10. Once pushed you can check the image by running below command - ```bash - az acr repository list --name $MY_ACR --output table - ``` +1. Inspect the `lab3/dashboard-vs` manifest. This will deploy a `Service` and a `VirtualServer` resource that will be used to expose the NGINX Plus Ingress Controller Dashboard outside the cluster, so you can see what it is doing. -<< we need the output here, to show the ingress-demo and Nic images exist >> + ```bash + kubectl apply -f lab3/dashboard-vs.yaml + ``` -
+ ```bash + ##Sample Output## + service/dashboard-svc created + virtualserver.k8s.nginx.org/dashboard-vs created + ``` -## Deploy Nginx Plus Ingress Controller to both clusters +1. Verify the Service and Virtual Server were created in first cluster and are Valid: -< NIC deployment steps here - use nodeport-static Manifest at the end > + ```bash + kubectl get svc,vs -n nginx-ingress + ``` -## Introduction + ```bash + ##Sample Output## + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + service/dashboard-svc ClusterIP 10.0.197.220
+ ```bash + # Set context to 2nd cluster(n4a-aks2) + kubectl config use-context n4a-aks2 -## Learning Objectives + kubectl apply -f lab3/dashboard-vs.yaml + ``` -- Install NGINX Ingress Controller using manifest files -- Check your NGINX Ingress Controller -- Deploy the NGINX Ingress Controller Dashboard -- (Optional Section): Look "under the hood" of NGINX Ingress Controller + ```bash + ##Sample Output## + Switched to context "n4a-aks2". + service/dashboard-svc created + virtualserver.k8s.nginx.org/dashboard-vs created + ``` -## Install NGINX Ingress Controller using Manifest files +1. Verify the Service and Virtual Server were created in second cluster and are Valid: -1. Make sure your AKS cluster is running. If it is in stopped state then you can start it using below command. ```bash - MY_RESOURCEGROUP=s.dutta - MY_AKS=aks-shouvik - - az aks start --resource-group $MY_RESOURCEGROUP --name $MY_AKS + kubectl get svc,vs -n nginx-ingress ``` - >**Note**: The FQDN for API server for AKS might change on restart of the cluster which would result in errors running `kubectl` commands from your workstation. To update the FQDN re-import the credentials again using below command. This command would prompt about overwriting old objects. Enter "y" to overwrite the existing objects. - >```bash - >az aks get-credentials --resource-group $MY_RESOURCEGROUP --name $MY_AKS - >``` - >```bash - >###Sample Output### - >A different object named aks-shouvik already exists in your kubeconfig file. - >Overwrite? (y/n): y - >A different object named clusterUser_s.dutta_aks-shouvik already exists in your kubeconfig file. - >Overwrite? (y/n): y - >Merged "aks-shouvik" as current context in /Users/shodutta/.kube/config - >``` -2. Clone the Nginx Ingress Controller repo and navigate into the /deployments folder to make it your working directory: ```bash - git clone https://github.com/nginxinc/kubernetes-ingress.git --branch v3.2.1 - cd kubernetes-ingress/deployments + ##Sample Output## + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + service/dashboard-svc ClusterIP 10.0.145.255
+1. Deploy a NodePort Service within first cluster to expose the NGINX Plus Ingress Controller outside the cluster. - Navigate back to the Workshop's `labs` directory - ```bash - cd ../../labs - ``` - - Observe the `lab3/nginx-plus-ingress.yaml` looking at below details: - - On line #36, the `nginx-plus-ingress:3.2.1` placeholder is changed to the workshop image that you pushed to your private ACR registry as instructed in a previous step. - - >**Note:** Make sure you replace the image with the appropriate image that you pushed in your ACR registry. - - On lines #50-51, we have added TCP port 9000 for the Plus Dashboard. - - On lines #96-97, we have enabled the Dashboard and set the IP access controls to the Dashboard. - - On lines #16-19, we have enabled Prometheus related annotations. - - On line #106, we have enabled Prometheus to collect metrics from the NGINX Plus stats API. - - On line #95, uncomment to make use of default TLS secret. - - On line #109, uncomment to enable the use of Global Configurations. - - Now deploy NGINX Ingress Controller as a Deployment using your updated manifest file. - ```bash - kubectl apply -f lab3/nginx-plus-ingress.yaml - ``` + ```bash + # Set context to 1st cluster(n4a-aks1) + kubectl config use-context n4a-aks1 -## Check your NGINX Ingress Controller + kubectl apply -f lab3/nodeport-static.yaml + ``` -1. Verify the NGINX Plus Ingress controller is up and running correctly in the Kubernetes cluster: +1. Verify the NodePort Service was created within first cluster: ```bash - kubectl get pods -n nginx-ingress + kubectl get svc nginx-ingress -n nginx-ingress ``` ```bash - ###Sample Output### - NAME READY STATUS RESTARTS AGE - nginx-ingress-5764ddfd78-ldqcs 1/1 Running 0 17s + ##Sample Output## + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + nginx-ingress NodePort 10.0.211.17
-## References: +## References: - [Deploy AKS cluster using Azure CLI](https://learn.microsoft.com/en-us/azure/aks/learn/quick-kubernetes-deploy-cli) - [Azure CLI command list for AKS](https://learn.microsoft.com/en-us/cli/azure/aks?view=azure-cli-latest) -- [Create private container registry using Azure CLI](https://learn.microsoft.com/en-us/azure/container-registry/container-registry-get-started-azure-cli) -- [Azure CLI command list for ACR](https://learn.microsoft.com/en-us/cli/azure/acr?view=azure-cli-latest) -- [Authenticate with ACR from AKS cluster](https://learn.microsoft.com/en-us/azure/container-registry/authenticate-kubernetes-options#scenarios) -- [Pulling NGINX Plus Ingress Controller Image](https://docs.nginx.com/nginx-ingress-controller/installation/pulling-ingress-controller-image) -- [Add Client Certificate Mac](https://docs.docker.com/desktop/faqs/macfaqs/#add-client-certificates) -- [Add Client Certificate Windows](https://docs.docker.com/desktop/faqs/windowsfaqs/#how-do-i-add-client-certificates) -- [Docker Engine Security Documentation](https://docs.docker.com/engine/security/certificates/) +- [Installing NGINX Plus Ingress Controller Image](https://docs.nginx.com/nginx-ingress-controller/installation/nic-images/using-the-jwt-token-docker-secret/) - [Latest NGINX Plus Ingress Images](https://docs.nginx.com/nginx-ingress-controller/technical-specifications/#images-with-nginx-plus) +- [Nginx Live Monitoring Dashboard](https://docs.nginx.com/nginx/admin-guide/monitoring/live-activity-monitoring/) +- [Nginx Ingress Controller Product](https://docs.nginx.com/nginx-ingress-controller/) +- [Nginx Ingress Installation - the REAL Nginx](https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-manifests/) +- [Nginx Blog - Which Ingress AM I RUNNING?](https://www.nginx.com/blog/guide-to-choosing-ingress-controller-part-1-identify-requirements/)
- ### Authors - Chris Akker - Solutions Architect - Community and Alliances @ F5, Inc. - Shouvik Dutta - Solutions Architect - Community and Alliances @ F5, Inc. -- Adam Currier - Solutions Architect - Community and Alliances @ F5, Inc. ------------- -Navigate to ([Lab5](../lab5/readme.md) | [LabX](../labX/readme.md)) +Navigate to ([Lab4](../lab4/readme.md) | [LabGuide](../readme.md)) diff --git a/ca-notes/aks/cafe/cafe-vs.yaml b/labs/lab4/cafe-vs.yaml similarity index 76% rename from ca-notes/aks/cafe/cafe-vs.yaml rename to labs/lab4/cafe-vs.yaml index 0d109bd..99ec01a 100644 --- a/ca-notes/aks/cafe/cafe-vs.yaml +++ b/labs/lab4/cafe-vs.yaml @@ -5,7 +5,7 @@ kind: VirtualServer metadata: name: cafe-vs spec: - host: cafe.nginxazure.build + host: cafe.example.com #tls: #secret: cafe-secret #redirect: @@ -15,8 +15,6 @@ spec: - name: tea service: tea-svc port: 80 - slow-start: 20s - # lb-method: least_time last_byte healthCheck: enable: true path: /tea @@ -29,8 +27,6 @@ spec: - name: coffee service: coffee-svc port: 80 - slow-start: 20s - # lb-method: least_time last_byte healthCheck: enable: true path: /coffee @@ -44,7 +40,7 @@ spec: - path: / action: redirect: - url: http://cafe.nginxazure.build/coffee + url: http://cafe.example.com/coffee code: 302 - path: /tea action: @@ -52,10 +48,10 @@ spec: - path: /coffee action: pass: coffee - - path: /n4a + - path: /workshop action: return: code: 200 type: text/html - body: "Welcome to Nginx4Azure & NGINX Plus Ingress Controller !!" + body: "Welcome to Nginx4Azure Workshop !!" diff --git a/ca-notes/aks/cafe/cafe.yaml b/labs/lab4/cafe.yaml similarity index 100% rename from ca-notes/aks/cafe/cafe.yaml rename to labs/lab4/cafe.yaml diff --git a/ca-notes/aks/nic/global-configuration-redis.yaml b/labs/lab4/global-configuration-redis.yaml similarity index 95% rename from ca-notes/aks/nic/global-configuration-redis.yaml rename to labs/lab4/global-configuration-redis.yaml index d822c4c..0fd4c7c 100644 --- a/ca-notes/aks/nic/global-configuration-redis.yaml +++ b/labs/lab4/global-configuration-redis.yaml @@ -1,4 +1,4 @@ - +# Nginx For Azure # NIC Global Config manifest for custom TCP ports for Redis # Chris Akker Jan 2024 # diff --git a/labs/lab4/media/azure-icon.png b/labs/lab4/media/azure-icon.png new file mode 100644 index 0000000..51ffec5 Binary files /dev/null and b/labs/lab4/media/azure-icon.png differ diff --git a/labs/lab4/media/cafe-icon.png b/labs/lab4/media/cafe-icon.png new file mode 100644 index 0000000..0ec53be Binary files /dev/null and b/labs/lab4/media/cafe-icon.png differ diff --git a/labs/lab4/media/lab4_cafe-upstreams-2.png b/labs/lab4/media/lab4_cafe-upstreams-2.png new file mode 100644 index 0000000..6d3f77d Binary files /dev/null and b/labs/lab4/media/lab4_cafe-upstreams-2.png differ diff --git a/labs/lab4/media/lab4_cafe-upstreams-3.png b/labs/lab4/media/lab4_cafe-upstreams-3.png new file mode 100644 index 0000000..ac47b85 Binary files /dev/null and b/labs/lab4/media/lab4_cafe-upstreams-3.png differ diff --git a/labs/lab4/media/lab4_diagram.png b/labs/lab4/media/lab4_diagram.png new file mode 100644 index 0000000..5aff4f0 Binary files /dev/null and b/labs/lab4/media/lab4_diagram.png differ diff --git a/labs/lab4/media/lab4_http-zones.png b/labs/lab4/media/lab4_http-zones.png new file mode 100644 index 0000000..302a2e5 Binary files /dev/null and b/labs/lab4/media/lab4_http-zones.png differ diff --git a/labs/lab4/media/lab4_redis-upstreams.png b/labs/lab4/media/lab4_redis-upstreams.png new file mode 100644 index 0000000..f17353f Binary files /dev/null and b/labs/lab4/media/lab4_redis-upstreams.png differ diff --git a/labs/lab4/media/lab4_redis-zones.png b/labs/lab4/media/lab4_redis-zones.png new file mode 100644 index 0000000..dbe2828 Binary files /dev/null and b/labs/lab4/media/lab4_redis-zones.png differ diff --git a/labs/lab4/media/nginx-ingress-icon.png b/labs/lab4/media/nginx-ingress-icon.png new file mode 100644 index 0000000..0196a5a Binary files /dev/null and b/labs/lab4/media/nginx-ingress-icon.png differ diff --git a/labs/lab4/media/readme.md b/labs/lab4/media/readme.md new file mode 100644 index 0000000..a20b986 --- /dev/null +++ b/labs/lab4/media/readme.md @@ -0,0 +1,666 @@ +# Cafe Demo / Redis Deployment + +## Introduction + +In this lab, you deploy the Nginx Cafe Demo, and Redis In Memory cache applications. You will configure Nginx for Azure to expose these applications to the Internet. Then you will test and load test them, to make sure they perform as expected. You will use the Nginx Plus Dashboard and Azure Monitoring to watch the metrics about your traffic. + +NGINX aaS | Cafe | Redis +:-------------------:|:-------------------:|:-------------------: + | | + +## Learning Objectives + +By the end of the lab you will be able to: + +- Deploying the Cafe Demo application +- Deploying the Redis In Memory cache +- Expose the Cafe Demo app with Nginx for Azure +- Expose the Redis Cache with Nginx for Azure + + +## Pre-Requisites + +- You must have both AKS clusters up and running +- You must have both Nginx Ingress Controllers running +- You must have the NIC Dashboard available +- See `Lab0` for instructions on setting up your system for this Workshop +- Familiarity with basic Linux commands and commandline tools +- Familiarity with basic Kubernetes concepts and commands +- Familiarity with basic HTTP protocol + +
+ +## Deploy the Nginx CAFE Demo app + +In this section, you will deploy the "Cafe Nginx" Ingress Demo, which represents a Coffee Shop website with Coffee and Tea applications. You will be adding the following components to your Kubernetes Cluster: Coffee and Tea pods, matching coffee and tea services, and a Cafe VirtualServer. + +The Cafe application that you will deploy looks like the following diagram below. Coffee and Tea pods and services, with NGINX Ingress routing the traffic for /coffee and /tea routes, using the `cafe.example.com` Hostname. There is also a hidden third service - more on that later! + +< cafe diagram here > + +1. Inspect the `lab4/cafe.yaml` manifest. You will see we are deploying 3 replicas of each the coffee and tea Pods, and create a matching Service for each. + +1. Inspect the `lab4/cafe-vs.yaml` manifest. This is the VirtualServer CRD used by Nginx Ingress to expose these apps, using the `cafe.example.com` Hostname. + +1. Deploy the Cafe application by applying these two manifests: + +```bash +kubectl apply -f lab4/cafe.yaml +kubectl apply -f lab4/cafe-vs.yaml + +``` + +```bash +###Sample output### +deployment.apps/coffee created +service/coffee-svc created +deployment.apps/tea created +service/tea-svc created +virtualserver.k8s.nginx.org/cafe-vs created + +``` + +1. Check that all pods and services are running, you should see three Coffee and three Tea pods: + +```bash +kubectl get pods,svc +###Sample output### +NAME READY STATUS RESTARTS AGE +coffee-56b7b9b46f-9ks7w 1/1 Running 0 28s +coffee-56b7b9b46f-mp9gs 1/1 Running 0 28s +coffee-56b7b9b46f-v7xxp 1/1 Running 0 28s +tea-568647dfc7-54r7k 1/1 Running 0 27s +tea-568647dfc7-9h75w 1/1 Running 0 27s +tea-568647dfc7-zqtzq 1/1 Running 0 27s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/kubernetes ClusterIP 10.0.0.1
+ +**This completes Lab4.** + +
+ +## References: + +- [NGINX As A Service for Azure](https://docs.nginx.com/nginxaas/azure/) +- [NGINX Plus Product Page](https://docs.nginx.com/nginx/) +- [NGINX Ingress Controller](https://docs.nginx.com//nginx-ingress-controller/) +- [NGINX Directives Index](https://nginx.org/en/docs/dirindex.html) +- [NGINX Variables Index](https://nginx.org/en/docs/varindex.html) +- [NGINX Technical Specs](https://docs.nginx.com/nginx/technical-specs/) +- [NGINX - Join Community Slack](https://community.nginx.org/joinslack) + +
+ +### Authors + +- Chris Akker - Solutions Architect - Community and Alliances @ F5, Inc. +- Shouvik Dutta - Solutions Architect - Community and Alliances @ F5, Inc. +- Adam Currier - Solutions Architect - Community and Alliances @ F5, Inc. + +------------- + +Navigate to ([Lab5](../lab5/readme.md) | [LabX](../labX/readme.md)) diff --git a/labs/lab4/media/readme.md.old b/labs/lab4/media/readme.md.old deleted file mode 100644 index 27aa48e..0000000 --- a/labs/lab4/media/readme.md.old +++ /dev/null @@ -1,77 +0,0 @@ -# NGINX for Azure Overview and Deployment - -## Introduction - -In this lab, you will build ( x,y,x ). - -< Lab specific Images here, in the /media sub-folder > - -NGINX aaS | Docker -:-------------------------:|:-------------------------: - | - -## Learning Objectives - -By the end of the lab you will be able to: - -- Introduction to `xx` -- Build an `yyy` Nginx configuration -- Test access to your lab enviroment with Curl and Chrome -- Investigate `zzz` - - -## Pre-Requisites - -- You must have `aaaa` installed and running -- You must have `bbbbb` installed -- See `Lab0` for instructions on setting up your system for this Workshop -- Familiarity with basic Linux commands and commandline tools -- Familiarity with basic Docker concepts and commands -- Familiarity with basic HTTP protocol - -
- -### Lab exercise 1 - -
- -**This completes Lab2.** - -
- -## References: - -- [NGINX As A Service for Azure](https://docs.nginx.com/nginxaas/azure/) -- [NGINX Plus Product Page](https://docs.nginx.com/nginx/) -- [NGINX Ingress Controller](https://docs.nginx.com//nginx-ingress-controller/) -- [NGINX on Docker](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-docker/) -- [NGINX Directives Index](https://nginx.org/en/docs/dirindex.html) -- [NGINX Variables Index](https://nginx.org/en/docs/varindex.html) -- [NGINX Technical Specs](https://docs.nginx.com/nginx/technical-specs/) -- [NGINX - Join Community Slack](https://community.nginx.org/joinslack) - -
- -### Authors - -- Chris Akker - Solutions Architect - Community and Alliances @ F5, Inc. -- Shouvik Dutta - Solutions Architect - Community and Alliances @ F5, Inc. -- Adam Currier - Solutions Architect - Community and Alliances @ F5, Inc. - -------------- - -Navigate to ([Lab3](../lab3/readme.md) | [LabX](../labX/readme.md)) diff --git a/labs/lab4/media/redis-icon.png b/labs/lab4/media/redis-icon.png new file mode 100644 index 0000000..9d34aff Binary files /dev/null and b/labs/lab4/media/redis-icon.png differ diff --git a/ca-notes/aks/nic/nodeport-static.yaml b/labs/lab4/nodeport-static-redis.yaml similarity index 83% rename from ca-notes/aks/nic/nodeport-static.yaml rename to labs/lab4/nodeport-static-redis.yaml index e0d60d5..a23806f 100644 --- a/ca-notes/aks/nic/nodeport-static.yaml +++ b/labs/lab4/nodeport-static-redis.yaml @@ -1,3 +1,6 @@ +# Nginx 4 Azure, AKS2 NIC NodePort for Redis +# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 +# apiVersion: v1 kind: Service metadata: diff --git a/labs/lab4/readme.md b/labs/lab4/readme.md new file mode 100644 index 0000000..06f9c17 --- /dev/null +++ b/labs/lab4/readme.md @@ -0,0 +1,442 @@ +# Cafe Demo / Redis Deployment + +## Introduction + +In this lab, you will deploy the Nginx Cafe Demo, and Redis In Memory cache applications to your AKS Clusters. You will configure Nginx Ingress to expose these applications external to the Clusters. You will use the Nginx Plus Dashboard to watch the Ingress Resources. + +
+ +Nginx Ingress | Cafe | Redis +:--------------:|:--------------:|:--------------: + | | + +
+ +## Learning Objectives + +By the end of the lab you will be able to: + +- Deploy the Cafe Demo application +- Deploy the Redis In Memory Cache +- Expose the Cafe Demo app with NodePort +- Expose the Redis Cache with NodePort +- Monitor with Nginx Plus Ingress dashboard + +## Pre-Requisites + +- You must have both AKS Clusters up and running +- You must have both Nginx Ingress Controllers running +- You must have both the NIC Dashboards available +- Familiarity with basic Linux commands and commandline tools +- Familiarity with basic Kubernetes concepts and commands +- Familiarity with Kubernetes NodePort +- Familiarity with Nginx Ingress Controller CRDs - Custom Resource Definitions +- Familiartiy with Nginx Ingress VirtualServers and TransportServers +- See `Lab0` for instructions on setting up your system for this Workshop + +
+ +## Deploy the Nginx Cafe Demo app + + + +In this section, you will deploy the "Cafe Nginx" Ingress Demo, which represents a Coffee Shop website with Coffee and Tea applications. You will be adding the following components to your Kubernetes Clusters: + +- Coffee and Tea pods +- Matching coffee and tea services +- Cafe VirtualServer + +The Cafe application that you will deploy looks like the following diagram below. *BOTH* AKS clusters will have the Coffee and Tea pods and services, with NGINX Ingress routing the traffic for /coffee and /tea routes, using the `cafe.example.com` Hostname. There is also a third hidden service, more on that later! + + + +1. Inspect the `lab4/cafe.yaml` manifest. You will see we are deploying 3 replicas of each the coffee and tea Pods, and create a matching Service for each. + +2. Inspect the `lab4/cafe-vs.yaml` manifest. This is the Nginx Ingress VirtualServer CRD (Custom Resource Definition) used by Nginx Ingress to expose these apps, using the `cafe.example.com` Hostname. You will also see that active healthchecks are enabled, and the /coffee and /tea routes are being used. (NOTE: The VirtualServer CRD from Nginx is an `upgrade` to the standard Kubernetes Ingress object). + +3. Deploy the Cafe application by applying these two manifests in first cluster: + + > Make sure your Terminal is the `nginx-azure-workshops/labs` directory for all commands during this Workshop. + + ```bash + # Set context to 1st cluster(n4a-aks1) + kubectl config use-context n4a-aks1 + + kubectl apply -f lab4/cafe.yaml + kubectl apply -f lab4/cafe-vs.yaml + + ``` + + ```bash + ##Sample Output## + Switched to context "n4a-aks1". + deployment.apps/coffee created + service/coffee-svc created + deployment.apps/tea created + service/tea-svc created + virtualserver.k8s.nginx.org/cafe-vs created + ``` + +4. Check that all pods and services are running within first cluster, you should see three Coffee and three Tea pods, and the coffee-svc and tea-svc Services. + + ```bash + kubectl get pods,svc + ``` + + ```bash + ##Sample Output## + NAME READY STATUS RESTARTS AGE + coffee-56b7b9b46f-9ks7w 1/1 Running 0 28s + coffee-56b7b9b46f-mp9gs 1/1 Running 0 28s + coffee-56b7b9b46f-v7xxp 1/1 Running 0 28s + tea-568647dfc7-54r7k 1/1 Running 0 27s + tea-568647dfc7-9h75w 1/1 Running 0 27s + tea-568647dfc7-zqtzq 1/1 Running 0 27s + + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + service/kubernetes ClusterIP 10.0.0.1
+ +## Deploy Redis In Memory Caching in AKS Cluster 2 (n4a-aks2) + +Azure | Redis +:--------------:|:--------------: + |  + +
+ +In this exercise, you will deploy Redis in your second cluster (`n4a-aks2`), and use both Nginx Ingress and Nginx for Azure to expose this Redis Cache to the Internet. Similar to the Cafe Demo deployment, you will deploy: + +- `Redis Leader and Follower` pods and services in n4a-aks2 cluster. +- Add Nginx Ingress `Transport Server` for TCP traffic. +- Expose Redis with NodePorts. + +>**NOTE:** As Redis operates at the TCP level, you will be using the `Nginx stream` context in your Nginx Ingress configurations, not the HTTP context. + +### Deploy Redis Leader and Follower in AKS2 + +1. Inspect the Redis-Leader and Redis-Follower manifests. `Shout-out: Thank You to our friends at Google` for this sample Redis for Kubernetes configuration, it works well. You will see a single Leader Pod, and 2 Follower pods with matching services. + +1. Deploy Redis Leader and Follower to your AKS2 Cluster. + + ```bash + kubectl config use-context n4a-aks2 + kubectl apply -f lab4/redis-leader.yaml + kubectl apply -f lab4/redis-follower.yaml + ``` + + ```bash + ##Sample Output## + Switched to context "n4a-aks2". + deployment.apps/redis-leader created + service/redis-leader created + deployment.apps/redis-follower created + service/redis-follower created + ``` + +1. Check they are running: + + ```bash + kubectl get pods,svc -l app=redis + ``` + + ```bash + ##Sample Output## + NAME READY STATUS RESTARTS AGE + pod/redis-follower-847b67dd4f-f8ct5 1/1 Running 0 22h + pod/redis-follower-847b67dd4f-rt5hg 1/1 Running 0 22h + pod/redis-leader-58b566dc8b-8q55p 1/1 Running 0 22h + + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + service/redis-follower ClusterIP 10.0.222.46
+ +**This completes Lab4.** + +
+ +## References: + +- [NGINX As A Service for Azure](https://docs.nginx.com/nginxaas/azure/) +- [NGINX Cafe Demo](https://hub.docker.com/r/nginxinc/ingress-demo) +- [Redis Product Page](https://redis.io/) +- [Redis with Nginx Ingress Lab](https://github.com/nginxinc/nginx-ingress-workshops/tree/main/AdvancedNIC/labs/lab9) +- [NGINX Plus Product Page](https://docs.nginx.com/nginx/) +- [NGINX Ingress Controller](https://docs.nginx.com//nginx-ingress-controller/) +- [NGINX Ingress Transport Server CRD](https://docs.nginx.com/nginx-ingress-controller/configuration/transportserver-resource/) +- [NGINX Directives Index](https://nginx.org/en/docs/dirindex.html) +- [NGINX Variables Index](https://nginx.org/en/docs/varindex.html) +- [NGINX Technical Specs](https://docs.nginx.com/nginx/technical-specs/) +- [NGINX - Join Community Slack](https://community.nginx.org/joinslack) + +
+ +### Authors + +- Chris Akker - Solutions Architect - Community and Alliances @ F5, Inc. +- Shouvik Dutta - Solutions Architect - Community and Alliances @ F5, Inc. +- Adam Currier - Solutions Architect - Community and Alliances @ F5, Inc. + +------------- + +Navigate to ([Lab5](../lab5/readme.md) | [LabGuide](../readme.md)) diff --git a/ca-notes/aks/redis/redis-follower-ts.yaml b/labs/lab4/redis-follower-ts.yaml similarity index 92% rename from ca-notes/aks/redis/redis-follower-ts.yaml rename to labs/lab4/redis-follower-ts.yaml index b846333..6e7f559 100644 --- a/ca-notes/aks/redis/redis-follower-ts.yaml +++ b/labs/lab4/redis-follower-ts.yaml @@ -1,5 +1,5 @@ # NIC Plus TransportServer file -# Add ports 6380 for Redis Follower +# Add ports 6379 for Redis Follower # Chris Akker, Jan 2024 # apiVersion: k8s.nginx.org/v1alpha1 diff --git a/ca-notes/aks/redis/redis-follower.yaml b/labs/lab4/redis-follower.yaml similarity index 100% rename from ca-notes/aks/redis/redis-follower.yaml rename to labs/lab4/redis-follower.yaml diff --git a/ca-notes/aks/redis/redis-leader-ts.yaml b/labs/lab4/redis-leader-ts.yaml similarity index 100% rename from ca-notes/aks/redis/redis-leader-ts.yaml rename to labs/lab4/redis-leader-ts.yaml diff --git a/ca-notes/aks/redis/redis-leader.yaml b/labs/lab4/redis-leader.yaml similarity index 100% rename from ca-notes/aks/redis/redis-leader.yaml rename to labs/lab4/redis-leader.yaml diff --git a/labs/lab5/aks1-upstreams.conf b/labs/lab5/aks1-upstreams.conf index 4497ea9..4791c22 100644 --- a/labs/lab5/aks1-upstreams.conf +++ b/labs/lab5/aks1-upstreams.conf @@ -8,11 +8,12 @@ upstream aks1_ingress { least_time last_byte; - # from nginx-ingress NodePort Service / aks Node names + # from nginx-ingress NodePort Service / aks1 Node names # Note: change servers to match # - server aks-userpool-76919110-vmss000002:32080; #aks1 node1: - server aks-userpool-76919110-vmss000003:32080; #aks1 node2: + server aks-userpool-76919110-vmss000001:32080; #aks1 node1 + server aks-userpool-76919110-vmss000002:32080; #aks1 node2 + server aks-userpool-76919110-vmss000003:32080; #aks1 node3 keepalive 32; diff --git a/labs/lab5/aks2-nic-headless.conf b/labs/lab5/aks2-nic-headless.conf new file mode 100644 index 0000000..f333f91 --- /dev/null +++ b/labs/lab5/aks2-nic-headless.conf @@ -0,0 +1,24 @@ +# Nginx 4 Azure direct to NIC for Upstreams +# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 +# +# direct to nginx ingress Headless Service ( no NodePort ) +# +upstream aks2_nic_headless { + zone aks2_nic_headless 256k; + + least_time last_byte; + + # direct to nginx-ingress Headless Service Endpoint Cluster IP + # Resolvers set to kube-dns Endpoints List + + resolver 172.16.4.64 172.16.4.224 valid=10s ipv6=off status_zone=kube-dns; + + # Server name must follow this Kubernetes Service Name format + # server
-< Lab specific Image here > + -
### Nginx 4 Azure Proxy to AKS Clusters -This exercise will create Nginx Upstream configurations for the AKS clusters. You will add the NodePorts of the Nginx Ingress Controllers running in AKS cluster 1, and AKS Cluster 2. These were previously deployed and configured in a previous lab. Now the fun part, sending traffic to them! +This exercise will create Nginx Upstream configurations for the AKS Clusters. You will use the Nodepool node names, and you will add the port number `32080` from the NodePort of the Nginx Ingress Controllers running in AKS cluster 1, and AKS Cluster 2. These were previously deployed and configured in a previous lab. Now the fun part, sending traffic to them! -Using the Nginx4Azure configuration tool, create a new file called `/etc/nginx/conf.d/aks1-upstreams.conf`. Copy and Paste the contents of the provided file. You will have to EDIT this example config file, and change the `server` entries to match your AKS Cluster1 nodepool node names. You can find your AKS1 nodepool nodenames from the Azure Portal. Make sure you use `:32080` for the port number, this is the static `nginx-ingress NodePort Service` for HTTP traffic that was defined earlier. +Configure the Upstream for AKS Cluster1. -```nginx +1. Using kubectl, get the Nodepool nodes for AKS Cluster1: (You can also find these in your Azure Portal - AKS Nodepool definitions.) -# Nginx 4 Azure to NIC, AKS Nodes for Upstreams -# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 -# -# AKS1 nginx ingress upstreams -# -upstream aks1_ingress { - zone aks1_ingress 256k; + ```bash + kubectl config use-context n4a-aks1 + kubectl get nodes + ``` - least_time last_byte; - - # from nginx-ingress NodePort Service / aks Node names - # Note: change servers to match - # - server aks-userpool-76919110-vmss000002:32080; #aks1 node1: - server aks-userpool-76919110-vmss000003:32080; #aks1 node2: + ```bash + ##Sample Output## + aks-userpool-76919110-vmss000001 #aks1 node1 + aks-userpool-76919110-vmss000002 #aks1 node2 + aks-userpool-76919110-vmss000003 #aks1 node3 + ``` - keepalive 32; +1. Using the Nginx4Azure configuration tool, create a new file called `/etc/nginx/conf.d/aks1-upstreams.conf`. Copy and Paste the contents of the provided file. You will have to EDIT this example config file, and change the `server` entries to match your AKS Cluster1 Nodepool node names. You can find your AKS1 nodepool nodenames from `kubectl get nodes` or the Azure Portal. Make sure you use `:32080` for the port number, this is the static `nginx-ingress NodePort Service` for HTTP traffic that was defined earlier. -} + ```nginx + # Nginx 4 Azure to NIC, AKS Nodes for Upstreams + # Chris Akker, Shouvik Dutta, Adam Currier, Steve Wagner - Mar 2024 + # + # AKS1 nginx ingress upstreams + # + upstream aks1_ingress { + zone aks1_ingress 256k; -``` + least_time last_byte; + + # from nginx-ingress NodePort Service / aks Node names + # Note: change servers to match + # + server aks-userpool-76919110-vmss000001:32080; #aks1 node1 + server aks-userpool-76919110-vmss000002:32080; #aks1 node2 + server aks-userpool-76919110-vmss000003:32080; #aks1 node3 -Submit your Changes. If you have the Server names:port correct, Nginx4Azure will validate and return a Success message. + keepalive 32; -**Important!** If you stop then re-start your AKS cluster, or scale up/down, or add/remove VMSS worker nodes in the AKS NodePools, this Upstream list `WILL` have to be updated to match! Any changes to the Worker nodes in the Cluster will need to be matched exactly, as it is a static configuration that must match the Worker Nodes:NodePort definition in your AKS cluster. If you change the static nginx-ingress NodePort Service, you will have to match it here as well. + } + ``` -Repeat the step above, but create a new file called `/etc/nginx/conf.d/aks2-upstreams.conf`, for your second, AKS2 Cluster: + Submit your Nginx Configuration. If you have the Server names:port correct, Nginx for Azure will validate and return a Success message. -```nginx -# Nginx 4 Azure to NIC, AKS Node for Upstreams -# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 -# -# AKS2 nginx ingress upstreams -# -upstream aks2_ingress { - zone aks2_ingress 256k; + >**Important!** If you stop then re-start your AKS cluster, or scale the Nodepool up/down, or add/remove nodes in the AKS NodePools, this Upstream list `WILL have to be updated to match!`. Any changes to the Worker nodes in the Cluster will need to be matched exactly, as it is a static Nginx configuration that must match the Kubernetes workers - Nodes:NodePort definition in your AKS cluster. If you change the static nginx-ingress NodePort Service Port number, you will have to match it here as well. - least_time last_byte; - - # from nginx-ingress NodePort Service / aks Node names - # Note: change servers to match - # - server aks-nodepool1-19485366-vmss00000h:32080; #aks node1: - server aks-nodepool1-19485366-vmss00000i:32080; #aks node2: - server aks-nodepool1-19485366-vmss00000j:32080; #aks node3: + *Currently, there is no auto-magic way to synchronize the Nginx for Azure upstream list with the AKS node list, but don't worry - Nginx Devs are working on that!* - keepalive 32; +Configure the Upstream for AKS Cluster2. -} +1. Repeat the previous Step, using the Nginx4Azure configuration tool, create a new file called `/etc/nginx/conf.d/aks2-upstreams.conf`. Copy and Paste the contents of the provided file. You will have to EDIT this example config file and change the `server` entries to match your AKS Cluster2 Nodepool node names. You can find your AKS2 nodepool nodenames from `kubectl get nodes` or the Azure Portal. Make sure you use `:32080` for the port number; this is the static `nginx-ingress NodePort Service` for HTTP traffic that was defined earlier. -``` +1. Using kubectl, get the Nodepool nodes for AKS Cluster2: (You can also find these in your Azure Portal - AKS Nodepool definitions.) -Note, there are 3 upstreams, matching the 3 Worker Nodes in AKS2 cluster. + ```bash + kubectl config use-context n4a-aks2 + kubectl get nodes + ``` -Submit your Changes. If you have the Server name:port correct, Nginx4Azure will validate and return a Success message. + ```bash + ##Sample Output## + aks-nodepool1-19485366-vmss000003 #aks2 node1 + aks-nodepool1-19485366-vmss000004 #aks2 node2 + aks-nodepool1-19485366-vmss000005 #aks2 node3 + aks-nodepool1-19485366-vmss000006 #aks2 node4 + ``` -**Warning:** If you stop and start your AKS cluster, or add/remove Nodes in the Pools, this Upstream list `WILL` have to be updated to match. It is a static configuration that must match the Worker Nodes:NodePort definition in your AKS cluster. If you change the static nginx-ingress NodePort Service, you will have to match it here as well. Unfortunately, there are no auto-magic way to synchronize AKS/NodePorts with N4A Upstreams... yet :-) + ```nginx + # Nginx 4 Azure to NIC, AKS Node for Upstreams + # Chris Akker, Shouvik Dutta, Adam Currier, Steve Wagner - Mar 2024 + # + # AKS2 nginx ingress upstreams + # + upstream aks2_ingress { + zone aks2_ingress 256k; -### Test Nginx 4 Azure to AKS1 Cluster Ingress Controller + least_time last_byte; + + # from nginx-ingress NodePort Service / aks Node names + # Note: change servers to match + # + server aks-nodepool1-19485366-vmss000003:32080; #aks2 node1 + server aks-nodepool1-19485366-vmss000004:32080; #aks2 node2 + server aks-nodepool1-19485366-vmss000005:32080; #aks2 node3 + server aks-nodepool1-19485366-vmss000006:32080; #aks2 node4 -Now that you have these new Nginx Upstream blocks created, you can test them. + keepalive 32; -Inspect, then modify the `# comments for proxy_pass` in the `location /` block in the `/etc/nginx/conf.d/cafe.example.com.conf` file, to disable the proxy_pass to `cafe-nginx`, and enable the proxy_pass to `aks1_ingress`, as shown: + } + ``` -```nginx -... +Note that there are 4 upstreams, matching the 4 Nodepool nodes in AKS2 cluster. - location / { - # - # return 200 "You have reached cafe.example.com, location /\n"; - - # proxy_pass http://cafe_nginx; # Proxy AND load balance to a list of servers - # proxy_pass http://vm1:32779; # Proxy to another server - # proxy_pass http://nginx.org; # Proxy to another website - - proxy_pass http://aks1_ingress; # Proxy to AKS1 Nginx Ingress Controller NodePort - add_header X-Proxy-Pass aks1_ingress; # Custom Header +Submit your Nginx Configuration. If you have the Server name:port correct, Nginx4Azure will validate and return a Success message. + +**Warning:** The same warning applies to Upstream configuration for AKS2, *if you make any Nodepool changes, Nginx must be updated to match those changes.* + +### Update Nginx Config and add HTTP/1.1 Keepalive + +In order for Nginx 4 Azure and Nginx Ingress to work correctly, the HTTP Host Headers, and perhaps other headers, will need to be passed. This is done by changing the HTTP Version to 1.1, so that the Host Header can be included. + +1. Inspect the `lab5/keepalive.conf`. This is where the HTTP Protocol and Headers are set for proxied traffic. This is a common requirement so it is shared among all the different Nginx configurations. + + Using the Nginx for Azure console, create a new file, `/etc/nginx/includes/keepalive.conf`. Use the example provided, just copy/paste. + + ```nginx + # Nginx 4 Azure - Mar 2024 + # Chris Akker, Shouvik Dutta, Adam Currier, Steve Wagner - Mar 2024 + # + # Default is HTTP/1.0 to upstreams, keepalives is only enabled for HTTP/1.1 + proxy_http_version 1.1; + + # Set the Connection header to empty + proxy_set_header Connection ""; + + # Host request header field, or the server name matching a request + proxy_set_header Host $host; + ``` + + Submit your Nginx Configuration. + +1. Inspect the `lab5/nginx.conf` file. Uncomment the `include` directive near the bottom, as shown: + + ```nginx + # Nginx 4 Azure - Default - Updated Nginx.conf + # Chris Akker, Shouvik Dutta, Adam Currier, Steve Wagner - Mar 2024 + # + user nginx; + worker_processes auto; + worker_rlimit_nofile 8192; + pid /run/nginx/nginx.pid; + + events { + worker_connections 4000; + } + + error_log /var/log/nginx/error.log error; + + http { + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log off; + server_tokens ""; + server { + listen 80 default_server; + server_name localhost; + location / { + # Points to a directory with a basic html index file with + # a "Welcome to NGINX as a Service for Azure!" page + root /var/www; + index index.html; + } + } + + include /etc/nginx/conf.d/*.conf; + include /etc/nginx/includes/*.conf; # shared files + + } + + # stream { - # proxy_pass http://aks2_ingress; # Proxy to AKS2 Nginx Ingress Controller NodePort - # proxy_pass http://aks1_nic_direct; # Proxy to AKS Nginx Ingress Controller Direct - # proxy_pass http://$upstream; # Use Split Clients config + # include /etc/nginx/stream/*.conf; # Stream TCP nginx files + + # } + ``` + +Submit your Nginx Configuration. + +### Test Nginx 4 Azure to AKS1 Cluster Nginx Ingress Controller + +Now that you have these new Nginx Upstream blocks created, you can test them. + +1. Inspect, then modify the `# comments for proxy_pass` in the `location /` block in the `/etc/nginx/conf.d/cafe.example.com.conf` file, making these changes as shown. + + - Disable the proxy_pass to `cafe-nginx` + - Enable the proxy_pass to `aks1_ingress` + - Change the comments for the X-Proxy-Pass Header as well. + + ```nginx + ... + + location / { + # + # return 200 "You have reached cafe.example.com, location /\n"; + # proxy_pass http://cafe_nginx; # Proxy AND load balance to a list of servers + # add_header X-Proxy-Pass cafe_nginx; # Custom Header + + proxy_pass http://aks1_ingress; # Proxy to AKS1 Nginx Ingress Controller NodePort + add_header X-Proxy-Pass aks1_ingress; # Custom Header + + # proxy_pass http://aks2_ingress; # Proxy to AKS2 Nginx Ingress Controller NodePort + # add_header X-Proxy-Pass aks2_ingress; # Custom Header + + } + ... + ``` + + This changes where Nginx will `proxy_pass` the requests. Nginx will now proxy and load balance requests to your AKS1 Nginx Ingress Controller, listening on port 32080 on each AKS1 Node. The X-Proxy-Pass Header will now show `aks1-ingress` instead of cafe-nginx. + + Submit your Nginx Configuration. + +1. Test your change with curl. Do you see the X-Proxy-Pass Header that you added, so you know which Upstream block is being used ? + + ```bash + curl -I http://cafe.example.com/coffee + ``` + + ```bash + ##Sample Output## + HTTP/1.1 200 OK + Server: N4A-1.25.1-cakker + Date: Fri, 05 Apr 2024 20:08:24 GMT + Content-Type: text/html; charset=utf-8 + Connection: keep-alive + Expires: Fri, 05 Apr 2024 20:08:23 GMT + Cache-Control: no-cache + X-Proxy-Pass: aks1_ingress + ``` + +1. Test your change in proxy_pass with Chrome at http://cafe.example.com/coffee, hitting Refresh several times - what do you see - Docker Containers or AKS1 Pods? + + Cafe Docker | Cafe AKS1 + :-----------------:|:-----------------: +  |  + +1. Verify this with `kubectl`. Set your Kubectl Config Context to n4a-aks1: + + ```bash + kubectl config use-context n4a-aks1 + kubectl get pods + ``` + + ```bash + ##Sample Output## + NAME READY STATUS RESTARTS AGE + coffee-869854dd6-bs8t6 1/1 Running 0 3d3h + coffee-869854dd6-wq9lp 1/1 Running 0 3d3h + ... + ``` + + Notice the Names of the `coffee` pods. For the coffee Pod IPs, check the `coffee-svc` Endpoints: + + ```bash + kubectl describe svc coffee-svc + ``` + + ```bash + ##Sample Output## + Name: coffee-svc + Namespace: default + Labels:
+As you have seen, using Nginx for Azure is quite easy to create various backend Systems, Services, platforms of different types and have Nginx Load Balance them through a single entry point. Using Advanced Nginx directives/configs with Resolver, Nginx Ingress Controllers, Headless, and even Split Clients help you control and manage dev/test/pre-prod and even Production workloads with ease. Dashboards and Monitoring give you insight with over 240 useful metrics, providing data needed for decisions based on both real time and historical metadata about your Apps and Traffic. **This completes Lab5.** -
## References: @@ -278,14 +1275,14 @@ During this Lab exercise, you created and tested THREE different Upstream config - [NGINX Technical Specs](https://docs.nginx.com/nginx/technical-specs/) - [NGINX - Join Community Slack](https://community.nginx.org/joinslack) -
### Authors - Chris Akker - Solutions Architect - Community and Alliances @ F5, Inc. - Shouvik Dutta - Solutions Architect - Community and Alliances @ F5, Inc. - Adam Currier - Solutions Architect - Community and Alliances @ F5, Inc. +- Steve Wagner - Solutions Architect - Community and Alliances @ F5, Inc. ------------- -Navigate to ([Lab6](../lab6/readme.md) | [LabX](../labX/readme.md)) +Navigate to ([Lab6](../lab6/readme.md) | [LabGuide](../readme.md)) diff --git a/labs/lab5/redis-leader-upstreams.conf b/labs/lab5/redis-leader-upstreams.conf new file mode 100644 index 0000000..91fa8d4 --- /dev/null +++ b/labs/lab5/redis-leader-upstreams.conf @@ -0,0 +1,16 @@ +# Nginx 4 Azure to NIC, AKS Node for Upstreams +# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 +# +# nginx ingress upstreams for Redis Leader +# +upstream aks2_redis_leader { + zone aks2_redis_leader 256k; + + least_time last_byte; + + # from nginx-ingress NodePort Service / aks Node IPs + server aks-nodepool1-19485366-vmss000003:32379; #aks2 node1: + server aks-nodepool1-19485366-vmss000004:32379; #aks2 node2: + server aks-nodepool1-19485366-vmss000005:32379; #aks2 node3: + +} diff --git a/ca-notes/n4a-configs/stream/redis-leader.conf b/labs/lab5/redis.example.com.conf similarity index 56% rename from ca-notes/n4a-configs/stream/redis-leader.conf rename to labs/lab5/redis.example.com.conf index c0387d4..79185c0 100644 --- a/ca-notes/n4a-configs/stream/redis-leader.conf +++ b/labs/lab5/redis.example.com.conf @@ -1,12 +1,11 @@ # Nginx 4 Azure to NIC, AKS Node for Upstreams +# Stream for Redis Leader # Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 # -# nginx ingress upstreams for Redis Leader -# server { - listen 6379; - status_zone redis-leader; + listen 6379; # Standard Redis Port + status_zone aks2-redis-leader; proxy_pass aks2_redis_leader; diff --git a/ca-notes/n4a-configs/includes/split-clients.conf b/labs/lab5/split-clients.conf similarity index 100% rename from ca-notes/n4a-configs/includes/split-clients.conf rename to labs/lab5/split-clients.conf diff --git a/labs/lab6/media/add_certificate.png b/labs/lab6/media/add_certificate.png deleted file mode 100644 index 1374ad7..0000000 Binary files a/labs/lab6/media/add_certificate.png and /dev/null differ diff --git a/labs/lab6/media/add_certificate_sucess.png b/labs/lab6/media/add_certificate_sucess.png deleted file mode 100644 index a16605b..0000000 Binary files a/labs/lab6/media/add_certificate_sucess.png and /dev/null differ diff --git a/labs/lab6/media/docker-icon.png b/labs/lab6/media/docker-icon.png new file mode 100644 index 0000000..02ee3f1 Binary files /dev/null and b/labs/lab6/media/docker-icon.png differ diff --git a/labs/lab6/media/keyvault_screen.png b/labs/lab6/media/keyvault_screen.png deleted file mode 100644 index 27572fd..0000000 Binary files a/labs/lab6/media/keyvault_screen.png and /dev/null differ diff --git a/labs/lab6/media/lab6_cafe_access_log_update.png b/labs/lab6/media/lab6_cafe_access_log_update.png new file mode 100644 index 0000000..dc130ee Binary files /dev/null and b/labs/lab6/media/lab6_cafe_access_log_update.png differ diff --git a/labs/lab6/media/lab6_cafe_query.png b/labs/lab6/media/lab6_cafe_query.png new file mode 100644 index 0000000..b83b0b4 Binary files /dev/null and b/labs/lab6/media/lab6_cafe_query.png differ diff --git a/labs/lab6/media/lab6_cafe_query_details.png b/labs/lab6/media/lab6_cafe_query_details.png new file mode 100644 index 0000000..09ad979 Binary files /dev/null and b/labs/lab6/media/lab6_cafe_query_details.png differ diff --git a/labs/lab6/media/lab6_cafe_query_save.png b/labs/lab6/media/lab6_cafe_query_save.png new file mode 100644 index 0000000..6571476 Binary files /dev/null and b/labs/lab6/media/lab6_cafe_query_save.png differ diff --git a/labs/lab6/media/lab6_create_dashboard.png b/labs/lab6/media/lab6_create_dashboard.png new file mode 100644 index 0000000..9ec79ed Binary files /dev/null and b/labs/lab6/media/lab6_create_dashboard.png differ diff --git a/labs/lab6/media/lab6_default_chart.png b/labs/lab6/media/lab6_default_chart.png new file mode 100644 index 0000000..9cf06bd Binary files /dev/null and b/labs/lab6/media/lab6_default_chart.png differ diff --git a/labs/lab6/media/lab6_default_query.png b/labs/lab6/media/lab6_default_query.png new file mode 100644 index 0000000..590602d Binary files /dev/null and b/labs/lab6/media/lab6_default_query.png differ diff --git a/labs/lab6/media/lab6_main_access_log_update.png b/labs/lab6/media/lab6_main_access_log_update.png new file mode 100644 index 0000000..7225b7d Binary files /dev/null and b/labs/lab6/media/lab6_main_access_log_update.png differ diff --git a/labs/lab6/media/lab6_main_ext_logformat_add.png b/labs/lab6/media/lab6_main_ext_logformat_add.png new file mode 100644 index 0000000..97e5380 Binary files /dev/null and b/labs/lab6/media/lab6_main_ext_logformat_add.png differ diff --git a/labs/lab6/media/lab6_nginx_conf_editor.png b/labs/lab6/media/lab6_nginx_conf_editor.png new file mode 100644 index 0000000..616f48e Binary files /dev/null and b/labs/lab6/media/lab6_nginx_conf_editor.png differ diff --git a/labs/lab6/media/lab6_pin_upstream_chart.png b/labs/lab6/media/lab6_pin_upstream_chart.png new file mode 100644 index 0000000..5d0a48c Binary files /dev/null and b/labs/lab6/media/lab6_pin_upstream_chart.png differ diff --git a/labs/lab6/media/lab6_server_request_chart.png b/labs/lab6/media/lab6_server_request_chart.png new file mode 100644 index 0000000..bcfbf65 Binary files /dev/null and b/labs/lab6/media/lab6_server_request_chart.png differ diff --git a/labs/lab6/media/lab6_show_dashboard.png b/labs/lab6/media/lab6_show_dashboard.png new file mode 100644 index 0000000..1da2f59 Binary files /dev/null and b/labs/lab6/media/lab6_show_dashboard.png differ diff --git a/labs/lab6/media/lab6_upstream_chart_dashboard.png b/labs/lab6/media/lab6_upstream_chart_dashboard.png new file mode 100644 index 0000000..282d892 Binary files /dev/null and b/labs/lab6/media/lab6_upstream_chart_dashboard.png differ diff --git a/labs/lab6/media/lab6_upstream_response_time_chart.png b/labs/lab6/media/lab6_upstream_response_time_chart.png new file mode 100644 index 0000000..b4bf7fb Binary files /dev/null and b/labs/lab6/media/lab6_upstream_response_time_chart.png differ diff --git a/labs/lab6/media/nginx-azure-icon.png b/labs/lab6/media/nginx-azure-icon.png new file mode 100644 index 0000000..70ab132 Binary files /dev/null and b/labs/lab6/media/nginx-azure-icon.png differ diff --git a/labs/lab6/media/nginx4a_logs.png b/labs/lab6/media/nginx4a_logs.png new file mode 100644 index 0000000..638ab58 Binary files /dev/null and b/labs/lab6/media/nginx4a_logs.png differ diff --git a/labs/lab6/readme.md b/labs/lab6/readme.md index 12c2132..dd30e4b 100644 --- a/labs/lab6/readme.md +++ b/labs/lab6/readme.md @@ -1,10 +1,8 @@ -# Azure Key Vault / TLS Essentials +# Azure Montoring / Logging Analytics ## Introduction -In this lab, you will create a new key-vault resource that would be storing self-signed certificates. You will then configure Nginx for Azure to listen for https traffic and then terminate TLS before proxying and load balancing back to the backend system. - -< Lab specific Images here, in the /media sub-folder > +In this lab, you will explore Azure based monitoring and Logging capabilities. You will create the basic access log_format within NGINX for Azure resource. As the basic log_format only contains a fraction of the information, you will then extend it and create a new log_format to include much more information, especially about the Upstream backend servers. You will add access logging to your NGINX for Azure resource and finally capture/see those logs within Azure monitoring tools. NGINX aaS | Docker :-------------------------:|:-------------------------: @@ -14,196 +12,222 @@ NGINX aaS | Docker By the end of the lab you will be able to: -- Build your own Azure Key Vault resource. -- Create your self-signed TLS certificate. -- Configure NGINX for Azure to listen for HTTPS traffic -- Test and validate TLS traffic components and settings +- Enable basic log format within NGINX for Azure resource + +- Create enhance log format with additional logging metrics + +- Test access logs within log analytics workspace + +- Explore Azure Monitoring for NGINX for Azure ## Pre-Requisites -- You must have your Nginx for Azure resource up and running -- You must have `Owner` role on the resource group that includes NGINX for Azure resource -- You must also have backend system resources up and running. -- See `Lab0` for instructions on setting up your system for this Workshop +- Within your NGINX for Azure resource, you must have enabled sending metrics to Azure monitor. + +- You must have created `Log Analytics workspace`. +- You must have created an Azure diagnostic settings resource that will stream the NGINX logs to the Log Analytics workspace. +- See `Lab1` for instructions if you missed any of the above steps. -### Create Azure Key Vault resource +
-1. Create an Azure key vault within the same resource group which holds your NGINX for azure resource. +### Enable basic log format - ```bash - ## Set environment variable - MY_RESOURCEGROUP=s.dutta-workshop +1. Within Azure portal, open your resource group and then open your NGINX for Azure resource (nginx4a). From the left pane click on `Settings > NGINX Configuration`. This should open the configuration editor section. Open `nginx.conf` file. + +  + +1. You will notice in previous labs, you have added the default basic log format inside the `http` block within the `nginx.conf` file as highlighted in above screenshot. You will make use of this log format initially to capture some useful metrics within NGINX logs. + + ```nginx + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; ``` - Once the environment variables are all set, run below command to create the key vault resource +1. Update the `access_log` directive to enable logging. Within this directive, you will pass the full path of the log file (eg. `/var/log/nginx/access.log`) and also the `main` log format that you created in previous step. Click on `Submit` to apply the changes. - ```bash - az keyvault create \ - --resource-group $MY_RESOURCEGROUP \ - --name n4a-keyvault \ - --enable-rbac-authorization false + ```nginx + access_log /var/log/nginx/access.log main; ``` -2. The above command should provide a json output. If you look at its content then it should have a `provisioningState` key with `Succeeded` as it value. This field is an easy way to validate the command successfully provisioned the resource. +  -3. Next you would provide permissions to access this keyvault to the user assigned managed identity that you created while creating NGINX for Azure resource. -4. Copy the `PrincipalID` of the user identity into an environment variable using below command. +1. In subsequent sections you will test out the logs inside log analytics workspace. - ```bash - ## Set environment variable - MY_PRINCIPALID=$(az identity show \ - --resource-group $MY_RESOURCEGROUP \ - --name n4a-useridentity \ - --query principalId \ - --output tsv) +### Create enhance log format with additional logging metrics + +In this section you will create an extended log format which you will use with `cafe.example.com` server's access log. + +1. Within the NGINX for Azure resource (nginx4a), open the `Settings > NGINX Configuration` pane. + +1. Within the `nginx.conf` file add a new extended log format named `main_ext` as shown in the below screenshot. Click on `Submit` to save the config file + + ```nginx + # Extended Log Format + log_format main_ext 'remote_addr="$remote_addr", ' + '[time_local=$time_local], ' + 'request="$request", ' + 'status="$status", ' + 'http_referer="$http_referer", ' + 'body_bytes_sent="$body_bytes_sent", ' + 'Host="$host", ' + 'sn="$server_name", ' + 'request_time=$request_time, ' + 'http_user_agent="$http_user_agent", ' + 'http_x_forwarded_for="$http_x_forwarded_for", ' + 'request_length="$request_length", ' + 'upstream_address="$upstream_addr", ' + 'upstream_status="$upstream_status", ' + 'upstream_connect_time="$upstream_connect_time", ' + 'upstream_header_time="$upstream_header_time", ' + 'upstream_response_time="$upstream_response_time", ' + 'upstream_response_length="$upstream_response_length", '; ``` -5. Now assign GET secrets and GET certificates permission to this user assigned managed identity for your keyvault using below command. +  - ```bash - az keyvault set-policy \ - --name n4a-keyvault \ - --certificate-permissions get \ - --secret-permissions get \ - --object-id $MY_PRINCIPALID +1. Once the extended log format has been created, open `cafe.example.com.conf` file and update the `access_log` to make use of the extended log format as shown in the below screenshot. Click on `Submit` to apply the changes. + + ```nginx + access_log /var/log/nginx/cafe.example.com.log main_ext; ``` -### Create a self-signed TLS certificate +  + +1. In subsequent sections you will test out the extended log format within inside log analytics workspace. + +### Test the access logs within log analytics workspace -1. In this section, you will create a self-signed certificate using the Azure CLI. +1. To test out access logs, generate some traffic on your `cafe.example.com` server. - **NOTE:** It should be clearly understood, that Self-signed certificates are exactly what the name suggest - they are created and signed by you or someone else. **They are not signed by any official Certificate Authority**, so they are not recommended for any use other than testing in lab exercises within this workshop. Most Modern Internet Browsers will display Security Warnings when they receive a Self-Signed certificate from a webserver. In some environments, the Browser may actually block access completely. So use Self-signed certificates with **CAUTION**. +1. You can generate some traffic using your local Docker Desktop. Start and run the `WRK` load generation tool from a container using below command to generate traffic: -2. Create a self-signed certificate by running the below command. + First save your NGINX for Azure resource public IP in a environment variable. ```bash - az keyvault certificate create \ - --vault-name n4a-keyvault \ - --name n4a-cert \ - --policy @labs/lab6/self-certificate-policy.json + ## Set environment variables + export MY_RESOURCEGROUP=s.dutta-workshop + export MY_N4A_IP=$(az network public-ip show \ + --resource-group $MY_RESOURCEGROUP \ + --name n4a-publicIP \ + --query ipAddress \ + --output tsv) ``` -3. The above command should provide a json output. If you look at its content then it should have a `status` key with `completed` as it value. This field is an easy way to validate the command successfully created the certificate. + Make request to the default server block which is using the `main` log format for access logging by running below command. -4. Now log into Azure portal and navigate to your resource-group and then click on the `n4a-keyvault` key vault resource. + ```bash + docker run --name wrk --rm williamyeh/wrk -t4 -c200 -d1m --timeout 2s http://$MY_N4A_IP + ``` -5. Within the keyvault resources window, click on `Certificates` from the left pane. You should see a self-signed certificate named `n4a-cert` within the certificates pane. -  + Make request to the `cafe.example.com` server block which is using the `main_ext` log format for access logging by running below command. -6. Click on the newly created certificate and then open up `Issuance Policy` tab for more details on the certificate. You will use this certificate with NGINX for Azure resource to listen for HTTPS traffic. + ```bash + docker run --name wrk --rm williamyeh/wrk -t4 -c200 -d1m --timeout 2s -H 'Host: cafe.example.com' http://$MY_N4A_IP/coffee + ``` -### Configure NGINX for Azure to listen for HTTPS traffic +1. Within Azure portal, open your NGINX for Azure resource (nginx4a). From the left pane click on `Monitoring > Logs`. This should open a new Qeury pane. Select `Resource type` from drop down and then type in `nginx` in the search box. This should show all the sample queries related to NGINX for Azure. Under `Show NGINXaaS access logs` click on `Run` button -Now that you have a self signed TLS certificate for testing, you will configure NGINX for Azure resource to use them. +  -1. Within your resource-group, click on the NGINX for Azure resource (`nginx4a`). +1. This should open a `new query` window, which is made up of a query editor pane at the top and query result pane at the bottom as shown in below screenshot. -1. From the left pane, click on `NGINX certificates` under `Settings` and then click on the `+ Add certificate` button to add your self signed certificate that you created in previous section. +  -  + > **NOTE:** The logs may take couple of minutes to show up. If the results pane doesn't show the logs then wait for a minute and then click on the `Run` button to run the query again. -1. Within the `Add Certificate` pane, fill in below details: - - **Preferred name:** Any unique name for the certificate (eg. n4a-cert) - - **Certificate path:** Logical path where the certificate would recide. (eg. /etc/nginx/cert/n4a-cert.cert) - - **Key path:** Logical path where the key would recide. (eg. /etc/nginx/cert/n4a-cert.key) - - **Key vault:** Select your key vault (eg. n4a-keyvault) - - **Certificate name:** Select a certificate (eg. n4a-cert) - - Once all the fields have been filled, click on `Save` to save the certificate within NGINX for Azure. -  +1. Azure makes use of Kusto Query Language(KQL) to query logs. Have a look in the [references](#references) section to learn more about KQL. -1. You should see your certificate in a `Succeeded` status if the values that you entered in previous step was all correct. +1. You will modify the default query to show logs for `cafe.example.com` server block. Update the default query with the below query in the query editor pane. Click on the `Run` button to execute the query. -  + ```kql + // Show NGINXaaS access logs + // A list of access logs sorted by time. + NGXOperationLogs + | where FilePath == "/var/log/nginx/cafe.example.com.log" + | sort by TimeGenerated desc + | project TimeGenerated, FilePath, Message + | limit 100 + ``` -1. Now you will modify your `cafe.example.com.conf` file that you created in `lab2` to set up cafe.example.com as a HTTPS server. First you will add the `ssl` parameter to the `listen` directive in the `server` block. You will then specify the server certificate and private key file within the configuration to point to the certificate that you added in previous steps. +  -1. Open `lab6/cafe.example.com.conf`. Below is the list of changes that you can observe which has changed from `lab2/cafe.example.com.conf` file to enable HTTPS traffic on cafe.example.com. - - On line #6, the listen port has been updated from port 80 to 443. Also `ssl` parameter has been added to enable TLS termination for this `server` block. - - On line #11-12, the `ssl_certificate` and `ssl_certificate_key` directives have been added and points to the certificate path that you provided when you added certificate to the NGINX for Azure resource. - - ```nginx - server { - - listen 443 ssl; # Listening on port 443 with "ssl" parameter for terminating TLS on all IP addresses on this machine +1. Within the Results pane, expand one of the logs to look into its details. You can also hover your mouse over the message to show the message details as shown in below screenshot. Note that the message follows the `main_ext` log format. + +  - server_name cafe.example.com; # Set hostname to match in request - status_zone cafe.example.com; # Metrics zone name +1. You can save the custom query if you wish by clicking on the `Save` button and then selecting `Save as query`. Within the `Save as query` pane provide a query name and optional description and then finally click on `Save` button. - ssl_certificate /etc/nginx/certs/n4a-cert.cert; - ssl_certificate_key /etc/nginx/certs/n4a-cert.key; +  - snip... - } +### Explore Azure Monitoring for NGINX for Azure + +1. Generate some steady traffic using your local Docker Desktop. Start and run the `WRK` load generation tool from a container using below command to generate traffic: + + ```bash + docker run --name wrk --rm williamyeh/wrk -t4 -c200 -d30m --timeout 2s http://cafe.example.com/coffee ``` -1. Within the Azure portal, open your resource-group, click on the NGINX for Azure resource (`nginx4a`). + The above command would run for 30 minutes and send request to `http://cafe.example.com/coffee` using 4 threads and 200 connections. -1. From the left pane, click on `NGINX configuration` under `Settings` and then open the `cafe.example.com.conf` file under `/etc/nginx/conf.d` directory. This would open the config file in the editor. +1. Within Azure portal, open your NGINX for Azure resource (nginx4a). From the left pane click on `Monitoring > Metrics`. This should open a new Chart pane. -1. Copy the content of `lab6/cafe.example.com.conf` file and replace the existing `cafe.example.com.conf` content with it. +  -1. Click on `Submit` to push the config changes to the NGINX for Azure resource. +1. For the first chart, within **Metric Namespace** drop-down, select `nginx requests and response statistics`. For the **metrics** drop-down, select `plus.http.request.count`. For the **Aggregation** drop-down, select `Avg`. -### Test and validate TLS traffic components and settings + Click on the **Apply Splitting** button. Within the **Values** drop-down, select `server_zone`. From top right change the **Time range** to `Last 30 minutes`. This should generate a chart similar to below screenshot. -1. Make sure you have mapped your NGINX for Azure resource public IP to `cafe.example.com` hostname within your host file. If not present then please do insert it as you would require the mapping for testing. +  - ```bash - cat /etc/hosts | grep cafe.example.com - ``` +1. You will now save this chart in a new custom dashboard. Within the chart pane, click on `Save to dashboard > Pin to dashboard`. -1. Using your terminal, try to run the below curl command + Within the `Pin to dashboard` pane, select the `Create new` tab to create your new custom dashboard. Provide a name for your custom dashboard. Once done click on `Create and pin` button to finish dashboard creation. - ```bash - curl https://cafe.example.com - ``` +  - ```bash - ##Sample Output## - curl: (60) SSL certificate problem: unable to get local issuer certificate - More details here: https://curl.se/docs/sslcerts.html +1. To view your newly created dashboard, within Azure portal, navigate to `Dashboard` resource. - curl failed to verify the legitimacy of the server and therefore could not establish a secure connection to it. To learn more about this situation and how to fix it, please visit the web page mentioned above. - ``` + By default, this should open the default `My Dashboard` private dashboard. From the top drop-down select your custom dashboard name (`Nginx4a Dashboard`in the screenshot). This should open your custom dashboard which includes the pinned server request chart. - As you can see, **curl reports an error** that the certificate is not legitimate (because it is self-signed) and refuses to complete the request. Adding the `-k` flag means `-insecure`, would tell curl to ignore this error. This flag is required for self-signed certificates. +  -1. Try again now with a `-k` flag added to curl +1. Now you will add some more charts to your newly created dashboard. Navigate back to NGINX for Azure resource (nginx4a) and from the left pane click on `Monitoring > Metrics`. - ```bash - curl -k https://cafe.example.com - ``` +1. Within the chart pane, click on **Metric Namespace** drop-down and select `nginx upstream statistics`. For the **metrics** drop-down, select `plus.http.upstream.peers.response.time`. For the **Aggregation** drop-down, select `Avg`. - << Copy sample output once cafe upstream has been added >> + Click on the **Add filter** button. Within the **Property** drop-down, select `upstream`. Leave the **Operator** to `=`. In **values** drop-down, select `aks1_ingress`, `aks2_ingress` and `cafe_nginx`. -1. Now try it with a browser, go to https://cafe.example.com. YIKES - what's this?? Most modern browsers will display an **Error or Security Warning**: + Click on the **Apply Splitting** button. Within the **Values** drop-down, select `upstream`. From top right change the **Time range** to `Last 30 minutes`. This should generate a chart similar to below screenshot. -  +  -1. You can use browser's built-in certificate viewer to look at the details of the TLS certificate that was sent from NGINX to your browser. In address bar, click on the `Not Secure` icon, then click on `Certificate is not valid`. This will display the certificate. You can verify looking at the `Comman Name` field that this is the same certificate that you provided to NGINX for Azure resource. +1. You will now pin this chart to your custom dashboard. Within the chart pane, click on `Save to dashboard > Pin to dashboard`. -  + Within the `Pin to dashboard` pane, by default the `Existing` tab should be open with your recent created dashboard selected. Click on `Pin` button to pin the chart to your dashboard. -1. Within the browser, close the Certificate Viewer, click on the `Advanced` button, and then click on `Proceed to cafe.example.com (unsafe)` link, to bypass the warning and continue. - > CAUTION: Ignoring Browser Warnings is **Dangerous**, only Ignore these warnings if you are 100% sure it is safe to proceed!! +  -1. After you safely Proceed, you should see the cafe.example.com output as below +1. Navigate back to `Dashboard` resource within Azure portal and select your dashboard. You will notice that the chart that you recently pinned shows up in your dashboard. - << Add output screenshot once cafe upstream has been added >> +  -
+ You can also edit your dashboard by clicking on the pencil icon to reposition/resize charts as per your taste. -**This completes Lab6.** +1. Please look into the [References](#references) section to check the metric catalog and explore various other metrics available with NGINX for Azure. Feel free to play around and pin multiple metrics to your dashboard.
+**This completes Lab6.** + ## References: - [NGINX As A Service for Azure](https://docs.nginx.com/nginxaas/azure/) -- [NGINX As A Service SSL/TLS Docs](https://docs.nginx.com/nginxaas/azure/getting-started/ssl-tls-certificates/) -- [NGINX Directives Index](https://nginx.org/en/docs/dirindex.html) +- [Kusto Query Language](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/tutorials/learn-common-operators) + +- [NGINX Metrics catalog](https://docs.nginx.com/nginxaas/azure/monitoring/metrics-catalog/) + - [NGINX - Join Community Slack](https://community.nginx.org/joinslack)
@@ -216,4 +240,4 @@ Now that you have a self signed TLS certificate for testing, you will configure ------------- -Navigate to ([Lab7](../lab7/readme.md) | [LabX](../labX/readme.md)) +Navigate to ([Lab7](../lab7/readme.md) | [LabGuide](../readme.md)) diff --git a/labs/lab7/cafe.example.com.conf b/labs/lab7/cafe.example.com.conf new file mode 100644 index 0000000..e471ad2 --- /dev/null +++ b/labs/lab7/cafe.example.com.conf @@ -0,0 +1,37 @@ +# Nginx 4 Azure - Cafe Nginx HTTPS +# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 +# +server { + + listen 443 ssl; # Listening on port 443 with "ssl" parameter for terminating TLS on all IP addresses on this machine + + server_name cafe.example.com; # Set hostname to match in request + status_zone cafe.example.com; # Metrics zone name + + ssl_certificate /etc/nginx/cert/n4a-cert.cert; + ssl_certificate_key /etc/nginx/cert/n4a-cert.key; + + access_log /var/log/nginx/cafe.example.com.log main; + error_log /var/log/nginx/cafe.example.com_error.log info; + + location / { + # + # return 200 "You have reached cafe.example.com, location /\n"; + + proxy_pass http://cafe_nginx; # Proxy AND load balance to a list of servers + add_header X-Proxy-Pass cafe_nginx; # Custom Header + + # proxy_pass http://windowsvm; # Proxy AND load balance to a list of servers + # add_header X-Proxy-Pass windowsvm; # Custom Header + + # proxy_pass http://aks1_ingress; # Proxy AND load balance to AKS1 Nginx Ingress + # add_header X-Proxy-Pass aks1_ingress; # Custom Header + + # proxy_pass http://aks2_ingress; # Proxy AND load balance to AKS2 Nginx Ingress + # add_header X-Proxy-Pass aks1_ingress; # Custom Header + + # proxy_pass http://$upstream; # Use Split Clients config + # add_header X-Proxy-Pass $upstream; # Custom Header + + } +} \ No newline at end of file diff --git a/labs/lab7/media/docker-icon.png b/labs/lab7/media/docker-icon.png new file mode 100644 index 0000000..02ee3f1 Binary files /dev/null and b/labs/lab7/media/docker-icon.png differ diff --git a/labs/lab7/media/lab7_add_certificate1.png b/labs/lab7/media/lab7_add_certificate1.png new file mode 100644 index 0000000..c09d55f Binary files /dev/null and b/labs/lab7/media/lab7_add_certificate1.png differ diff --git a/labs/lab7/media/lab7_add_certificate2.png b/labs/lab7/media/lab7_add_certificate2.png new file mode 100644 index 0000000..415ea9c Binary files /dev/null and b/labs/lab7/media/lab7_add_certificate2.png differ diff --git a/labs/lab7/media/lab7_add_certificate_save.png b/labs/lab7/media/lab7_add_certificate_save.png new file mode 100644 index 0000000..ce9c4e2 Binary files /dev/null and b/labs/lab7/media/lab7_add_certificate_save.png differ diff --git a/labs/lab7/media/lab7_add_certificate_success.png b/labs/lab7/media/lab7_add_certificate_success.png new file mode 100644 index 0000000..969584b Binary files /dev/null and b/labs/lab7/media/lab7_add_certificate_success.png differ diff --git a/labs/lab6/media/browser_cert_details.png b/labs/lab7/media/lab7_browser_cert_details.png similarity index 100% rename from labs/lab6/media/browser_cert_details.png rename to labs/lab7/media/lab7_browser_cert_details.png diff --git a/labs/lab6/media/browser_cert_invalid.png b/labs/lab7/media/lab7_browser_cert_invalid.png similarity index 100% rename from labs/lab6/media/browser_cert_invalid.png rename to labs/lab7/media/lab7_browser_cert_invalid.png diff --git a/labs/lab7/media/lab7_browser_success.png b/labs/lab7/media/lab7_browser_success.png new file mode 100644 index 0000000..fe45efc Binary files /dev/null and b/labs/lab7/media/lab7_browser_success.png differ diff --git a/labs/lab7/media/lab7_certificate_issuance.png b/labs/lab7/media/lab7_certificate_issuance.png new file mode 100644 index 0000000..ca02368 Binary files /dev/null and b/labs/lab7/media/lab7_certificate_issuance.png differ diff --git a/labs/lab7/media/lab7_keyvault_screen.png b/labs/lab7/media/lab7_keyvault_screen.png new file mode 100644 index 0000000..bdb271a Binary files /dev/null and b/labs/lab7/media/lab7_keyvault_screen.png differ diff --git a/labs/lab6/media/n4a_cert_screen.png b/labs/lab7/media/lab7_n4a_cert_screen.png similarity index 100% rename from labs/lab6/media/n4a_cert_screen.png rename to labs/lab7/media/lab7_n4a_cert_screen.png diff --git a/labs/lab7/media/nginx-azure-icon.png b/labs/lab7/media/nginx-azure-icon.png new file mode 100644 index 0000000..70ab132 Binary files /dev/null and b/labs/lab7/media/nginx-azure-icon.png differ diff --git a/labs/lab7/readme.md b/labs/lab7/readme.md index 9cac8be..f5e2c4a 100644 --- a/labs/lab7/readme.md +++ b/labs/lab7/readme.md @@ -1,10 +1,8 @@ -# Azure Montoring / Logging Analytics +# Azure Key Vault / TLS Essentials ## Introduction -In this lab, you will build ( x,y,x ). - -< Lab specific Images here, in the /media sub-folder > +In this lab, you will create a new key-vault resource that would be storing self-signed certificates. You will then configure Nginx for Azure to listen for https traffic and then terminate TLS before proxying and load balancing back to the backend system. NGINX aaS | Docker :-------------------------:|:-------------------------: @@ -14,38 +12,290 @@ NGINX aaS | Docker By the end of the lab you will be able to: -- Introduction to `xx` -- Build an `yyy` Nginx configuration -- Test access to your lab enviroment with Curl and Chrome -- Investigate `zzz` - +- Build your own Azure Key Vault resource. +- Create your self-signed TLS certificate. +- Configure NGINX for Azure to listen and terminate TLS traffic +- Test and validate TLS traffic components and settings ## Pre-Requisites -- You must have `aaaa` installed and running -- You must have `bbbbb` installed +- You must have your Nginx for Azure resource up and running +- You must have `Owner` role on the resource group that includes NGINX for Azure resource +- You must also have backend system resources up and running. - See `Lab0` for instructions on setting up your system for this Workshop -- Familiarity with basic Linux commands and commandline tools -- Familiarity with basic Docker concepts and commands -- Familiarity with basic HTTP protocol -
+### Create Azure Key Vault resource + +1. Create an Azure key vault within the same resource group which holds your NGINX for azure resource. + + ```bash + ## Set environment variable + export MY_RESOURCEGROUP=s.dutta-workshop + export MY_INITIALS=sdutta + export MY_KEYVAULT=n4a-keyvault-$MY_INITIALS + ``` + + Once the environment variables are all set, run below command to create the key vault resource + + ```bash + az keyvault create \ + --resource-group $MY_RESOURCEGROUP \ + --name $MY_KEYVAULT \ + --enable-rbac-authorization false + ``` + + ```bash + ##Sample Output## + { + "id": "/subscriptions/
@@ -56,12 +306,9 @@ By the end of the lab you will be able to: ## References: - [NGINX As A Service for Azure](https://docs.nginx.com/nginxaas/azure/) -- [NGINX Plus Product Page](https://docs.nginx.com/nginx/) -- [NGINX Ingress Controller](https://docs.nginx.com//nginx-ingress-controller/) -- [NGINX on Docker](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-docker/) + +- [NGINX As A Service SSL/TLS Docs](https://docs.nginx.com/nginxaas/azure/getting-started/ssl-tls-certificates/) - [NGINX Directives Index](https://nginx.org/en/docs/dirindex.html) -- [NGINX Variables Index](https://nginx.org/en/docs/varindex.html) -- [NGINX Technical Specs](https://docs.nginx.com/nginx/technical-specs/) - [NGINX - Join Community Slack](https://community.nginx.org/joinslack)
@@ -74,4 +321,4 @@ By the end of the lab you will be able to: ------------- -Navigate to ([Lab8](../lab8/readme.md) | [LabX](../labX/readme.md)) +Navigate to ([Lab8](../lab8/readme.md) | [LabGuide](../readme.md)) diff --git a/labs/lab6/self-certificate-policy.json b/labs/lab7/self-certificate-policy.json similarity index 100% rename from labs/lab6/self-certificate-policy.json rename to labs/lab7/self-certificate-policy.json diff --git a/labs/lab8/MyGarage-Home.png b/labs/lab8/MyGarage-Home.png new file mode 100644 index 0000000..2635caa Binary files /dev/null and b/labs/lab8/MyGarage-Home.png differ diff --git a/labs/lab8/MyGarage-PhotoGallery.png b/labs/lab8/MyGarage-PhotoGallery.png new file mode 100644 index 0000000..53e777a Binary files /dev/null and b/labs/lab8/MyGarage-PhotoGallery.png differ diff --git a/labs/lab8/MyGarage-SeedData.png b/labs/lab8/MyGarage-SeedData.png new file mode 100644 index 0000000..1ef1f47 Binary files /dev/null and b/labs/lab8/MyGarage-SeedData.png differ diff --git a/labs/lab8/MyGarage-Vehicles.png b/labs/lab8/MyGarage-Vehicles.png new file mode 100644 index 0000000..2dcddc3 Binary files /dev/null and b/labs/lab8/MyGarage-Vehicles.png differ diff --git a/labs/lab8/my-garage-deployment.yaml b/labs/lab8/my-garage-deployment.yaml new file mode 100644 index 0000000..6ae67a7 --- /dev/null +++ b/labs/lab8/my-garage-deployment.yaml @@ -0,0 +1,69 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: my-garage-deployment +spec: + replicas: 1 + selector: + matchLabels: + app: my-garage + template: + metadata: + labels: + app: my-garage + spec: + containers: + - name: my-garage + image: ghcr.io/ciroque/my-garage:18fb641c + ports: + - containerPort: 80 + env: + - name: ASPNETCORE_ENVIRONMENT + value: "Production" + +--- +apiVersion: v1 +kind: Service +metadata: + name: my-garage-svc +spec: + type: ClusterIP + clusterIP: None + ports: + - name: http + protocol: TCP + port: 80 + targetPort: 80 + selector: + app: my-garage + +--- +apiVersion: k8s.nginx.org/v1 +kind: VirtualServer +metadata: + name: my-garage-vs +spec: + host: my-garage.example.com + upstreams: + - name: my-garage + service: my-garage-svc + port: 80 + healthCheck: + enable: true + path: / + interval: 10s + jitter: 3s + fails: 3 + passes: 2 + connect-timeout: 30s + read-timeout: 20s + routes: + - path: / + action: + pass: my-garage + - path: /garage-lab + action: + return: + code: 200 + type: text/html + body: "Welcome to Nginx4Azure Garage Lab !!" \ No newline at end of file diff --git a/labs/lab8/readme.md b/labs/lab8/readme.md index 43cec3f..f2d26b2 100644 --- a/labs/lab8/readme.md +++ b/labs/lab8/readme.md @@ -1,51 +1,423 @@ -# NGINX Garage or Azure Petshop +# NGINX Garage (UNDER CONSTRUCTION) + ## Introduction -In this lab, you will build ( x,y,x ). -< Lab specific Images here, in the /media sub-folder > +In this lab, you will install the My Garage application, configure it for external access, learn to scale the web service, and set up caching for the image gallery (optional). + +The My Garage application is a modern web application built using Microsoft .Net technologies. It is comprised of a frontend application and supporting web service backend. The front-end is a Single Page Application (SPA) that uses [Blazor WebAssembly](https://dotnet.microsoft.com/en-us/apps/aspnet/web-apps/blazor) to render the UI in the browser. The back-end is a RESTful API built using [ASP.Net Core MVC](https://learn.microsoft.com/en-us/aspnet/core/mvc/overview?view=aspnetcore-8.0). + +|  |  | +|------|------| +|  |  | + -NGINX aaS | Docker -:-------------------------:|:-------------------------: - | - ## Learning Objectives By the end of the lab you will be able to: -- Introduction to `xx` -- Build an `yyy` Nginx configuration -- Test access to your lab enviroment with Curl and Chrome -- Investigate `zzz` - +- Create all the resources necessary to deploy the My Garage application +- Ensure the My Garage application is accessible from the internet +- Monitor traffic to the My Garage application using the NGINX Dashboard ## Pre-Requisites -- You must have `aaaa` installed and running -- You must have `bbbbb` installed -- See `Lab0` for instructions on setting up your system for this Workshop +You need to have followed the labs up to this point. Specifically, Lab 0 and Lab 4 are required to have been completed. + +- You must have the Azure CLI installed and configured to manage Azure Resources - Familiarity with basic Linux commands and commandline tools - Familiarity with basic Docker concepts and commands -- Familiarity with basic HTTP protocol +- You must have created a Resource Group
+Set some environment variables to be used when creating the Azure Resources: + +```shell +# Modify these as desired +export MY_RESOURCEGROUP="${MY_RESOURCEGROUP:-$(whoami)-n4a-workshop}" +export SAS_EXPIRY=2024-12-31Z23:59:59 +export REDIS_CONNECTION_STRING=redis-leader +export LOCATION=westus2 + +# Using the ticks value ensures the naming requirements for Azure Resources are met +timestamp=$(date +%s) +milliseconds=$(date +%N | awk '{print $1 / 1000}') +export TICKS="$timestamp$milliseconds" +export STORAGE_ACCOUNT_NAME="mygsa$TICKS" +export STORAGE_CONTAINER_NAME="mygsa$TICKS" +export SAS_TOKEN_NAME="mygsas$TICKS" +export APP_CONFIG_NAME="mygac$TICKS" +export OWNER=$(whoami) + +# These correspond to the keys in the AppConfig that the My Garage application will use to access the resources, do not modify +export SAS_TOKEN_APP_CONFIG_KEY="AzureStorageSasToken" +export STORAGE_CONNECTION_STRING_CONFIG_KEY="AzureStorageConnectionString" +export STORAGE_CONTAINER_NAME_CONFIG_KEY="AzureStorageContainerName" +export REDIS_CONNECTION_STRING_CONFIG_KEY="RedisConnectionString" +``` + ### Lab exercise 1 -
@@ -71,7 +443,8 @@ By the end of the lab you will be able to: - Chris Akker - Solutions Architect - Community and Alliances @ F5, Inc. - Shouvik Dutta - Solutions Architect - Community and Alliances @ F5, Inc. - Adam Currier - Solutions Architect - Community and Alliances @ F5, Inc. +- Steve Wagner - Solutions Architect - Community and Alliances @ F5, Inc. ------------- -Navigate to ([Lab9](../lab9/readme.md) | [LabX](../labX/readme.md)) +Navigate to ([Lab9](../lab9/readme.md) | [LabGuide](../readme.md)) diff --git a/labs/lab8/the-garage-deployment.yaml b/labs/lab8/the-garage-deployment.yaml new file mode 100644 index 0000000..ad95d2c --- /dev/null +++ b/labs/lab8/the-garage-deployment.yaml @@ -0,0 +1,83 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: the-garage-deployment +spec: + replicas: 1 + selector: + matchLabels: + app: the-garage + template: + metadata: + labels: + app: the-garage + spec: + containers: + - name: the-garage + image: ghcr.io/ciroque/the-garage:18fb641c + ports: + - containerPort: 8080 + env: + - name: ASPNETCORE_ENVIRONMENT + value: "Production" + - name: ConnectionStrings__AppConfig + value: "Endpoint=https://mygac1719520913635582.azconfig.io;Id=wOL3;Secret=Bnt3dVF8vU4ZqnmPyl2i6Zroi7z5GUOCu6Zjr6rViB0FUCTkwyFDJQQJ99AFAC8vTIns17ujAAABAZACiTCv" + livenessProbe: + httpGet: + path: "/vehicles" + port: 8080 + initialDelaySeconds: 30 + periodSeconds: 10 + readinessProbe: + httpGet: + path: "/vehicles" + port: 8080 + initialDelaySeconds: 30 + periodSeconds: 10 + +--- +apiVersion: v1 +kind: Service +metadata: + name: the-garage-svc +spec: + type: ClusterIP + clusterIP: None + ports: + - name: http-alt + protocol: TCP + port: 8080 + targetPort: 8080 + selector: + app: the-garage + +--- +apiVersion: k8s.nginx.org/v1 +kind: VirtualServer +metadata: + name: the-garage-vs +spec: + host: the-garage.example.com + upstreams: + - name: the-garage + service: the-garage-svc + port: 8080 + healthCheck: + enable: true + path: /vehicles + interval: 10s + jitter: 3s + fails: 3 + passes: 2 + connect-timeout: 30s + read-timeout: 20s + routes: + - path: / + action: + pass: the-garage + - path: /garage-lab + action: + return: + code: 200 + type: text/html + body: "You have found The Garage" \ No newline at end of file diff --git a/ca-notes/aks/juiceshop/juiceshop-vs.yaml b/labs/lab9/juiceshop-vs.yaml similarity index 70% rename from ca-notes/aks/juiceshop/juiceshop-vs.yaml rename to labs/lab9/juiceshop-vs.yaml index 09a5d36..346c8df 100644 --- a/ca-notes/aks/juiceshop/juiceshop-vs.yaml +++ b/labs/lab9/juiceshop-vs.yaml @@ -6,14 +6,20 @@ metadata: name: juiceshop-vs namespace: juice spec: - host: juiceshop.nginxazure.build - tls: - secret: juice-secret + host: juiceshop.example.com + #tls: + #secret: juice-secret upstreams: - name: juiceshop service: juiceshop-svc port: 80 - slow-start: 5s + #slow-start: 5s + sessionCookie: + enable: true + name: srv_id + path: / + expires: 1h + domain: .example.com healthCheck: enable: true port: 3000 diff --git a/ca-notes/n4a-configs/juiceshop.nginxazure.build.conf b/labs/lab9/juiceshop.example.com.conf similarity index 72% rename from ca-notes/n4a-configs/juiceshop.nginxazure.build.conf rename to labs/lab9/juiceshop.example.com.conf index 5ef1197..d483206 100644 --- a/ca-notes/n4a-configs/juiceshop.nginxazure.build.conf +++ b/labs/lab9/juiceshop.example.com.conf @@ -10,12 +10,12 @@ server { listen 80; # Listening on port 80 on all IP addresses on this machine - server_name juiceshop.nginxazure.build; # Set hostname to match in request + server_name juiceshop.example.com; # Set hostname to match in request status_zone juiceshop; # access_log /var/log/nginx/juiceshop.log main; - access_log /var/log/nginx/juiceshop.nginxazure.build.log main_ext; # Extended Logging - error_log /var/log/nginx/juiceshop.nginxazure.build_error.log info; + access_log /var/log/nginx/juiceshop.example.com.log main_ext; # Extended Logging + error_log /var/log/nginx/juiceshop.example.com_error.log info; location / { @@ -23,17 +23,17 @@ server { # Set Rate Limit, uncomment below # limit_req zone=limit100; #burst=110; # Set Limit and burst here - # limit_req_status 429; # Set HTTP Return Code, better than 503s + # limit_req_status 429; # Set HTTP Status Code, better than 503s # limit_req_dry_run on; # Test the Rate limit, logged, but not enforced # add_header X-Ratelimit-Status $limit_req_status; # Add a custom status header proxy_pass http://aks1_ingress; # Proxy to AKS1 Nginx Ingress Controllers - # proxy_pass http://aks1_juice_direct; # Proxy directly to Juiceshop Headless Service + add_header X-Proxy-Pass aks1_ingress_juiceshop; # Custom Header } # Cache Proxy example for static images / page components - # Match common files + # Match common files with Regex location ~* \.(?:ico|jpg|png)$ { ### Uncomment for new status_zone in dashboard @@ -52,8 +52,8 @@ server { add_header X-Cache-Status $upstream_cache_status; - #proxy_pass http://aks1_ingress; # Proxy AND load balance to AKS1 NIC - proxy_pass http://aks1_juice_direct; # Proxy directly to Juiceshop Headless Service + proxy_pass http://aks1_ingress; # Proxy AND load balance to AKS1 NIC + add_header X-Proxy-Pass nginxazure_imagecache; # Custom Header } diff --git a/ca-notes/aks/juiceshop/juiceshop.yaml b/labs/lab9/juiceshop.yaml similarity index 100% rename from ca-notes/aks/juiceshop/juiceshop.yaml rename to labs/lab9/juiceshop.yaml diff --git a/labs/lab9/media/juiceshop-icon.png b/labs/lab9/media/juiceshop-icon.png new file mode 100644 index 0000000..afb8ed1 Binary files /dev/null and b/labs/lab9/media/juiceshop-icon.png differ diff --git a/labs/lab9/media/lab9_chrome-add-headers.png b/labs/lab9/media/lab9_chrome-add-headers.png new file mode 100644 index 0000000..8065937 Binary files /dev/null and b/labs/lab9/media/lab9_chrome-add-headers.png differ diff --git a/labs/lab9/media/lab9_chrome-hit-miss-expired.png b/labs/lab9/media/lab9_chrome-hit-miss-expired.png new file mode 100644 index 0000000..e16e626 Binary files /dev/null and b/labs/lab9/media/lab9_chrome-hit-miss-expired.png differ diff --git a/labs/lab9/media/lab9_chrome-manage-headers.png b/labs/lab9/media/lab9_chrome-manage-headers.png new file mode 100644 index 0000000..48ca807 Binary files /dev/null and b/labs/lab9/media/lab9_chrome-manage-headers.png differ diff --git a/labs/lab9/media/lab9_chrome-new-columns.png b/labs/lab9/media/lab9_chrome-new-columns.png new file mode 100644 index 0000000..6dc1725 Binary files /dev/null and b/labs/lab9/media/lab9_chrome-new-columns.png differ diff --git a/labs/lab9/media/lab9_diagram.png b/labs/lab9/media/lab9_diagram.png new file mode 100644 index 0000000..df8573e Binary files /dev/null and b/labs/lab9/media/lab9_diagram.png differ diff --git a/labs/lab9/media/lab9_juiceshop-upstreams.png b/labs/lab9/media/lab9_juiceshop-upstreams.png new file mode 100644 index 0000000..f7edc70 Binary files /dev/null and b/labs/lab9/media/lab9_juiceshop-upstreams.png differ diff --git a/labs/lab9/media/lab9_rate-100.png b/labs/lab9/media/lab9_rate-100.png new file mode 100644 index 0000000..1a758bd Binary files /dev/null and b/labs/lab9/media/lab9_rate-100.png differ diff --git a/labs/lab9/media/lab9_rate-1000.png b/labs/lab9/media/lab9_rate-1000.png new file mode 100644 index 0000000..f56c5ae Binary files /dev/null and b/labs/lab9/media/lab9_rate-1000.png differ diff --git a/labs/lab9/media/lab9_ratelimit-429.png b/labs/lab9/media/lab9_ratelimit-429.png new file mode 100644 index 0000000..fa2f483 Binary files /dev/null and b/labs/lab9/media/lab9_ratelimit-429.png differ diff --git a/labs/lab9/media/lab9_ratelimit-503.png b/labs/lab9/media/lab9_ratelimit-503.png new file mode 100644 index 0000000..bbcb1da Binary files /dev/null and b/labs/lab9/media/lab9_ratelimit-503.png differ diff --git a/labs/lab9/media/lab9_ratelimit-dry-run.png b/labs/lab9/media/lab9_ratelimit-dry-run.png new file mode 100644 index 0000000..689a952 Binary files /dev/null and b/labs/lab9/media/lab9_ratelimit-dry-run.png differ diff --git a/labs/lab9/media/mygarage-icon.png b/labs/lab9/media/mygarage-icon.png new file mode 100644 index 0000000..4e52540 Binary files /dev/null and b/labs/lab9/media/mygarage-icon.png differ diff --git a/labs/lab9/media/nginx-azure-icon.png b/labs/lab9/media/nginx-azure-icon.png new file mode 100644 index 0000000..70ab132 Binary files /dev/null and b/labs/lab9/media/nginx-azure-icon.png differ diff --git a/labs/lab9/media/nginx-cache-icon.png b/labs/lab9/media/nginx-cache-icon.png new file mode 100644 index 0000000..462c184 Binary files /dev/null and b/labs/lab9/media/nginx-cache-icon.png differ diff --git a/labs/lab9/media/speedometer-icon.jpeg b/labs/lab9/media/speedometer-icon.jpeg new file mode 100644 index 0000000..0a50e0f Binary files /dev/null and b/labs/lab9/media/speedometer-icon.jpeg differ diff --git a/ca-notes/n4a-configs/nginx.conf b/labs/lab9/nginx.conf similarity index 88% rename from ca-notes/n4a-configs/nginx.conf rename to labs/lab9/nginx.conf index 119ba14..c0dbadf 100644 --- a/ca-notes/n4a-configs/nginx.conf +++ b/labs/lab9/nginx.conf @@ -1,3 +1,6 @@ +# Nginx 4 Azure - Default - Updated Nginx.conf +# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 +# user nginx; worker_processes auto; worker_rlimit_nofile 8192; @@ -34,17 +37,13 @@ http { 'upstream_response_length="$upstream_response_length", ' 'cachestatus=“$upstream_cache_status“, ' 'limitstatus=“$limit_req_status“ '; - - # access_log off; - server_tokens N4A-$nginx_version; + + access_log off; + server_tokens ""; server { - # listen 80 default_server; - listen 80; - server_name www.nginxazure.build; # nginxazure.com; - status_zone www.nginxazure.build; - add_header X-Host-Header $host; + listen 80 default_server; + server_name localhost; location / { - status_zone /; # Points to a directory with a basic html index file with # a "Welcome to NGINX as a Service for Azure!" page root /var/www; diff --git a/ca-notes/n4a-configs/includes/rate_limits.conf b/labs/lab9/rate_limits.conf similarity index 68% rename from ca-notes/n4a-configs/includes/rate_limits.conf rename to labs/lab9/rate_limits.conf index 2b5c03f..9011890 100644 --- a/ca-notes/n4a-configs/includes/rate_limits.conf +++ b/labs/lab9/rate_limits.conf @@ -1,3 +1,8 @@ +# Nginx 4 Azure - Mar 2024 +# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 +# +# Define HTTP Request Limit Zones +# limit_req_zone $binary_remote_addr zone=limitone:10m rate=1r/s; limit_req_zone $binary_remote_addr zone=limit10:10m rate=10r/s; limit_req_zone $binary_remote_addr zone=limit100:10m rate=100r/s; diff --git a/labs/lab9/readme.md b/labs/lab9/readme.md index 892935b..753b6e0 100644 --- a/labs/lab9/readme.md +++ b/labs/lab9/readme.md @@ -1,51 +1,503 @@ -# NGINX and AzureAD / Entra Integration +# Nginx Caching / Rate Limits / Juiceshop ## Introduction -In this lab, you will build ( x,y,x ). +In this lab, you will deploy an image rich application, and use Nginx Caching to cache images to improve performance and provide a better user experience. This will offload the image delivery workload from your applications, saving resources. You will also explore, configure, and test Rate Limits with Nginx for Azure, allowing you to control incoming request levels for different applications. -< Lab specific Images here, in the /media sub-folder > +
-NGINX aaS | Docker -:-------------------------:|:-------------------------: - | +NGINXaaS Azure | Cache | Juiceshop | My Garage +:-----------------:|:-----------------:|:-----------------:|:-----------------: + | | | ## Learning Objectives -By the end of the lab you will be able to: - -- Introduction to `xx` -- Build an `yyy` Nginx configuration -- Test access to your lab enviroment with Curl and Chrome -- Investigate `zzz` - +- Deploy JuiceShop in AKS cluster. +- Expose JuiceShop with Nginx Ingress Controller. +- Configure Nginx for Azure for load balancing JuiceShop. +- Configure Nginx for Azure for load balancing Mygarage. +- Add Nginx Caching to improve delivery of images. +- Explore, configure, and test HTTP Request Limits ## Pre-Requisites -- You must have `aaaa` installed and running -- You must have `bbbbb` installed +- You must have your Nginx for Azure instance running +- You must have your AKS1 Cluster running - See `Lab0` for instructions on setting up your system for this Workshop - Familiarity with basic Linux commands and commandline tools -- Familiarity with basic Docker concepts and commands - Familiarity with basic HTTP protocol +- Familiarity with HTTP Caching parameters, directives, headers + +
+ + + +Lab9 Diagram + +
+ +## Deploy Juiceshop to AKS Cluster #1 + + + +
+ +In this exercise, you will deploy the demo Juiceshop application to AKS1. Juiceshop is a demo application example of a Retail store, selling different juices, smoothies, and snacks. The images used on the various web pages make ideal candidates for image caching. You will also configure Nginx Ingress Controller for this application. + +1. Inspect the `lab9/juiceshop.yaml` and then `lab9/juiceshop-vs.yaml` manifests. You will see definitions for three Juiceshop application pods being deployed, and a new VirtualServer being added to Nginx Ingress to expose the app outside the Cluster. + +1. Using your Terminal, create a new namespace `juice` and deploy the Juiceshop application to AKS1. Also deploy the Nginx Ingress VirtualServer, to create the Service and VirtualServer for `juiceshop.example.com`. Use the Manifests provided in the `lab9` folder: + + ```bash + kubectl config use-context n4a-aks1 + kubectl create namespace juice + kubectl apply -f lab9/juiceshop.yaml + kubectl apply -f lab9/juiceshop-vs.yaml + + ``` + + ```bash + #Sample output + namespace/juice created + deployment.apps/juiceshop created + service/juiceshop-svc created + secret/juice-secret created + virtualserver.k8s.nginx.org "juiceshop-vs" deleted + + ``` + +1. Check your Nginx Ingress Dashboard for AKS1, you should now find `juiceshop.example.com` in the HTTP Zones, and a new `vs_juice_juiceshop-vs_juiceshop` Upstream block in the HTTP Upstreams tab, with 3 Pods running on port 3000. + + + +## Update Nginx.conf for Extended Logging + +During the next exercises, an updated `main_ext` logging format will be used, to capture Nginx Cache and Rate Limit logging variables. + +1. Inspect, then modify your `/etc/nginx/nginx.conf` file to match the `main_ext logging format` example provided. You will noticed two new Nginx Logging variables have been added to this log format, `$upstream_cache_status` and `$limit_req_status.` These will be used in the next few exercises. + +```nginx +# Nginx 4 Azure - Default - Updated Nginx.conf +# Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 +# +user nginx; +worker_processes auto; +worker_rlimit_nofile 8192; +pid /run/nginx/nginx.pid; + +events { + worker_connections 4000; +} + +error_log /var/log/nginx/error.log error; + +http { + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + log_format main_ext 'remote_addr="$remote_addr", ' + '[time_local=$time_local], ' + 'request="$request", ' + 'status="$status", ' + 'http_referer="$http_referer", ' + 'body_bytes_sent="$body_bytes_sent", ' + 'Host="$host", ' + 'sn="$server_name", ' + 'request_time=$request_time, ' + 'http_user_agent="$http_user_agent", ' + 'http_x_forwarded_for="$http_x_forwarded_for", ' + 'request_length="$request_length", ' + 'upstream_address="$upstream_addr", ' + 'upstream_status="$upstream_status", ' + 'upstream_connect_time="$upstream_connect_time", ' + 'upstream_header_time="$upstream_header_time", ' + 'upstream_response_time="$upstream_response_time", ' + 'upstream_response_length="$upstream_response_length", ' + 'cachestatus=“$upstream_cache_status“, ' + 'limitstatus=“$limit_req_status“ '; + + access_log off; + server_tokens ""; + server { + listen 80 default_server; + server_name localhost; + location / { + # Points to a directory with a basic html index file with + # a "Welcome to NGINX as a Service for Azure!" page + root /var/www; + index index.html; + } + } + + include /etc/nginx/conf.d/*.conf; + include /etc/nginx/includes/*.conf; # shared files + +} + +stream { + + include /etc/nginx/stream/*.conf; # Stream TCP nginx files + +} + +``` + +Submit your Nginx Configuration. + +## Add Caching to Nginx for Azure + +
-### Lab exercise 1 +In this exercise, you will create an Nginx for Azure configuration, to add Caching for the images of the Juiceshop application. You will also configure Nginx for Azure to expose your Juiceshop application to the Internet. You will test it, and use various tools to verify that caching is working as expected. + +1. Inspect the `lab9/juiceshop.example.com.conf` configuration file. Make note of the following items, which enable `Nginx Proxy Caching` for images: + + - Line #7 - create the Cache - /path on disk, cache name=image_cache, :memory zone and size, max image size, disable temp files. + - Line #13 - set the hostname + - Line #14 - create a status zone for metrics + - Line #17,18 - set the logging filenames + - Line #30 - send requests to Nginx Ingress in AKS1 + - Line #31 - set the Header for tracking + - Lines #37-62 - A new `location block`, with the following parameters + - - Line #39 - Use a Regular Expression (regex) to identify image types. + - - Line #42 - new status zone for image metrics + - - - Lines #44-46 - use the `image_cache` created earlier on Line #7 + - - - cache 200 responses for 60 seconds + - - - use a cache key, made up of three Nginx request $variables + - - Lines #49-51 - Set and Control Caching Headers + - - Line #55 - Set a Custom Header for Cache Status = HIT, MISS, EXPIRED + - - Line #57 - Send requests to Nginx Ingress in AKS1 + - - Line #58 - Set another Custom Header for tracking + + As you can see, there are quite a few Caching directives and parameters that must be set properly. There are Advanced Nginx Caching classes available from Nginx University that cover architectures and many more details and use cases if you would like to learn more. You will also find quite a few blogs and a Webinar on Nginx Caching, it is a popular topic. See the References Section. + + But for this exercise, you will just enable it with the minimal configuration and test it out with Chrome. + +1. Create the Nginx for Azure configuration needed for `juiceshop.example.com.` + + Using the Nginx for Azure Console, create a new config file, `/etc/nginx/conf.d/juiceshop.example.com.conf`. You can use the example file provided, just Copy/Paste. + + ```nginx + # Nginx 4 Azure - Juiceshop Nginx HTTP + # Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 + # + # Image Caching for Juiceshop + # Rate Limits testing + # + proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=image_cache:10m max_size=100m use_temp_path=off; + # + server { + + listen 80; # Listening on port 80 on all IP addresses on this machine + + server_name juiceshop.example.com; # Set hostname to match in request + status_zone juiceshop; + + # access_log /var/log/nginx/juiceshop.log main; + access_log /var/log/nginx/juiceshop.example.com.log main_ext; # Extended Logging + error_log /var/log/nginx/juiceshop.example.com_error.log info; + + location / { + + # return 200 "You have reached juiceshop server block, location /\n"; + + # Set Rate Limit, uncomment below + # limit_req zone=limit100; #burst=110; # Set Limit and burst here + # limit_req_status 429; # Set HTTP Return Code, better than 503s + # limit_req_dry_run on; # Test the Rate limit, logged, but not enforced + # add_header X-Ratelimit-Status $limit_req_status; # Add a custom status header + + proxy_pass http://aks1_ingress; # Proxy to AKS1 Nginx Ingress Controllers + add_header X-Proxy-Pass aks1_ingress_juiceshop; # Custom Header + + } + + # Cache Proxy example for static images / page components + # Match common files with Regex + location ~* \.(?:ico|jpg|png)$ { + + ### Uncomment for new status_zone in dashboard + status_zone images; + + proxy_cache image_cache; + proxy_cache_valid 200 60s; + proxy_cache_key $scheme$proxy_host$request_uri; + + # Override cache control headers + proxy_ignore_headers X-Accel-Expires Expires Cache-Control Set-Cookie; + expires 365d; + add_header Cache-Control "public"; + + # Add a Cache status header - MISS, HIT, EXPIRED + + add_header X-Cache-Status $upstream_cache_status; + + proxy_pass http://aks1_ingress; # Proxy AND load balance to AKS1 NIC + add_header X-Proxy-Pass nginxazure_imagecache; # Custom Header + + } + + } + + ``` + + Submit your Nginx Configuration. + +1. Update your local DNS `/etc/hosts` file, and add `juiceshop.example.com` to your list of Hostnames for this Workshop, using the Public IP of your Nginx for Azure instance. This now makes it FOUR hostnames active on 1 IP Address. + + ```bash + cat /etc/hosts + # Nginx for Azure Workshop + 13.86.100.10 cafe.example.com dashboard.example.com redis.example.com juiceshop.example.com + + ``` + +### Test out Nginx for Azure Caching with Juiceshop + +1. Open Chrome and go to `http://juiceshop.example.com`. You should see the main Juiceshop page, explore around a bit if you like, find a great tasting smoothy. + +1. Right+Click, and choose `Inspect` on the Chrome menu to open Developer tools. On the top Nav bar, click the `Network Tab`, and make sure the `Disable cache` is checked, you don't want Chrome caching any images for this exercise. + +1. Click Refresh, and you will see a long list of items being sent from the application. + +1. In the Object Details Display Bar, where you see `Name Status Type Size, Time, etc`, Right+Click again, then `Response Headers`, then `Manage Header Columns`. + +  + + You will be adding your THREE custom Nginx headers to the display for easy viewing. Click on `Add custom header...` , input these names one at a time: + + - X-Cache-Status + - X-Proxy-Pass + - X-RateLimit-Status + + This added these Headers to the display, making it easy to see the Header Values. + +  + +  + + *Now your Object Details Display should have these three new Headers you can watch.* + +1. Click Refresh again, what do you see? `The X-Cache-Status` header will display `HIT, MISS, EXPIRED`, depending on how Nginx is caching, or not caching, each object. A MISS means the object was not in the cache at all, of course. Clear the Dev tool display, and Click Refresh a couple more times - see if you can find some HITS? If you wait more than 60 seconds, Refresh again, and these same objects will show EXPIRED. Click on one of the objects of interest, and check the Response Headers. + +  + + What does X-Proxy-Pass show? Does it show 2 different Values? + - one for `aks1_ingress_juiceshop` for your first `location / block` + - and `nginxazure_imagecache` for your `REGEX location block` for the image types? -
+ +## Nginx for Azure Caching Wrap Up + +Notice that is was pretty easy to define, and enable Nginx Caching for images and even other static page objects. Also notice that you set the Valid time = 60 seconds. This was intentional so you can see objects Expire quickly. However, in Production, you will coordinate with your app team to determine the proper Cache age timer for different object types. You can create multiple caches, with different names and Regex's, to have granular control over type, age time, size, etc. It's EASY with Nginx for Azure! + +
+ +## Explore, configure, and test HTTP Request Limits + + + +In this exercise, Nginx HTTP Request Rate Limiting will be explored. You will configure some Limits, apply them to the Juiceshop application, and see the results of introducing various Limits. Rate Limiting has many practical use cases - limiting attacks, limiting bots, protecting request load sensitive URLs/APIs, classes of service, and others. + +1. Inspect the `lab9/rate-limits.conf` file. You will see 4 different Rate Limits defined, using the `limit_req_zone` directive. This directive creates an Nginx memory zone where the limit Keys and counters are stored. When a request matches a Key, the counter is incremented. If no key exists, it is added to the zone and the counter is incremented, as you would expect. Keys are ephemeral, they are lost if you restart Nginx, but are preserved during an Nginx Reload. + + **Example: limit_req_zone $binary_remote_addr zone=limitone:10m rate=1r/s;** + + A. The first parameter, `$binary_remote_addr` is the Key used in the memory zone for tracking. In this example, the client's IP Address in binary format is used. Binary being shorter, using less memory, than a dot.ted.dec.imal IP Address string. You can use whatever Key $variable you like, as long as it is an Nginx $variable available when Nginx receives an HTTP request - like a cookie, URL argument, HTTP Header, TLS Serial Number, etc. There are literally hundreds of request $variables you could use, and you can combine multiple $variables together. + + B. The second parmater, `zone=limitXYZ:10m`, is the name of the zone, and the size is 10MB. You can define larger memory zones if needed, 10MB is a good starting point. Each zone must have a unique name, which matches the actual limit being defined in this example. The size needed depends on how many Keys are stored. + + - - limitone is the zone for 1 request/second + - - limit10 is the zone for 10 requests/second + - - limit100 is the zone for 100 requests/second + - - limit1000 is the zone for 1,000 requests/second + + C. The third parameter is the actual Rate Limit Value, expressed as `r/s` for `requests/second`. + + - You can define as many zones as you need, as long as you have enough memory for it. You can use a zone more than once in an Nginx configuration. You can see the number of requests that are being counted in each limit zone with Azure Monitoring. You can also use Nginx Logging $variables to track when Limits are being counted and used for the request. You will create an HTTP Header that will also show you the limit status of the request when Nginx sends back the response. *So you will have very good visibility into how/when the limits are being used.* + +1. Using the Nginx for Azure Console, create a new file called `/etc/nginx/includes/rate-limits.conf`. You can use the example file provided, just Copy/Paste. + + ```nginx + # Nginx 4 Azure - Mar 2024 + # Chris Akker, Shouvik Dutta, Adam Currier - Mar 2024 + # + # Define HTTP Request Limit Zones + # + limit_req_zone $binary_remote_addr zone=limitone:10m rate=1r/s; + limit_req_zone $binary_remote_addr zone=limit10:10m rate=10r/s; + limit_req_zone $binary_remote_addr zone=limit100:10m rate=100r/s; + limit_req_zone $binary_remote_addr zone=limit1000:10m rate=1000r/s; + + ``` + +1. Enable the Rate Limit section of the `/etc/nginx/conf.d/juiceshop.example.com.conf` file, by removing the comments - these have already been provided for this exercise. You will test them all, but one at a time, starting with `limit100`, 100 reqs/second: + + ```nginx + ... + + location / { + + # return 200 "You have reached juiceshop server block, location /\n"; + + # Set Rate Limit, uncomment below + limit_req zone=limit100; #burst=110; # Set Limit and burst here + # limit_req_status 429; # Set HTTP Return Code, better than 503s + # limit_req_dry_run on; # Test the Rate limit, logged, but not enforced + add_header X-Ratelimit-Status $limit_req_status; # Add a custom status header + + proxy_pass http://aks1_ingress; # Proxy to AKS1 Nginx Ingress Controllers + add_header X-Proxy-Pass aks1_ingress_juiceshop; # Custom Header + + } + + ``` + +Notice the 2 directives enabled: + +- `limit_req` sets the active zone being used, in this example, limit100, meaning 100 requests/second. `Burst` is optional, allowing you to define an overage, allowing for some elasticity in the limit enforcement. +- `add_header` creates a Custom Header, and adds the `limit_req_status $variable`, so you can see it with Chrome Dev Tools or curl. + +Submit your Nginx Configuration. + +### Test Rate Limit on Juiceshop application + +1. Using Chrome, navigate to `http://juiceshop.example.com`, and open the Chrome Dev Tools. You previously added the Nginx Custom Headers to the display, so you should already have a Header Column labeled `X-Ratelimit-Status`. Click Refresh Several times, what do you see? + +  + + You will see a partial Juiceshop webpage, as Nginx is only allowing your computer to send 100 req/s. You see the Header status set to PASSED for requests that were allowed. Other requests were stopped for `Exceeding the Rate Limit`. Check the HTTP Status Code on an item that failed, you will find the `503 Service Temporarily Unavailable`. Well, this is not actually the real situation, right? You have set a limit, not turned off the Service. So you will `change the HTTP Status code`, using the `limit_req_status` directive, which lets you set a custom HTTP Status code. The HTTP standard for "excessive requests" is normally `429.` So you will change it to that. + +  + +1. Using the Nginx for Azure Console, uncomment the `limit_req_status 429`, as shown. This will change the 503 Status Code to a more friendly HTTP Status Code of 429, which means `Too Many Requests`. This could be useful for clients that can use this 429 code to perform a back-off of the Requests, and try again after a time delay. (Most Browsers do not do this). + + ```nginx + ... + location / { + + # return 200 "You have reached juiceshop server block, location /\n"; + + # Set Rate Limit, uncomment below + limit_req zone=limit100; #burst=110; # Set Limit and burst here + limit_req_status 429; # Set HTTP Status Code, better than 503s + # limit_req_dry_run on; # Test the Rate limit, logged, but not enforced + add_header X-Ratelimit-Status $limit_req_status; # Add a custom status header + + proxy_pass http://aks1_ingress; # Proxy to AKS1 Nginx Ingress Controllers + add_header X-Proxy-Pass aks1_ingress_juiceshop; # Custom Header + + } + + ``` + + Submit your Nginx Configuration. + +1. Test again with Chrome and Dev Tools, Refresh, and verify that you now see `HTTP Status Code 429` for limited requests. + +  + + Ater consulation with your Dev team and testing different limits, you decide to change the limit to `1,000 Requests/Second`, to accommodate the normal traffic profile of the Juiceshop application. *This is likely an exercise you will have to do for every application using limits - determine normal traffic patterns and backend system performance, and set limits appropriately.* + +1. Using the Nginx for Azure Console, change the `req_limit` to the `limit1000` zone as shown (and change Burst value if using it): + + ```nginx + location / { + + # return 200 "You have reached juiceshop server block, location /\n"; + + # Set Rate Limit, uncomment below + limit_req zone=limit1000; #burst=1100; # Set Limit and burst here + limit_req_status 429; # Set HTTP Status Code, better than 503s + # limit_req_dry_run on; # Test the Rate limit, logged, but not enforced + add_header X-Ratelimit-Status $limit_req_status; # Add a custom status header + + proxy_pass http://aks1_ingress; # Proxy to AKS1 Nginx Ingress Controllers + add_header X-Proxy-Pass aks1_ingress_juiceshop; # Custom Header + + } + + ``` + + Submit your Nginx Configuration. + +1. Clear the Chrome Dev Tools display, and try Juiceshop again, much better! + + However, 1,000 Reqs/s may not quite be enough, if you Refresh several times, you will still see some 429 Status codes. + +  + +### Test Nginx Rate Limit Dry Run + +To make it easier to `fine-tune and test` your Rate Limits, Nginx provides the `limit_req_dry_run` directive. This creates the limit, but DOES NOT enforce the limit. So you can see the impact of the limit without actually dropping traffic - a very nice tool indeed! + +1. Using the Nginx for Azure Console, uncomment the `limit_req_dry_run on` directive. + + ```nginx + location / { + + # return 200 "You have reached juiceshop server block, location /\n"; + + # Set Rate Limit, uncomment below + limit_req zone=limit1000; #burst=1100; # Set Limit and burst here + limit_req_status 429; # Set HTTP Status Code, better than 503s + limit_req_dry_run on; # Test the Rate limit, logged, but not enforced + add_header X-Ratelimit-Status $limit_req_status; # Add a custom status header + + proxy_pass http://aks1_ingress; # Proxy to AKS1 Nginx Ingress Controllers + add_header X-Proxy-Pass aks1_ingress_juiceshop; # Custom Header + + } + + ``` + + Submit your Nginx Configuration. + +1. Test again with Chrome and Dev Tools. What do you see? You should see the `X-RateLimit-Status` Header now have some more metadata, like `REJECTED_DRY_RUN` - which means, this request exceeded the limit and `would be dropped` if Dry Run is disabled. You will also find this info in your Enhanced Logging format, using the $limit_req_status logging variable. + +  + +>**IMPORTANT NOTE:** The Rate Limit does NOT apply to the Regex location block for Juiceshop Images, WHY? Because you enabled the limit in the `/ location` block. As image requests do NOT match that location block REGEX, the Rate Limit (limit1000) does not apply, and the X-RateLimit-Status is not set. That is pretty cool, you can be very exact in where, how, and at what rate you apply Request Limits with Nginx! + +
-
@@ -56,6 +508,9 @@ By the end of the lab you will be able to: ## References: - [NGINX As A Service for Azure](https://docs.nginx.com/nginxaas/azure/) +- [NGINX Caching](https://www.nginx.com/products/nginx/caching/) +- [NGINX Caching Blog](https://www.nginx.com/blog/nginx-caching-guide/) +- [NGINX Caching Admin Guide](https://docs.nginx.com/nginx/admin-guide/content-cache/content-caching/) - [NGINX Plus Product Page](https://docs.nginx.com/nginx/) - [NGINX Ingress Controller](https://docs.nginx.com//nginx-ingress-controller/) - [NGINX on Docker](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-docker/) @@ -74,4 +529,4 @@ By the end of the lab you will be able to: ------------- -Navigate to ([Lab10](../lab10/readme.md) | [LabX](../labX/readme.md)) +Navigate to ([Lab10](../lab10/readme.md) | [LabGuide](../readme.md)) diff --git a/labs/labs-optional/readme.md b/labs/labs-optional/readme.md index 3d162ba..0a5109e 100644 --- a/labs/labs-optional/readme.md +++ b/labs/labs-optional/readme.md @@ -1,4 +1,4 @@ -# Optional Exercises / Grafana +# Optional Exercises / Grafana ## Introduction @@ -31,9 +31,75 @@ By the end of the lab you will be able to:
-### Lab exercise 1 +## Create and attach Azure Container Registry (ACR) + +1. Create a container registry using the `az acr create` command. The registry name must be unique within Azure, and contain 5-50 alphanumeric characters + ```bash + MY_RESOURCEGROUP=s.dutta + MY_ACR=acrshouvik + + az acr create \ + --resource-group $MY_RESOURCEGROUP \ + --name $MY_ACR \ + --sku Basic + ``` + +2. From the output of the `az acr create` command, make a note of the `loginServer`. The value of `loginServer` key is the fully qualified registry name. In our example the registry name is `acrshouvik` and the login server name is `acrshouvik.azurecr.io`. + +3. Login to the registry using below command. Make sure your local Docker daemon is up and running. + ```bash + MY_ACR=acrshouvik + + az acr login --name $MY_ACR + ``` + At the end of the output you should see `Login Succeeded`! + +### Test access to your Azure ACR + +We can quickly test the ability to push images to our Private ACR from our client machine. + +1. If you do not have a test container image to push to ACR, you can use a simple container for testing, e.g.[nginxinc/ingress-demo](https://hub.docker.com/r/nginxinc/ingress-demo). You will use this same container for the lab exercises. + + ```bash + az acr import --name $MY_ACR --source docker.io/nginxinc/ingress-demo:latest --image nginxinc/ingress-demo:v1 + ``` + The above command pulls the `nginxinc/ingress-demo` image from docker hub and pushes it to Azure ACR. + +2. Check if the image was successfully pushed to ACR using the azure cli command below: + + ```bash + MY_ACR=acrshouvik + az acr repository list --name $MY_ACR --output table + ``` + ```bash + ###Sample Output### + Result + --------------------- + nginxinc/ingress-demo + ``` + +### Attach an Azure Container Registry (ACR) to Azure Kubernetes cluster (AKS) + +1. You will attach the newly created ACR to both AKS clusters. This will enable you to pull private images within AKS clusters directly from your ACR. Run below command to attach ACR to 1st AKS cluster: + ```bash + MY_RESOURCEGROUP=s.dutta + MY_AKS=aks-shouvik # first cluster + MY_ACR=acrshouvik + + az aks update -n $MY_AKS -g $MY_RESOURCEGROUP --attach-acr $MY_ACR + ``` + +1. Change the $MY_AKS environment variable, so you can attach your ACR to your second Cluster: + ```bash + MY_RESOURCEGROUP=s.dutta + MY_AKS=aks2-shouvik # change to second cluster + MY_ACR=acrshouvik + + az aks update -n $MY_AKS -g $MY_RESOURCEGROUP --attach-acr $MY_ACR + ``` + + **NOTE:** You need the Owner, Azure account administrator, or Azure co-administrator role on your Azure subscription. To avoid needing one of these roles, you can instead use an existing managed identity to authenticate ACR from AKS. See [references](#references) for more details. -
-### Lab 0 - Prequesites - Subscription / Resources - -- Overview -In this Lab, the requirements for both the student and the Azure environment will be detailed. It is imperative that you have the appropriate computer, tools, and Azure access to successfully complete the workshop. The lab exercises must be done sequentially to build the environment as described. - -- Learning Objectives -Verify you have the proper computer requirements - hardware and software. -- Hardware: Laptop, Admin rights, Internet connection -- Software: Visual Studio, Terminal, Chrome, Docker, AKS and AZ CLI. -Verify you have proper computer skills. -- Computer skills: Linux CLI, files, SSH/Terminal, Docker/Compose, Azure Portal, HTTP/S, Load Balancing concepts -- Optional: TLS, DNS, FIPS, HTTP caching, Prometheus, Grafana -Verify you have the proper access to Azure resources. -- Azure subscription, list of Azure Roles/permissions here - -- Nginx for Azure Workshop has minimum REQUIRED Nginx Skills -Students must be familiar with Nginx basic operation, configurations, and concepts for HTTP traffic. --- The Nginx Basics Workshop is HIGHLY recommended, students should have taken this workshop prior. --- The Nginx Plus Ingress Controller workshop is also HIGHLY, students should have taken this workshop prior. +## NGINXaaS for Azure Workshop
-### Lab 1 - Azure VNet and Subnet and Network Security Group - -- Overview -In this lab, you will be adding and configuring the Azure Networking components needed for this workshop. This will require only a few network resources, and a Network Security Group to allow incoming traffic to your Azure resources. - -- Learning Objectives -Setup your Azure Vnet -Setup your Azure Subnets -Setuo your Azure Network Security group for inbound traffic +### Overview
-### Lab 2 - Nginx for Azure Overview and Deployment - -- Overview -In this lab, you will deploy and config a new Nginx for Azure instance. - -- Learning Objectives -Deploy Nginx for Azure -Enable Log Analytics -Test basic HTTP traffic -Create inital Nginx configurations to test with +> >Welcome to the NGINXaaS for Azure Workshop!
-### Lab 3 - UbuntuVM, Docker, and Cafe or Garage Demo Deployment +This **NGINXperts Workshop** will introduce **`NGINXaaS for Azure`** with hands-on practice through lab exercises. -## Overview -In this lab, you will deploy an Ubuntu VM, install Docker and Docker Compose, and configure it for a Legacy web application. You will configure Nginx for Azure to Load balance this application. +You will learn and explore NGINX for Azure, deploy and configure it with various Azure Resources. You will use many NGINX Plus features, for routing traffic, terminate TLS, splitting traffic, and caching. You will build a sample Enterprise environment in Azure with apps and services in Linux and Windows VMs, use Docker, and multiple Kubernetes AKS clusters. You will terminate TLS, route HTTP/S traffic, load balance running VMs, containers, pods and Nginx Ingress Controllers. You will configure Advanced Nginx Plus features like Caching and Dynamic Split Clients for Blue/Green testing, using live traffic. You will expose both Web and TCP applications on the Internet. You will explore the integrations of Nginx with Azure Cloud Resources like Key Vault, Monitoring, Logging/Analytics, and Grafana. -## Learning Objectives +The Hands-on Lab Exercises are designed to build upon each other, adding additional services and features as you progress through them, completing the labs in sequential order is required. You will follow along as an instructor guides you through these exercises. -- Deploy Ubuntu VM with Azure CLI -- Install Docker and Docker Compose -- Run Nginx demo application containers -- Test and validate your lab environment -- Configure Nginx for Azure to load balance the Docker Containers on the UbuntuVM -- Test your Nginx for Azure configs +This is the third Workshop in the `NGINXperts Series' from the Nginx Communities and Alliances Team at Nginx.
-### Lab 4 - AKS / Nginx Ingress / Redis / Cafe or Garage Demo Deployment - -- Overview -In this lab, you will deploy 2 AKS clusters, with Nginx Ingress Controllers, a Redis cluster, and a Modern Web Application. - -- Learning Objectives -Deploy 2 AKS clusters using the Azure AZ CLI. -Pull and Push the Nginx Plus Ingress Controller image to Azure Container Registry, and deploy to the Clusters. -Deploy and test a Redis In-Memory Cache to the AKS cluster. -Configure Nginx Ingress for Redis Leader traffic. -Test access to Redis Leader and Follower services. -Deploy a modern web application in the cluster. -Configure Nginx Ingress Controller to route traffic to the application. +NGINXaaS for Azure | NGINXperts Workshops +:-------------------------:|:-------------------------: + | 
-### Lab 5 - Nginx Load Balancing / Reverse Proxy - -- Overview -In this lab, you will configure Nginx for Azure to Load Balance various workloads running in Azure. After successful configuration and adding Best Practice Nginx parameters, you will Load Test these applications, and test multiple load balancing and request routing parameters to suit different use cases. +The Hands-On Lab Exercises are designed to build upon each other, adding additional services and features as you progress through them. `It is important to complete the lab exercises in sequential order.` --- Learning Objectives - -- Configure Nginx for Azure, to load balance traffic to both AKS Nginx Ingress Controllers. -- Expose the NIC Plus Dashboards externally for Live Monitoring -- Configure Nginx for Azure, to Proxy to Windows Server VM -- Run Load tests on the Legacy and Modern web applications. -- Configure HTTP Split Clients, and route traffic to 2 or 3 backend upstreams. -- Configure Nginx for Azure to load balance directly to Nginx Ingress Controllers without NodePort. -- Configure Nginx for Azure and Nginx Ingress for Redis Caching +By the end of this Workshop, you will have a working, operational NGINX for Azure deployment and Lab environment, with the skills to deploy and operate one for your Modern Application projects in Azure.
-### Lab 6 - Azure Key Vault / TLS Essentials - -- Overview -In this lab, you use Azure Key Vault for TLS certificates and keys. You will configure Nginx for Azure to use these Azure resources to terminate TLS. - -- Learning Objectives -Create a sample Azure Key Vault -Create a TLS cert/key -Configure and test Nginx for Azure to use the Azure Keys -Update the previous Nginx configurations to use TLS for apps -Update NSGs for TLS inbound traffic - -
+### Prerequisites -### Lab 7 - Azure Montoring / Logging Analytics +See the [Lab0 Readme.md](lab0/readme.md) for details on Student Prerequisites for this Workshop. -- Overview -Enable and configure Azure Monitoring for Nginx for Azure. Create custom Azure Dashboards for your applications. Gain experience using Azure Logs and logging tools. + -- Learning Objectives -Enable, configure, and test Azure Monitoring for Nginx for Azure. -Create a couple custom dashboards for your load balanced applications. -Explore the Azure logging and Analytics tools available. +NGINXaaS for Azure | NGINX Plus | Kubernetes | Docker | Redis +:-------------------------:|:-------------------------:|:-------------------------:|:-------------------------:|:-------------------------: + |  |  |  | 
-### Lab 8 - Nginx Garage or Azure Petshop +## Lab Outline -- Overview -In this lab, you will deploy a modern application in your AKS cluster. You will expose it with Nginx Ingress Controller and Nginx for Azure. +### Lab 0: Prerequesites - Azure Subscription / Resources +- [Lab 0: Prerequesites - Azure Subscription / Resources](lab0/readme.md) -- Learning Objectives -Deploy the modern app in AKS -Test and Verify the app is working correctly -Expose this application outside the cluster with Nginx Ingress Controller -Configure Nginx for Azure for this new application +### Lab 1: Azure VNet and Subnet and Network Security Group +- [Lab 1: Azure VNet and Subnet and Network Security Group](lab1/readme.md) -
+### Lab 2: Nginx for Azure Overview and Deployment +- [Lab 2: Nginx for Azure Overview and Deployment](lab2/readme.md) -### Lab 9 - Nginx and AzureAD / Entra Integration +### Lab 3: Ubuntu VM / Docker / Windows VM / Cafe Demo +- [Lab 3: Ubuntu VM / Docker / Windows VM / Cafe Demo](lab3/readme.md) -- Overview -In this lab, you will create and configure an Azure Active Directory integration that will provide User based authentication to your web application. You will then create and test the Nginx for Azure AD configuration that will enforce this user authentication requirement. +### Lab 4: AKS / Nginx Ingress Controller / Cafe Demo / Redis +- [Lab 4: AKS / Nginx Ingress Controller / Cafe Demo / Redis](lab4/readme.md) -- Learning Objectives -Create and configure Azure AD Security settings. -Create and configure Nginx for Azure to provide user authentication to your web application. -Test and validate AzureAD is working as expected. -Explore AzureAD related logging. +### Lab 5: Nginx Load Balancing / Blue-Green / Split Clients +- [Lab 5: Nginx Load Balancing / Blue-Green / Split Clients](lab5/readme.md) -
+### Lab 6: Azure Monitoring / Logging Analytics +- [Lab 6: Azure Monitoring / Logging Analytics](lab6/readme.md) -### Lab 10 - Nginx Caching / Garage / Juiceshop +### Lab 7: Azure Key Vault / TLS Essentials +- [Lab 7: Azure Key Vault / TLS Essentials](lab7/readme.md) -- Overview -In this lab, you will deploy an image rich application, and use Nginx Caching to cache images to improve performance. +### Lab 8: Nginx Garage Demo +- [Lab 8: Nginx Garage Demo](lab8/readme.md) -- Learning Objectives -Deploy JuiceShop in AKS cluster. -Expose JuiceShop with Nginx Ingress Controller -Configure Nginx for Azure for load balancing JuiceShop. -Add Nginx Caching to improve delivery of images. +### Lab 9: Nginx Caching / Rate Limits / Juiceshop +- [Lab9: Nginx Caching / Rate Limits / Juiceshop](lab9/readme.md) -
+### Lab 10: Nginx with Grafana for Azure +- [Lab10: Nginx with Grafana for Azure](lab10/readme.md) -### Lab 11 - Optional Exercises +#### Labs Optional: Optional Exercises +- [Labs Optional: Optional Exercises](labs-optional/readme.md) -Optional - Grafana with Azure -- Overview -In this lab, you will explore the Nginx and Grafana for Azure integration. +
-- Learning Objectives -Deploy Grafana for Azure. -Configure the Datasource -Explore a sample Grafana Dashboard for Nginx for Azure +### Authors +- Chris Akker - Solutions Architect - Community and Alliances @ F5, Inc. +- Shouvik Dutta - Solutions Architect - Community and Alliances @ F5, Inc. +- Adam Currier - Solutions Architect - Community and Alliances @ F5, Inc. +- Steve Wagner - Solutions Architect - Community and Alliances @ F5, Inc.
-### Summary and Wrap-up +Click [Lab0: Student Prerequisites](lab0/readme.md) for details on Student Prerequisite Requirements for this Workshop. + +Click [Lab1: Azure VNet and Subnet and Network Security Group](lab1/readme.md) to get started! -- Summary and Wrap-up comments here \ No newline at end of file diff --git a/n4a-configs/etc/nginx/conf.d/aks1-upstreams.conf b/n4a-configs/etc/nginx/conf.d/aks1-upstreams.conf new file mode 100644 index 0000000..444156e --- /dev/null +++ b/n4a-configs/etc/nginx/conf.d/aks1-upstreams.conf @@ -0,0 +1,15 @@ +upstream aks1_ingress { + zone aks1_ingress 256k; + + least_time last_byte; + + # from nginx-ingress NodePort Service / aks Node names + # Note: change servers to match + # + server aks-nodepool1-_AKS1_NODES_-vmss000000:32080; #aks1 node1 + server aks-nodepool1-_AKS1_NODES_-vmss000001:32080; #aks1 node2 + server aks-nodepool1-_AKS1_NODES_-vmss000002:32080; #aks1 node3 + + keepalive 32; + +} diff --git a/n4a-configs/etc/nginx/conf.d/aks2-upstreams.conf b/n4a-configs/etc/nginx/conf.d/aks2-upstreams.conf new file mode 100644 index 0000000..2657b55 --- /dev/null +++ b/n4a-configs/etc/nginx/conf.d/aks2-upstreams.conf @@ -0,0 +1,16 @@ +upstream aks2_ingress { + zone aks2_ingress 256k; + + least_time last_byte; + + # from nginx-ingress NodePort Service / aks Node names + # Note: change servers to match + # + server aks-nodepool1-_AKS2_NODES_-vmss000000:32080; #aks2 node1 + server aks-nodepool1-_AKS2_NODES_-vmss000001:32080; #aks2 node2 + server aks-nodepool1-_AKS2_NODES_-vmss000002:32080; #aks2 node3 + server aks-nodepool1-_AKS2_NODES_-vmss000003:32080; #aks2 node4 + + keepalive 32; + +} diff --git a/n4a-configs/etc/nginx/conf.d/my-garage-upstreams.conf b/n4a-configs/etc/nginx/conf.d/my-garage-upstreams.conf new file mode 100644 index 0000000..7ac3084 --- /dev/null +++ b/n4a-configs/etc/nginx/conf.d/my-garage-upstreams.conf @@ -0,0 +1,10 @@ + upstream my_garage { + zone my_garage 256k; + + server aks-nodepool1-_AKS2_NODES_-vmss000000:32080; #aks2 node1 + server aks-nodepool1-_AKS2_NODES_-vmss000001:32080; #aks2 node2 + server aks-nodepool1-_AKS2_NODES_-vmss000002:32080; #aks2 node3 + server aks-nodepool1-_AKS2_NODES_-vmss000003:32080; #aks2 node4 + + keepalive 8; + } \ No newline at end of file diff --git a/n4a-configs/etc/nginx/conf.d/my-garage.conf b/n4a-configs/etc/nginx/conf.d/my-garage.conf new file mode 100644 index 0000000..4108494 --- /dev/null +++ b/n4a-configs/etc/nginx/conf.d/my-garage.conf @@ -0,0 +1,14 @@ +server { + listen 80; + + server_name my-garage.example.com; + status_zone my-garage.example.com; # Metrics zone name + + access_log /var/log/nginx/my-garage.example.com.log main_ext; + error_log /var/log/nginx/my-garage.example.com_error.log info; + + location / { + proxy_pass http://my_garage; + add_header X-Proxy-Pass my-garage; # Custom Header + } +} diff --git a/n4a-configs/etc/nginx/conf.d/nic1-dashboard-upstreams.conf b/n4a-configs/etc/nginx/conf.d/nic1-dashboard-upstreams.conf new file mode 100644 index 0000000..43e37b4 --- /dev/null +++ b/n4a-configs/etc/nginx/conf.d/nic1-dashboard-upstreams.conf @@ -0,0 +1,11 @@ +upstream nic1_dashboard { + zone nic1_dashboard 256k; + + # from nginx-ingress NodePort Service / aks1 Node IPs + server aks-nodepool1-_AKS1_NODES_-vmss000000:32090; #aks1 node1 + server aks-nodepool1-_AKS1_NODES_-vmss000001:32090; #aks1 node2 + server aks-nodepool1-_AKS1_NODES_-vmss000002:32090; #aks1 node3 + + keepalive 8; + +} diff --git a/n4a-configs/etc/nginx/conf.d/nic1-dashboard.conf b/n4a-configs/etc/nginx/conf.d/nic1-dashboard.conf new file mode 100644 index 0000000..e9fc063 --- /dev/null +++ b/n4a-configs/etc/nginx/conf.d/nic1-dashboard.conf @@ -0,0 +1,18 @@ +server { + listen 9001; + server_name dashboard.example.com; + access_log off; + + location = /dashboard.html { + #return 200 "You have reached /nic1dashboard."; + + proxy_pass http://nic1_dashboard; + + } + + location /api/ { + + proxy_pass http://nic1_dashboard; + } + +} diff --git a/n4a-configs/etc/nginx/conf.d/nic2-dashboard-upstreams.conf b/n4a-configs/etc/nginx/conf.d/nic2-dashboard-upstreams.conf new file mode 100644 index 0000000..2fdfd9b --- /dev/null +++ b/n4a-configs/etc/nginx/conf.d/nic2-dashboard-upstreams.conf @@ -0,0 +1,11 @@ +upstream nic2_dashboard { + zone nic2_dashboard 256k; + + # from nginx-ingress NodePort Service / aks Node IPs + server aks-nodepool1-_AKS2_NODES_-vmss000000:32090; #aks2 node1 + server aks-nodepool1-_AKS2_NODES_-vmss000001:32090; #aks2 node2 + server aks-nodepool1-_AKS2_NODES_-vmss000002:32090; #aks2 node3 + server aks-nodepool1-_AKS2_NODES_-vmss000003:32090; #aks2 node4 + + keepalive 8; +} diff --git a/n4a-configs/etc/nginx/conf.d/nic2-dashboard.conf b/n4a-configs/etc/nginx/conf.d/nic2-dashboard.conf new file mode 100644 index 0000000..db349aa --- /dev/null +++ b/n4a-configs/etc/nginx/conf.d/nic2-dashboard.conf @@ -0,0 +1,18 @@ +server { + listen 9002; + server_name dashboard.example.com; + access_log off; + + location = /dashboard.html { + #return 200 "You have reached /nic2dashboard."; + + proxy_pass http://nic2_dashboard; + + } + + location /api/ { + + proxy_pass http://nic2_dashboard; + } + +} diff --git a/n4a-configs/etc/nginx/conf.d/the-garage-upstreams.conf b/n4a-configs/etc/nginx/conf.d/the-garage-upstreams.conf new file mode 100644 index 0000000..3387ffd --- /dev/null +++ b/n4a-configs/etc/nginx/conf.d/the-garage-upstreams.conf @@ -0,0 +1,14 @@ +upstream the_garage { + zone the_garage 256k; + + least_time last_byte; + + # NOTE: These are for use on port 8080 + server aks-nodepool1-_AKS2_NODES_-vmss000000:32080; #aks2 node1 + server aks-nodepool1-_AKS2_NODES_-vmss000001:32080; #aks2 node2 + server aks-nodepool1-_AKS2_NODES_-vmss000002:32080; #aks2 node3 + server aks-nodepool1-_AKS2_NODES_-vmss000003:32080; #aks2 node4 + + keepalive 32; + +} \ No newline at end of file diff --git a/n4a-configs/etc/nginx/conf.d/the-garage.conf b/n4a-configs/etc/nginx/conf.d/the-garage.conf new file mode 100644 index 0000000..51f11b2 --- /dev/null +++ b/n4a-configs/etc/nginx/conf.d/the-garage.conf @@ -0,0 +1,14 @@ +server { + listen 8080; + + server_name the-garage.example.com; + status_zone the-garage.example.com; # Metrics zone name + + access_log /var/log/nginx/the-garage.example.com.log main_ext; + error_log /var/log/nginx/the-garage.example.com_error.log info; + + location / { + proxy_pass http://the_garage; + add_header X-Proxy-Pass the-garage; # Custom Header + } +} diff --git a/n4a-configs/etc/nginx/includes/keepalive.conf b/n4a-configs/etc/nginx/includes/keepalive.conf new file mode 100644 index 0000000..17c6cc9 --- /dev/null +++ b/n4a-configs/etc/nginx/includes/keepalive.conf @@ -0,0 +1,8 @@ +# Default is HTTP/1.0 to upstreams, keepalives is only enabled for HTTP/1.1 +proxy_http_version 1.1; + +# Set the Connection header to empty +proxy_set_header Connection ""; + +# Host request header field, or the server name matching a request +proxy_set_header Host $host; diff --git a/n4a-configs/var/nginx.conf b/n4a-configs/var/nginx.conf new file mode 100644 index 0000000..d195ac9 --- /dev/null +++ b/n4a-configs/var/nginx.conf @@ -0,0 +1,55 @@ +user nginx; +worker_processes auto; +worker_rlimit_nofile 8192; +pid /run/nginx/nginx.pid; + +events { + worker_connections 4000; +} + +error_log /var/log/nginx/error.log error; + +http { + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + log_format main_ext 'remote_addr="$remote_addr", ' + '[time_local=$time_local], ' + 'request="$request", ' + 'status="$status", ' + 'http_referer="$http_referer", ' + 'body_bytes_sent="$body_bytes_sent", ' + 'Host="$host", ' + 'sn="$server_name", ' + 'request_time=$request_time, ' + 'http_user_agent="$http_user_agent", ' + 'http_x_forwarded_for="$http_x_forwarded_for", ' + 'request_length="$request_length", ' + 'upstream_address="$upstream_addr", ' + 'upstream_status="$upstream_status", ' + 'upstream_connect_time="$upstream_connect_time", ' + 'upstream_header_time="$upstream_header_time", ' + 'upstream_response_time="$upstream_response_time", ' + 'upstream_response_length="$upstream_response_length", '; + + access_log off; + server_tokens ""; + server { + listen 80 default_server; + server_name localhost; + location / { + # Points to a directory with a basic html index file with + # a "Welcome to NGINX as a Service for Azure!" page + root /var/www; + index index.html; + } + } + + include /etc/nginx/conf.d/*.conf; + include /etc/nginx/includes/*.conf; # shared files +} + +stream { + include /etc/nginx/stream/*.conf; # Stream TCP nginx files +} \ No newline at end of file diff --git a/n4a-configs/var/www/index.html b/n4a-configs/var/www/index.html new file mode 100644 index 0000000..867de81 --- /dev/null +++ b/n4a-configs/var/www/index.html @@ -0,0 +1,39 @@ + + + + +
Welcome to NGINX as a Service for Azure!
+If you see this page, the NGINX instance is successfully installed and + working. Further configuration is required.
+ +For online documentation, please refer to
+ docs.nginx.com/nginxaas/azure.
+
+
+
+
+ 
+
+
+
+
diff --git a/sd-notes/azure-cli.md b/sd-notes/azure-cli.md
deleted file mode 100644
index 0be44d6..0000000
--- a/sd-notes/azure-cli.md
+++ /dev/null
@@ -1,196 +0,0 @@
-## Azure CLI Basic Configuration Setting
-
-You will need Azure Command Line Interface (CLI) installed on your client machine to manage your Azure services. See [How to install the Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli)
-
-If you do not have Azure CLI installed, you will need to install it to continue with the lab exercises. To check Azure CLI version run below command:
-
-```bash
-az --version
-```
-
-1. Sign in with Azure CLI using your preferred method listed [here](https://learn.microsoft.com/en-us/cli/azure/authenticate-azure-cli).
-
- >**Note:** We made use of Sign in interactively method for this workshop
-
- ```bash
- az login
- ```
-
-1. Once you have logged in you can run below command to validate your tenent and subscription ID and name.
-
- ```bash
- az account show
- ```
-
-1. Optional: If you have multiple subscriptions and would like to change the current subscription to another then run below command.
-
- ```bash
- # change the active subscription using the subscription name
- az account set --subcription "{subscription name}"
-
- # OR
-
- # change the active subscription using the subscription ID
- az account set --subscription "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
- ```
-
-1. Create a new Azure Resource Group called `