Skip to content

Commit

Permalink
Merge pull request aws-observability#18 from open-o11y/update-readme
Browse files Browse the repository at this point in the history 
Update Readme and documentation for the ADOT Helm chart Repo
  • Loading branch information
@alolita
alolita authored Apr 6, 2022
2 parents 43d0210 + 1d1f5ed commit b6c6575
Show file tree
Hide file tree
Showing 8 changed files with 211 additions and 122 deletions.
2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
## Introduction
This AWS Distro for OpenTelemetry (ADOT) Helm Charts repository contains [Helm](https://helm.sh/) charts to provide easy mechanisms to setup the ADOT Collector and other collection agents such as Fluent Bit to collect telemetry data such as metrics, logs and traces to send to AWS monitoring services.

This repository contains a [Helm](https://helm.sh/) chart to provide easy to operate, end-to-end [AWS Elastic Kubernetes Service](https://aws.amazon.com/eks/) (EKS) on [AWS Elastic Compute Cloud](https://aws.amazon.com/ec2/) (EC2) monitoring with [AWS Distro for OpenTelemetry(ADOT) Collector](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-EKS-otel.html) for metrics and [Fluent Bit](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-logs-FluentBit.html) for logs. This Helm chart is useful for customers who use EKS on EC2 and want to collect metrics and logs to send to Amazon CloudWatch Container Insights.
This repository contains a [Helm](https://helm.sh/) chart to provide easy to operate, end-to-end [AWS Elastic Kubernetes Service](https://aws.amazon.com/eks/) (EKS) on [AWS Elastic Compute Cloud](https://aws.amazon.com/ec2/) (EC2) monitoring with [AWS Distro for OpenTelemetry(ADOT) Collector](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-EKS-otel.html) for metrics and [Fluent Bit](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-logs-FluentBit.html) for logs. This Helm chart is useful for customers who use EKS on EC2 and want to collect metrics and logs to send to Amazon CloudWatch Container Insights and Amazon Managed Service for Prometheus(AMP).

## Getting Started

Expand Down
144 changes: 23 additions & 121 deletions charts/adot-exporter-for-eks-on-ec2/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# ADOT Helm chart for EKS on EC2 metrics and logs to Amazon CloudWatch Container Insights
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)

This [Helm](https://helm.sh/) chart provides easy to use [AWS Elastic Kubernetes Service](https://aws.amazon.com/eks/) (EKS) on [AWS Elastic Compute Cloud](https://aws.amazon.com/ec2/) (EC2) monitoring with [AWS Distro for OpenTelemetry(ADOT) Collector](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-EKS-otel.html) for metrics and [Fluent Bit](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-logs-FluentBit.html) for logs. Therefore, the Helm chart is useful for customers who use EKS on EC2 and want to collect metrics and logs to send to Amazon CloudWatch Container Insights.
This [Helm](https://helm.sh/) chart provides easy to use [AWS Elastic Kubernetes Service](https://aws.amazon.com/eks/) (EKS) on [AWS Elastic Compute Cloud](https://aws.amazon.com/ec2/) (EC2) monitoring with [AWS Distro for OpenTelemetry(ADOT) Collector](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-EKS-otel.html) for metrics and [Fluent Bit](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-logs-FluentBit.html) for logs. Therefore, the Helm chart is useful for customers who use EKS on EC2 and want to collect metrics and logs from clusters. The metrics will be sent to Amazon CloudWatch Container Insights and [Amazon Managed Service for Prometheus](https://aws.amazon.com/prometheus/?trk=f6e79447-9b4c-4310-8415-1a76de2de47f&sc_channel=ps&sc_campaign=acquisition&sc_medium=ACQ-P|PS-GO|Non-Brand|Desktop|SU|Management%20Tools|Solution|US|EN|DSA&ef_id=CjwKCAiAg6yRBhBNEiwAeVyL0KLIKHm3fznhVURTI6T-WBvANCmqo3r0-pYp_U82lIDDMmXRXDk0DBoCohQQAvD_BwE:G:s&s_kwcid=AL!4422!3!579408286031!!!g!!) (AMP) and the logs will be sent to Amazon Cloudwatch.

The Helm chart configured in this repository deploys the ADOT Collector and Fluent Bit as [DaemonSets](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) and is ready to collect metrics and logs and send them to Amazon CloudWatch Container Insights.
The Helm chart configured in this repository deploys the ADOT Collector and Fluent Bit as [DaemonSets](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) and is ready to collect metrics and logs and send them to Amazon CloudWatch Container Insights and AMP.

## Helm Chart Structure
```console
Expand Down Expand Up @@ -35,6 +35,10 @@ adot-exporter-for-eks-on-ec2/
| | |-- serviceaccount.yaml
| | |-- sidecar.yaml
| | |-- sidecarnamespace.yaml
|-- documentation
| |-- metrics_to_CloudWatch_AMP.md
| |-- deploy_collector_as_sidcar.md
| |-- aws_logging_on_Fargate.md
|-- Chart.yaml
|-- values.schema.json
|-- values.yaml
Expand All @@ -58,6 +62,9 @@ The following prerequisites need to be set up in order to install this Helm char

- Your EKS Cluster on EC2
- [Amazon CloudWatch Container Insights prerequisites](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-prerequisites.html)
- Using the above instructions, please attach the ‘prometheusremotewriteacess’ policy to the nodes.
- Your working [Amazon Managed Service for Prometheus(AMP) workspace](https://docs.aws.amazon.com/prometheus/latest/userguide/AMP-onboard-create-workspace.html).
- [Amazon Managed Service for Grafana](https://aws.amazon.com/grafana/) is the go-to platform for visualizing AMP data.
- [Helm v3+](https://helm.sh/docs/helm/helm_install/)

## Get Repository Information
Expand All @@ -70,62 +77,6 @@ $ helm repo add [REPO_NAME] https://aws-observability.github.io/aws-otel-helm-ch
$ helm search repo [REPO_NAME] # Run this command in order to see the charts.
```

## Install Chart

```console
$ helm install \
[RELEASE_NAME] [REPO_NAME]/adot-exporter-for-eks-on-ec2 \
--set clusterName=[CLUSTER_NAME] --set awsRegion=[AWS_REGION]
```
`CLUSTER_NAME` and `AWS_REGION` must be specified with your own EKS cluster and the region.
You can find these values by executing following command.

```console
$ kubectl config current-context

[IAM_User_Name]@[CLUSTER_NAME].[AWS_REGION].eksctl.io
```

To verify the installation is successful, you can execute the following command.

```console
$ kubectl get pods --all-namespaces

NAMESPACE NAME READY STATUS RESTARTS AGE
amazon-cloudwatch fluent-bit-f27cz 1/1 Running 0 4s
amazon-cloudwatch fluent-bit-m2mkr 1/1 Running 0 4s
amzn-cloudwatch-metrics adot-collector-daemonset-7nrst 1/1 Running 0 4s
amzn-cloudwatch-metrics adot-collector-daemonset-x7n8x 1/1 Running 0 4s
```

If you see these four running pods, two for Fluent Bit and two for ADOT Collector as [DaemonSets](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) within the specified namespaces, they are successfully deployed.

### Verify the Helm chart works as expected
- Run chart validation test and lint from`MakeFile`.
```console
$ cd adot-exporter-for-eks-on-ec2
$ make install-tools # required initially
$ make all # to run chart validation test and lint
```

### Verify the metrics and logs are sent to Amazon CloudWatch

- Open Amazon CloudWatch console
- Select "Logs -> Log groups" on the left navigation bar.
- Check if following four log groups exist (performance log group will take longer than others).
```console
/aws/containerinsights/[CLUSTER_NAME]/application
/aws/containerinsights/[CLUSTER_NAME]/dataplane
/aws/containerinsights/[CLUSTER_NAME]/host
/aws/containerinsights/[CLUSTER_NAME]/performance
```
- Select "Insights -> Container Insights" on the left navigation bar.
- Choose Performance monitoring in the drop-down menu on the top-left side.
- Choose the levels such as EKS pods, EKS nodes, and EKS namespaces from the drop-down menu in the automated dashboard.
- If you observe metrics of the running pods for CPU Utilization, Memory Utilization, etc, the metrics are successfully collected and visualized in Container Insights.

![CWCI_dashboard](https://user-images.githubusercontent.com/38146012/141032708-9080ed8a-ff68-4227-8ea5-98c8fd5deff8.jpeg)

## Configuration
To see all configurable options with detailed comments:

Expand All @@ -137,77 +88,20 @@ By changing values in `values.yaml`, you are able to customize the chart to use

Following options are some useful configurations that can be applied to this Helm chart.

### Deploy ADOT Collector as a Sidecar

A sidecar is a microservice design pattern where a companion service runs next to your primary microservice, augmenting its abilities or intercepting resources it is utilizing. The sidecar pattern would be the best fit for a single application monitoring.
### Documentation

In order to deploy the ADOT Collector in Sidecar mode using the Helm chart, 1) update `sidecar.yaml` and `values.yaml` files in the Helm chart with the application configurations and 2) include the use of `--set` flag in the `helm install` command from [Install Chart](#install-chart).
Please follow the links to get more details for specific use cases and deployment of the collector.

```console
$ helm install \
[RELEASE_NAME] [REPO_NAME]/adot-exporter-for-eks-on-ec2 \
--set clusterName=[CLUSTER_NAME] --set awsRegion=[AWS_REGION] \
--set adotCollector.daemonSet.enabled=false --set adotCollector.sidecar.enabled=true
```
The use of `--set` flag with `enabled=true` or `enabled=false` can switch on/off the specified deployment mode. The command set `enabled=false` for ADOT Collector as DaemonSet and `enabled=true` to deploy ADOT Collector as Sidecar.
You can also check whether your applications are successfully deployed by executing the following command.

```console
$ kubectl get pods --all-namespaces
* To send metrics to CloudWatch and Amazon Managed Service for Prometheus(AMP) and logs to CloudWatch. Click [here](documentation/metrics_to_cloudwatch_amp.md).
* To deploy the Adot Collector as a Sidecar. Click [here](documentation/deploy_collector_as_sidecar.md).
* To deploy EKS on AWS Fargate. Click [here](documentation/aws_logging_on_fargate.md).
* The various scenarious under which the Helm chart was tested has been documented [here](documentation/helm_chart_test_doc.pdf).

NAMESPACE NAME READY STATUS RESTARTS AGE
adot-sidecar-namespace adot-sidecar-658dc9ffbb-w9zv2 2/2 Running 0 5m18s
amazon-cloudwatch fluent-bit-9dcql 1/1 Running 0 5m18s
amazon-cloudwatch fluent-bit-wqhmd 1/1 Running 0 5m18s
```

### Deploy ADOT Collector as Deployment and StatefulSet

Deploying ADOT Collector as Deployment and StatefulSet mode requires installing ADOT Operator. See [OpenTelemetry Operator Helm Chart](https://github.com/open-telemetry/opentelemetry-helm-charts/tree/main/charts/opentelemetry-operator) for detailed explanation.


### Deploy ADOT Collector with Prometheus Receiver for AWS Container Insights on EKS

Please refer to [deployment template](https://github.com/aws-observability/aws-otel-collector/blob/main/deployment-template/eks/otel-container-insights-prometheus.yaml) to deploy ADOT Collector with [Prometheus Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/d46048ac4dd01062c803867cb6a13377ea287a23/receiver/prometheusreceiver/README.md#prometheus-receiver) and [Amazon CloudWatch Embedded Metric Format (EMF) Exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/awsemfexporter#aws-cloudwatch-emf-exporter-for-opentelemetry-collector) for AWS Container Insights on EKS
via [configurations](https://github.com/aws-observability/aws-otel-collector/blob/main/config/eks/prometheus/config-all.yaml) in the Helm chart.

### AWS EKS on Fargate
The prerequisites for [Fargate logging](https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html) via Amazon EKS on [AWS Fargate](https://docs.aws.amazon.com/eks/latest/userguide/fargate.html) include: 1) [Create a Fargate profile for your cluster](https://docs.aws.amazon.com/eks/latest/userguide/fargate-getting-started.html#fargate-gs-create-profile)
and 2) [Create a Fargate pod execution role](https://docs.aws.amazon.com/eks/latest/userguide/fargate-getting-started.html#fargate-sg-pod-execution-role). <br>

Amazon EKS on Fargate features a Fluent Bit based built-in log router to send collected logs to various destinations, including Amazon CloudWatch.
Fargate utilizes [AWS for Fluent Bit](https://github.com/aws/aws-for-fluent-bit),
and the required configurations for Fargate to automatically detect and configure the log router are included in the Helm chart in `configmap.yaml` and `values.yaml` files based on the [Fargate logging](https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html) user guide.
The configurations in `configmap.yaml` must include the name: `aws-logging` and the namespace: `aws-observability` for Fargate logging. To deploy your application to Amazon EKS on Fargate, you need to include your application yaml file
in the `aws-fargate-logging` folder of the Helm chart with the same namespace as your [AWS Fargate profile](https://docs.aws.amazon.com/eks/latest/userguide/fargate-profile.html). For more detailed information about Fargate logging, such as deployment of a `sample-app.yaml` or your application and
the instructions to download, create, and attach IAM policy to the [pod execution role](https://docs.aws.amazon.com/eks/latest/userguide/pod-execution-role.html) for Fargate profile,
please refer to the user guide for [Fargate logging](https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html) and [Getting started with AWS Fargate using Amazon EKS](https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html).

This is an example of using the Helm chart for Fargate logging with the `sample-app.yaml` from [Fargate logging](https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html).
```console
$ helm install \
[RELEASE_NAME] [REPO_NAME]/adot-exporter-for-eks-on-ec2 \
--set clusterName=[CLUSTER_NAME] --set awsRegion=[AWS_REGION] \
--set fargateLogging.enabled=true
```
To confirm the `sample-app` is deployed and troubleshoot the logging is enabled/disabled, you can run the following commands.
```console
$ kubectl get pods --all-namespaces

NAMESPACE NAME READY STATUS RESTARTS AGE
aws-observability sample-app-86b8cc866b-cr5x6 1/1 Running 0 13m
aws-observability sample-app-86b8cc866b-q75z7 1/1 Running 0 13m
aws-observability sample-app-86b8cc866b-t615c 1/1 Running 0 13m
```

```console
$ kubectl describe po -n aws-observability sample-app-86b8cc866b-cr5x6

Events:
Type Reason Age From Message
---- ------ --- ---- -------
Normal LoggingEnabled 13m fargate-scheduler Successfully enabled logging for pod
```
## Uninstall Chart

The following command uninstalls the chart.
Expand All @@ -233,12 +127,20 @@ See [CONTRIBUTING.md](../../CONTRIBUTING.md).

[James Park](https://github.com/JamesJHPark)

[Ruthvik Ravindra](https://github.com/ruthvik17)

## Other useful links

[Set up Fluent Bit as a DaemonSet to send logs to CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-logs-FluentBit.html)

[Using AWS Distro for OpenTelemetry](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-EKS-otel.html)

[Learn more about OpenTelemetry](https://opentelemetry.io/docs/)

## Questions

To learn more about how to use AWS Distro for OpenTelemetry to collect data for your observability solution, check out the hands-on [AWS Observability workshop](https://observability.workshop.aws/en/adot.html). For any questions and issues, please feel free to reach out via [AWS OTeL Community](https://github.com/aws-observability/aws-otel-community/issues).


## License

Expand Down
36 changes: 36 additions & 0 deletions charts/adot-exporter-for-eks-on-ec2/documentation/aws_logging_on_fargate.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# AWS EKS on Fargate

The prerequisites for [Fargate logging](https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html) via Amazon EKS on [AWS Fargate](https://docs.aws.amazon.com/eks/latest/userguide/fargate.html) include: 1) [Create a Fargate profile for your cluster](https://docs.aws.amazon.com/eks/latest/userguide/fargate-getting-started.html#fargate-gs-create-profile) and 2) [Create a Fargate pod execution role](https://docs.aws.amazon.com/eks/latest/userguide/fargate-getting-started.html#fargate-sg-pod-execution-role).

Amazon EKS on Fargate features a Fluent Bit based built-in log router to send collected logs to various destinations, including Amazon CloudWatch. Fargate utilizes [AWS for Fluent Bit](https://github.com/aws/aws-for-fluent-bit), and the required configurations for Fargate to automatically detect and configure the log router are included in the Helm chart in configmap.yaml and values.yaml files based on the [Fargate logging](https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html) user guide. The configurations in configmap.yaml must include the name: aws-logging and the namespace: aws-observability for Fargate logging. To deploy your application to Amazon EKS on Fargate, you need to include your application yaml file in the aws-fargate-logging folder of the Helm chart with the same namespace as your [AWS Fargate profile](https://docs.aws.amazon.com/eks/latest/userguide/fargate-profile.html).

For more detailed information about Fargate logging, such as deployment of a sample-app.yaml or your application and the instructions to download, create, and attach IAM policy to the [pod execution role](https://docs.aws.amazon.com/eks/latest/userguide/pod-execution-role.html) for Fargate profile, please refer to the user guide for [Fargate logging](https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html) and [Getting started with AWS Fargate using Amazon EKS](https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html).
This is an example of using the Helm chart for Fargate logging with the sample-app.yaml from [Fargate logging](https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html).

```console
$ helm install \
[RELEASE_NAME] [REPO_NAME]/adot-exporter-for-eks-on-ec2 \
--set clusterName=[CLUSTER_NAME] --set awsRegion=[AWS_REGION] \
--set fargateLogging.enabled=true
```

To confirm the `sample-app` is deployed and troubleshoot the logging is enabled/disabled, you can run the following commands.

```console
$ kubectl get pods --all-namespaces

NAMESPACE NAME READY STATUS RESTARTS AGE
AGEaws-observability sample-app-86b8cc866b-cr5x6 1/1 Running 0 13m
aws-observability sample-app-86b8cc866b-q75z7 1/1 Running 0 13m
aws-observability sample-app-86b8cc866b-t615c 1/1 Running 0 13m
```


```console
$ kubectl describe po -n aws-observability

sample-app-86b8cc866b-cr5x6Events:
Type Reason Age From Message
---- ------ --- ---- -------
Normal LoggingEnabled 13m fargate-scheduler Successfully enabled logging for pod
```
Loading

0 comments on commit b6c6575

Please sign in to comment.