diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..863f5e4 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,3 @@ +Version 2.0.0 +- Porting to aws-cloudwatch-exts-commons 2.0.1.1 +- config file extension changed from yaml to yml \ No newline at end of file diff --git a/README.md b/README.md index 95df7ac..38a2851 100644 --- a/README.md +++ b/README.md @@ -3,128 +3,162 @@ ## Use Case Captures Custom Namespace statistics from Amazon CloudWatch and displays them in the AppDynamics Metric Browser. -**Note : By default, the Machine agent can only send a fixed number of metrics to the controller. This extension potentially reports thousands of metrics, so to change this limit, please follow the instructions mentioned [here](https://docs.appdynamics.com/display/PRO40/Metrics+Limits).** +## Prerequisites +1. Before the extension is installed, the prerequisites mentioned [here](https://community.appdynamics.com/t5/Knowledge-Base/Extensions-Prerequisites-Guide/ta-p/35213) need to be met. Please do not proceed with the extension installation if the specified prerequisites are not met. +2. Please give the following permissions to the account being used to with the extension. **cloudwatch:ListMetrics cloudwatch:GetMetricStatistics** ## Installation -1. Run 'mvn clean install' from aws-customnamespace-monitoring-extension -2. Copy and unzip AWSCustomNamespaceMonitor-\.zip from 'target' directory into \/monitors/ -3. Edit config.yaml file in AWSCustomNamespaceMonitor/conf and provide the required configuration (see Configuration section) -4. Restart the Machine Agent. +1. Run `mvn clean install` from aws-customnamespace-monitoring-extension +2. Copy and unzip AWSCustomNamespaceMonitor-\.zip from 'target' directory into \/monitors/ directory. Do not place the extension in the `extensions` directory of your Machine Agent installation directory. +3. Edit the config.yml file located at MachineAgent_Home/monitors/AWSCustomNamespaceMonitor and provide the required configuration (see Configuration section) +4. The metricPrefix of the extension has to be configured as specified [here](https://community.appdynamics.com/t5/Knowledge-Base/How-do-I-troubleshoot-missing-custom-metrics-or-extensions/ta-p/28695#Configuring%20an%20Extension). Please make sure that the right metricPrefix is chosen based on your machine agent deployment, otherwise this could lead to metrics not being visible in the controller. +5. Restart the Machine Agent. ## Configuration - -### config.yaml - -**Note: Please avoid using tab (\t) when editing yaml files. You may want to validate the yaml file using a [yaml validator](http://yamllint.com/).** - -| Section | Fields | Description | Example | -| ----- | ----- | ----- | ----- | -| **accounts** | | Fields under this section can be repeated for multiple accounts config | | -| | awsAccessKey | AWS Access Key | | -| | awsSecretKey | AWS Secret Key | | -| | displayAccountName | Display name used in metric path | "MyAWSCustomNamespace" | -| | regions | Regions where CustomNamespace is registered | **Allowed values:**
"ap-southeast-1",
"ap-southeast-2",
"ap-northeast-1",
"eu-central-1",
"eu-west-1",
"us-east-1",
"us-west-1",
"us-west-2",
"sa-east-1" | -| | namespaces | The custom namespaces registered for this account that you want to monitor | "MyCustomNamespace1" | -| **credentialsDecryptionConfig** | ----- | ----- | ----- | -| | enableDecryption | If set to "true", then all aws credentials provided (access key and secret key) will be decrypted - see AWS Credentials Encryption section | | -| | decryptionKey | The key used when encypting the credentials | | -| **proxyConfig** | ----- | ----- | ----- | -| | host | The proxy host (must also specify port) | | -| | port | The proxy port (must also specify host) | | -| | username | The proxy username (optional) | | -| | password | The proxy password (optional) | | -| **metricsConfig** | ----- | ----- | ----- | -| metricTypes | | Fields under this section can be repeated for multiple metric types override | | -| | metricName | The metric name | "CPUUtilization" | -| | statType | The statistic type | **Allowed values:**
"ave"
"max"
"min"
"sum"
"samplecount" | -| | ----- | ----- | ----- | -| | excludeMetrics | Metrics to exclude - supports regex | "CPUUtilization",
"Swap.*" | -| metricsTimeRange | | | | -| | startTimeInMinsBeforeNow | The no of mins to deduct from current time for start time of query | 5 | -| | endTimeInMinsBeforeNow | The no of mins to deduct from current time for end time of query.
Note, this must be less than startTimeInMinsBeforeNow | 0 | -| | ----- | ----- | ----- | -| | maxErrorRetrySize | The max number of retry attempts for failed retryable requests | 1 | -| **concurrencyConfig** | | | | -| | noOfNamespaceThreads | The no of threads to process multiple namespaces concurrently | 3 | -| | noOfAccountThreads | The no of threads to process multiple accounts concurrently | 3 | -| | noOfRegionThreadsPerAccount | The no of threads to process multiple regions per account concurrently | 3 | -| | noOfMetricThreadsPerRegion | The no of threads to process multiple metrics per region concurrently | 3 | -| | ----- | ----- | ----- | -| | metricPrefix | The path prefix for viewing metrics in the metric browser. | "Custom Metrics\|Amazon Custom Namespace\|" | - - -**Below is an example config for monitoring multiple accounts, regions and namespaces:** - -~~~ -accounts: - - awsAccessKey: "XXXXXXXX1" - awsSecretKey: "XXXXXXXXXX1" - displayAccountName: "TestAccount_1" - regions: ["us-east-1","us-west-1","us-west-2"] - namespaces: ["Custom1"] - - - awsAccessKey: "XXXXXXXX2" - awsSecretKey: "XXXXXXXXXX2" - displayAccountName: "TestAccount_2" - regions: ["eu-central-1","eu-west-1"] - namespaces: ["Custom2", "Custom3"] - -credentialsDecryptionConfig: - enableDecryption: "false" - decryptionKey: - -proxyConfig: - host: - port: - username: - password: - -metricsConfig: - metricTypes: - - metricName: "GroupInServiceInstances" - statType: "max" - - - metricName: "GroupMinSize" - statType: "sum" - - excludeMetrics: ["GroupMaxSize", "GroupTotal.*"] - - metricsTimeRange: - startTimeInMinsBeforeNow: 5 - endTimeInMinsBeforeNow: 0 - - maxErrorRetrySize: 0 - -concurrencyConfig: - noOfNamespaceThreads: 3 - noOfAccountThreads: 3 - noOfRegionThreadsPerAccount: 3 - noOfMetricThreadsPerRegion: 3 - -metricPrefix: "Custom Metrics|Amazon Custom Namespace|" -~~~ - -### AWS Credentials Encryption -To set an encrypted awsAccessKey and awsSecretKey in config.yaml, follow the steps below: - -1. Download the util jar to encrypt the AWS Credentials from [here](https://github.com/Appdynamics/maven-repo/blob/master/releases/com/appdynamics/appd-exts-commons/1.1.2/appd-exts-commons-1.1.2.jar). -2. Run command: - - ~~~ - java -cp appd-exts-commons-1.1.2.jar com.appdynamics.extensions.crypto.Encryptor EncryptionKey CredentialToEncrypt - - For example: - java -cp "appd-exts-commons-1.1.2.jar" com.appdynamics.extensions.crypto.Encryptor test myAwsAccessKey - - java -cp "appd-exts-commons-1.1.2.jar" com.appdynamics.extensions.crypto.Encryptor test myAwsSecretKey - ~~~ - -3. Set the decryptionKey field in config.yaml with the encryption key used, as well as the resulting encrypted awsAccessKey and awsSecretKey in their respective fields. +In order to use the extension, you need to update the config.yml file that is present in the extension folder. The following is an explanation of the configurable fields that are present in the config.yml file. + +1. If SIM is enabled, then use the following metricPrefix `metricPrefix: "Custom Metrics|AWS DynamoDB"` else configure the `COMPONENT_ID` under which the metrics need to be reported. This can be done by changing the value of in `metricPrefix: "Server|Component:|Custom Metrics|AWS DynamoDB|"`. + + For example, + + `metricPrefix: "Server|Component:100|Custom Metrics|Amazon Custom Namespace|"` + +2. Provide accessKey(required) and secretKey(required) of AWS account(s), also provide displayAccountName(any name that represents your account) and regions(required). If you are running this extension inside an EC2 instance which has IAM profile configured then awsAccessKey and awsSecretKey can be kept empty, extension will use IAM profile to authenticate. + ``` + accounts: + - awsAccessKey: "XXXXXXXX1" + awsSecretKey: "XXXXXXXXXX1" + displayAccountName: "TestAccount_1" + regions: ["us-east-1","us-west-1","us-west-2"] + + - awsAccessKey: "XXXXXXXX2" + awsSecretKey: "XXXXXXXXXX2" + displayAccountName: "TestAccount_2" + regions: ["eu-central-1","eu-west-1"] + ``` + +3. Please specify the AWS Namespace that you would like to monitor. Please note that only 1 AWS Namespace can be monitored using this extension. Inorder to monitor multiple Namespaces, please have multiple copies of this extension. + Example is `namespace: "AWS/DynamoDB"` + +4. If you want to encrypt the "awsAccessKey" and "awsSecretKey" then follow the "Credentials Encryption" section and provide the encrypted values in "awsAccessKey" and "awsSecretKey". Configure "enableDecryption" of "credentialsDecryptionConfig" to true and provide the encryption key in "encryptionKey" For example, + ``` + #Encryption key for Encrypted password. + credentialsDecryptionConfig: + enableDecryption: "true" + encryptionKey: "XXXXXXXX" + ``` +5. Configure proxy details if there is a proxy between MachineAgent and AWS. + ``` + proxyConfig: + host: + port: + username: + password: + ``` +6. To report metrics from specific dimension values, configure the `dimesions` section. The dimensions varies with AWS Namespace. Please refer to AWS [doc](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#Dimension) for details on dimensions. + Example below: + ``` + dimensions: + - name: "TableName" + displayName: "Table Name" + values: ["ABC", "DEF"] + - name: "AvailabilityZone" + displayName: "Availability Zone" + values: [] + ``` + name is the dimension name and values are comma separated values of the dimension. displayName is the name with which it appears on the AppDynamics Metric Browser. + If these fields are left empty, none of your instances will be monitored. In order to monitor everything under a dimension, you can simply use ".*" to pull everything from your AWS Environment. + +7. Configure the metrics section. + + For configuring the metrics, the following properties can be used: + + | Property | Default value | Possible values | Description | + | :---------------- | :-------------- | :------------------------------ | :------------------------------------------------------------------------------------------------------------- | + | alias | metric name | Any string | The substitute name to be used in the metric browser instead of metric name. | + | statType | "ave" | "AVERAGE", "SUM", "MIN", "MAX" | AWS configured values as returned by API | + | aggregationType | "AVERAGE" | "AVERAGE", "SUM", "OBSERVATION" | [Aggregation qualifier](https://docs.appdynamics.com/display/PRO44/Build+a+Monitoring+Extension+Using+Java) | + | timeRollUpType | "AVERAGE" | "AVERAGE", "SUM", "CURRENT" | [Time roll-up qualifier](https://docs.appdynamics.com/display/PRO44/Build+a+Monitoring+Extension+Using+Java) | + | clusterRollUpType | "INDIVIDUAL" | "INDIVIDUAL", "COLLECTIVE" | [Cluster roll-up qualifier](https://docs.appdynamics.com/display/PRO44/Build+a+Monitoring+Extension+Using+Java)| + | multiplier | 1 | Any number | Value with which the metric needs to be multiplied. | + | convert | null | Any key value map | Set of key value pairs that indicates the value to which the metrics need to be transformed. eg: UP:0, DOWN:1 | + | delta | false | true, false | If enabled, gives the delta values of metrics instead of actual values. | + + For example, + ``` + - name: "ConsumedReadCapacityUnits" + alias: "Consumed Read Capacity Units" + statType: "ave" + delta: false + multiplier: 1 + aggregationType: "AVERAGE" + timeRollUpType: "AVERAGE" + clusterRollUpType: "INDIVIDUAL" + ``` + + **All these metric properties are optional, and the default value shown in the table is applied to the metric(if a property has not been specified) by default.** + +8. For several services AWS CloudWatch does not instantly update the metrics but it goes back in time to update that information. + This delay sometimes can take upto 5 minutes. The extension runs every minute(Detailed) or every 5 minutes (Basic) and gets the latest value at that time. + There may be a case where the extension may miss the value before CloudWatch updates it. In order to make sure we don't do that, + the extension has the ability to look for metrics during a certain interval, where we already have set it to default at 5 minutes but you can + change it as per your requirements. + ``` + metricsTimeRange: + startTimeInMinsBeforeNow: 10 + endTimeInMinsBeforeNow: 5 + ``` + +9. This field is set as per the defaults suggested by AWS. You can change this if your limit is different. + ``` + getMetricStatisticsRateLimit: 400 + ``` +10. The maximum number of retry attempts for failed requests that can be retried. + ``` + maxErrorRetrySize: 3 + ``` +11. CloudWatch can be used in two formats, Basic and Detailed. You can specify how you would like to run the extension by specifying the chosen format here. + By default, the extension is set to Basic, which makes the extension run every 5 minutes. + Refer https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-new.html for more information. + + ``` + #Allowed values are Basic and Detailed. + # Basic will fire CloudWatch API calls every 5 minutes + # Detailed will fire CloudWatch API calls every 1 minutes + cloudWatchMonitoring: "Basic" + ``` + +Please avoid using tab (\t) when editing yaml files. Please copy all the contents of the config.yml file and go to [Yaml Validator](http://www.yamllint.com/) . On reaching the website, paste the contents and press the “Go” button on the bottom left. +If you get a valid output, that means your formatting is correct and you may move on to the next step. ## Metrics -Typical metric path: **Application Infrastructure Performance|\|Custom Metrics|Amazon Custom Namespace|\|\|\** followed by the metrics defined by your custom namespace. +Typical metric path: **Application Infrastructure Performance|\|Custom Metrics|Amazon Custom Namespace|\|\|Region|Dimension Name|Dimension Value|** followed by the AWS Namespace configured metrics + +## Credentials Encryption +Please visit [this page](https://community.appdynamics.com/t5/Knowledge-Base/How-to-use-Password-Encryption-with-Extensions/ta-p/29397) to get detailed instructions on encryption. The steps in this document will guide you through the whole process. + +## Extensions Workbench +Workbench is an inbuilt feature provided with each extension in order to assist you to fine tune the extension setup before you actually deploy it on the controller. Please review the following document on [How to use the Extensions WorkBench](https://community.appdynamics.com/t5/Knowledge-Base/How-to-use-the-Extensions-WorkBench/ta-p/30130) + +## Troubleshooting +Please look at the [troubleshooting document](https://community.appdynamics.com/t5/Knowledge-Base/How-do-I-troubleshoot-missing-custom-metrics-or-extensions/ta-p/28695) and make sure that everything is followed correctly. + +## Support Tickets +If after going through the [Troubleshooting Document](https://community.appdynamics.com/t5/Knowledge-Base/How-to-troubleshoot-missing-custom-metrics-or-extensions-metrics/ta-p/28695) you have not been able to get your extension working, please file a ticket and add the following information. + +Please provide the following in order for us to assist you better. +1. Stop the running machine agent. +2. Delete all existing logs under /logs. +3. Please enable debug logging by editing the file /conf/logging/log4j.xml. Change the level value of the following elements to debug. + + +4. Start the machine agent and please let it run for 10 mins. Then zip and upload all the logs in the directory /logs/*. +5. Attach the zipped /conf/* directory here. +6. Attach the zipped /monitors/ExtensionFolderYouAreHavingIssuesWith directory here. + For any support related questions, you can also contact [help@appdynamics.com](mailto:help@appdynamics.com). + ## Contributing Always feel free to fork and contribute any changes directly via [GitHub](https://github.com/Appdynamics/aws-customnamespace-monitoring-extension). @@ -133,6 +167,10 @@ Always feel free to fork and contribute any changes directly via [GitHub](https: Find out more in the [AppSphere](https://www.appdynamics.com/community/exchange/extension/aws-customnamespace-monitoring-extension) community. -## Support +## Version + | Name | Version | + |--------------------------|------------| + |Extension Version |2.0 | + |Last Update |25th March, 2019 | -For any questions or feature request, please contact [AppDynamics Center of Excellence](mailto:ace-request@appdynamics.com). +List of changes to this extension can be found [here](https://github.com/Appdynamics/aws-customnamespace-monitoring-extension/blob/master/CHANGELOG.md) diff --git a/pom.xml b/pom.xml index f87fe9d..eeb9113 100644 --- a/pom.xml +++ b/pom.xml @@ -10,7 +10,7 @@ 4.0.0 com.appdynamics.extensions aws-customnamespace-monitoring-extension - 1.0.2 + 2.0.0 AWS Custom Namespace Monitoring Extension @@ -30,7 +30,7 @@ com.appdynamics.extensions aws-cloudwatch-exts-commons - 1.0.6 + 2.0.1.1 junit @@ -60,8 +60,8 @@ maven-compiler-plugin 2.3.2 - 1.5 - 1.5 + 1.8 + 1.8 @@ -103,6 +103,7 @@ AWS Custom Namespace Monitor v${project.version} Build Date ${maven.build.timestamp} + com.appdynamics.extensions.workbench.WorkbenchServerLauncher @@ -123,8 +124,7 @@ - - + @@ -163,6 +163,28 @@ ${project.artifactId}-${project.version} + + com.appdynamics.maven.plugin + yaml-validate-maven-plugin + 1.0 + + + src/main/resources/conf/config.yml + src/test/resources/conf/itest-encrypted-config.yml + src/test/resources/conf/itest-proxy-config.yml + src/test/resources/conf/test-dynamodb-config.yml + src/test/resources/conf/test-sqs-config.yml + + + + + validate + + check + + + + diff --git a/src/main/java/com/appdynamics/extensions/aws/customnamespace/CustomNamespaceMetricsProcessor.java b/src/main/java/com/appdynamics/extensions/aws/customnamespace/CustomNamespaceMetricsProcessor.java index 7b029aa..13f6843 100644 --- a/src/main/java/com/appdynamics/extensions/aws/customnamespace/CustomNamespaceMetricsProcessor.java +++ b/src/main/java/com/appdynamics/extensions/aws/customnamespace/CustomNamespaceMetricsProcessor.java @@ -9,17 +9,22 @@ package com.appdynamics.extensions.aws.customnamespace; import com.amazonaws.services.cloudwatch.AmazonCloudWatch; -import com.amazonaws.services.cloudwatch.model.Metric; -import com.appdynamics.extensions.aws.config.MetricType; +import com.amazonaws.services.cloudwatch.model.DimensionFilter; +import com.appdynamics.extensions.aws.config.Dimension; +import com.appdynamics.extensions.aws.config.IncludeMetric; +import com.appdynamics.extensions.aws.dto.AWSMetric; import com.appdynamics.extensions.aws.metric.NamespaceMetricStatistics; import com.appdynamics.extensions.aws.metric.StatisticType; import com.appdynamics.extensions.aws.metric.processors.MetricsProcessor; import com.appdynamics.extensions.aws.metric.processors.MetricsProcessorHelper; +import com.appdynamics.extensions.aws.predicate.MultiDimensionPredicate; +import com.appdynamics.extensions.metrics.Metric; +import com.google.common.collect.Lists; +import java.util.HashMap; import java.util.List; import java.util.Map; -import java.util.Set; -import java.util.regex.Pattern; +import java.util.concurrent.atomic.LongAdder; /** * @author Florencio Sarmiento @@ -28,31 +33,47 @@ public class CustomNamespaceMetricsProcessor implements MetricsProcessor { private String namespace; - private List metricTypes; + private List includeMetrics; + private List dimensions; - private Pattern excludeMetricsPattern; - - public CustomNamespaceMetricsProcessor(List metricTypes, - Set excludeMetrics, + public CustomNamespaceMetricsProcessor(List includeMetrics, List dimensions, String namespace) { - this.metricTypes = metricTypes; - this.excludeMetricsPattern = MetricsProcessorHelper.createPattern(excludeMetrics); + this.includeMetrics = includeMetrics; + this.dimensions = dimensions; this.namespace = namespace; } - public List getMetrics(AmazonCloudWatch awsCloudWatch, String accountName) { - return MetricsProcessorHelper.getFilteredMetrics(awsCloudWatch, - namespace, - excludeMetricsPattern); + @Override + public List getMetrics(AmazonCloudWatch amazonCloudWatch, String s, LongAdder awsRequestsCounter) { + List dimensionFilters = getDimensionFilters(); + MultiDimensionPredicate predicate = new MultiDimensionPredicate(dimensions); + return MetricsProcessorHelper.getFilteredMetrics(amazonCloudWatch, awsRequestsCounter, + namespace, includeMetrics, dimensionFilters, predicate); + } + + private List getDimensionFilters() { + List dimensionFilters = Lists.newArrayList(); + for (Dimension dimension : dimensions) { + DimensionFilter dbDimensionFilter = new DimensionFilter(); + dbDimensionFilter.withName(dimension.getName()); + dimensionFilters.add(dbDimensionFilter); + } + return dimensionFilters; } - public StatisticType getStatisticType(Metric metric) { - return MetricsProcessorHelper.getStatisticType(metric, metricTypes); + @Override + public StatisticType getStatisticType(AWSMetric awsMetric) { + return MetricsProcessorHelper.getStatisticType(awsMetric.getIncludeMetric(), includeMetrics); } - public Map createMetricStatsMapForUpload(NamespaceMetricStatistics namespaceMetricStats) { + public List createMetricStatsMapForUpload(NamespaceMetricStatistics namespaceMetricStats) { + Map dimensionToMetricPathNameDictionary = new HashMap(); + for (Dimension dimension : dimensions) { + dimensionToMetricPathNameDictionary.put(dimension.getName(), dimension.getDisplayName()); + } + return MetricsProcessorHelper.createMetricStatsMapForUpload(namespaceMetricStats, - null, true); + dimensionToMetricPathNameDictionary, true); } public String getNamespace() { diff --git a/src/main/java/com/appdynamics/extensions/aws/customnamespace/CustomNamespaceMonitor.java b/src/main/java/com/appdynamics/extensions/aws/customnamespace/CustomNamespaceMonitor.java index cbdb1f4..ff744d5 100644 --- a/src/main/java/com/appdynamics/extensions/aws/customnamespace/CustomNamespaceMonitor.java +++ b/src/main/java/com/appdynamics/extensions/aws/customnamespace/CustomNamespaceMonitor.java @@ -8,85 +8,62 @@ package com.appdynamics.extensions.aws.customnamespace; -import static com.appdynamics.extensions.aws.Constants.METRIC_PATH_SEPARATOR; - -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; - -import org.apache.log4j.Logger; - -import com.appdynamics.extensions.aws.MultipleNamespaceCloudwatchMonitor; +import com.appdynamics.extensions.aws.SingleNamespaceCloudwatchMonitor; import com.appdynamics.extensions.aws.collectors.NamespaceMetricStatisticsCollector; -import com.appdynamics.extensions.aws.config.ConcurrencyConfig; -import com.appdynamics.extensions.aws.customnamespace.conf.CustomNamespaceAccount; import com.appdynamics.extensions.aws.customnamespace.conf.CustomNamespaceConfiguration; import com.appdynamics.extensions.aws.metric.processors.MetricsProcessor; +import org.apache.log4j.Logger; + +import static com.appdynamics.extensions.aws.Constants.METRIC_PATH_SEPARATOR; /** * @author Florencio Sarmiento - * */ -public class CustomNamespaceMonitor extends MultipleNamespaceCloudwatchMonitor { - - private static final Logger LOGGER = Logger.getLogger("com.singularity.extensions.aws.CustomNamespaceMonitor"); - - private static final String DEFAULT_METRIC_PREFIX = String.format("%s%s%s%s", - "Custom Metrics", METRIC_PATH_SEPARATOR, "Amazon Custom Namespace", METRIC_PATH_SEPARATOR); +public class CustomNamespaceMonitor extends SingleNamespaceCloudwatchMonitor { + + private static final Logger LOGGER = Logger.getLogger(CustomNamespaceMonitor.class); + + private static final String DEFAULT_METRIC_PREFIX = String.format("%s%s%s%s", + "Custom Metrics", METRIC_PATH_SEPARATOR, "Amazon Custom Namespace", METRIC_PATH_SEPARATOR); - public CustomNamespaceMonitor() { - super(CustomNamespaceConfiguration.class); - LOGGER.info(String.format("Using AWS Custom Namespace Monitor Version [%s]", - this.getClass().getPackage().getImplementationTitle())); - } + public CustomNamespaceMonitor() { + super(CustomNamespaceConfiguration.class); + } - @SuppressWarnings({ "unchecked", "rawtypes" }) - @Override - protected List getNamespaceMetricStatisticsCollectorList( - CustomNamespaceConfiguration config) { - - List collectors = new ArrayList(); - - for (CustomNamespaceAccount account : config.getAccounts()) { - for (String namespace : account.getNamespaces()) { - MetricsProcessor metricsProcessor = - new CustomNamespaceMetricsProcessor(config.getMetricsConfig().getMetricTypes(), - config.getMetricsConfig().getExcludeMetrics(), - namespace); + @Override + protected String getDefaultMetricPrefix() { + return DEFAULT_METRIC_PREFIX; + } - NamespaceMetricStatisticsCollector collector = new NamespaceMetricStatisticsCollector - .Builder((List) Arrays.asList(account), - (ConcurrencyConfig) config.getConcurrencyConfig(), - config.getMetricsConfig(), - metricsProcessor) - .withCredentialsEncryptionConfig(config.getCredentialsDecryptionConfig()) - .withProxyConfig(config.getProxyConfig()) - .build(); - - collectors.add(collector); - } - } - - if (collectors.isEmpty()) { - LOGGER.warn("No namespace is configured for monitoring"); - } - - return collectors; - } + @Override + public String getMonitorName() { + return "AWSCustomNamespaceMonitor"; + } - @Override - protected int getNoOfNamespaceThreads(CustomNamespaceConfiguration config) { - return config.getConcurrencyConfig().getNoOfNamespaceThreads(); - } + @Override + protected int getTaskCount() { + return 3; + } - @Override - protected String getMetricPrefix(CustomNamespaceConfiguration config) { - return DEFAULT_METRIC_PREFIX; - } + @Override + protected Logger getLogger() { + return LOGGER; + } - @Override - protected Logger getLogger() { - return LOGGER; - } + @Override + protected NamespaceMetricStatisticsCollector getNamespaceMetricsCollector(CustomNamespaceConfiguration config) { + MetricsProcessor metricsProcessor = + new CustomNamespaceMetricsProcessor(config.getMetricsConfig().getIncludeMetrics(), + config.getDimensions(), + config.getNamespace()); + return new NamespaceMetricStatisticsCollector + .Builder(config.getAccounts(), + config.getConcurrencyConfig(), + config.getMetricsConfig(), + metricsProcessor, config.getMetricPrefix()) + .withCredentialsDecryptionConfig(config.getCredentialsDecryptionConfig()) + .withProxyConfig(config.getProxyConfig()) + .build(); + } } diff --git a/src/main/java/com/appdynamics/extensions/aws/customnamespace/conf/CustomNamespaceAccount.java b/src/main/java/com/appdynamics/extensions/aws/customnamespace/conf/CustomNamespaceAccount.java deleted file mode 100644 index db4415e..0000000 --- a/src/main/java/com/appdynamics/extensions/aws/customnamespace/conf/CustomNamespaceAccount.java +++ /dev/null @@ -1,31 +0,0 @@ -/* - * Copyright 2018. AppDynamics LLC and its affiliates. - * All Rights Reserved. - * This is unpublished proprietary source code of AppDynamics LLC and its affiliates. - * The copyright notice above does not evidence any actual or intended publication of such source code. - * - */ - -package com.appdynamics.extensions.aws.customnamespace.conf; - -import java.util.Set; - -import com.appdynamics.extensions.aws.config.Account; - -/** - * @author Florencio Sarmiento - * - */ -public class CustomNamespaceAccount extends Account { - - private Set namespaces; - - public Set getNamespaces() { - return namespaces; - } - - public void setNamespaces(Set namespaces) { - this.namespaces = namespaces; - } - -} diff --git a/src/main/java/com/appdynamics/extensions/aws/customnamespace/conf/CustomNamespaceConcurrencyConfig.java b/src/main/java/com/appdynamics/extensions/aws/customnamespace/conf/CustomNamespaceConcurrencyConfig.java deleted file mode 100644 index 3f2e733..0000000 --- a/src/main/java/com/appdynamics/extensions/aws/customnamespace/conf/CustomNamespaceConcurrencyConfig.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright 2018. AppDynamics LLC and its affiliates. - * All Rights Reserved. - * This is unpublished proprietary source code of AppDynamics LLC and its affiliates. - * The copyright notice above does not evidence any actual or intended publication of such source code. - * - */ - -package com.appdynamics.extensions.aws.customnamespace.conf; - -import com.appdynamics.extensions.aws.config.ConcurrencyConfig; - -/** - * @author Florencio Sarmiento - * - */ -public class CustomNamespaceConcurrencyConfig extends ConcurrencyConfig { - - private int noOfNamespaceThreads; - - public int getNoOfNamespaceThreads() { - return noOfNamespaceThreads; - } - - public void setNoOfNamespaceThreads(int noOfNamespaceThreads) { - this.noOfNamespaceThreads = noOfNamespaceThreads; - } - -} diff --git a/src/main/java/com/appdynamics/extensions/aws/customnamespace/conf/CustomNamespaceConfiguration.java b/src/main/java/com/appdynamics/extensions/aws/customnamespace/conf/CustomNamespaceConfiguration.java index e0387e1..9a7ee13 100644 --- a/src/main/java/com/appdynamics/extensions/aws/customnamespace/conf/CustomNamespaceConfiguration.java +++ b/src/main/java/com/appdynamics/extensions/aws/customnamespace/conf/CustomNamespaceConfiguration.java @@ -8,78 +8,21 @@ package com.appdynamics.extensions.aws.customnamespace.conf; -import java.util.List; - -import com.appdynamics.extensions.aws.config.CredentialsDecryptionConfig; -import com.appdynamics.extensions.aws.config.MetricsConfig; -import com.appdynamics.extensions.aws.config.ProxyConfig; +import com.appdynamics.extensions.aws.config.Configuration; /** * @author Florencio Sarmiento * */ -public class CustomNamespaceConfiguration { - - private List accounts; - - private CredentialsDecryptionConfig credentialsDecryptionConfig; - - private ProxyConfig proxyConfig; +public class CustomNamespaceConfiguration extends Configuration { - private MetricsConfig metricsConfig; - - private CustomNamespaceConcurrencyConfig concurrencyConfig; - - private String metricPrefix; - - public List getAccounts() { - return accounts; - } - - public void setAccounts(List accounts) { - this.accounts = accounts; - } - - public CredentialsDecryptionConfig getCredentialsDecryptionConfig() { - return credentialsDecryptionConfig; - } - - public void setCredentialsDecryptionConfig( - CredentialsDecryptionConfig credentialsDecryptionConfig) { - this.credentialsDecryptionConfig = credentialsDecryptionConfig; - } + private String namespace; - public ProxyConfig getProxyConfig() { - return proxyConfig; + public String getNamespace() { + return namespace; } - public void setProxyConfig(ProxyConfig proxyConfig) { - this.proxyConfig = proxyConfig; + public void setNamespace(String namespace) { + this.namespace = namespace; } - - public MetricsConfig getMetricsConfig() { - return metricsConfig; - } - - public void setMetricsConfig(MetricsConfig metricsConfig) { - this.metricsConfig = metricsConfig; - } - - public CustomNamespaceConcurrencyConfig getConcurrencyConfig() { - return concurrencyConfig; - } - - public void setConcurrencyConfig( - CustomNamespaceConcurrencyConfig concurrencyConfig) { - this.concurrencyConfig = concurrencyConfig; - } - - public String getMetricPrefix() { - return metricPrefix; - } - - public void setMetricPrefix(String metricPrefix) { - this.metricPrefix = metricPrefix; - } - } diff --git a/src/main/resources/conf/config.yaml b/src/main/resources/conf/config.yaml deleted file mode 100644 index 137ac79..0000000 --- a/src/main/resources/conf/config.yaml +++ /dev/null @@ -1,63 +0,0 @@ -accounts: - - awsAccessKey: - awsSecretKey: - displayAccountName: "AppD Custom" - - # Regions where customnamespace is registered - # Allowed values: ap-southeast-1, ap-southeast-2, ap-northeast-1, eu-central-1, - # us-east-1, eu-west-1, us-west-1, us-west-2, sa-east-1 - # - regions: ["us-west-2"] - - # The custom namespaces registered for this account - namespaces: [ "AWS/ElasticBeanstalk" ] - -credentialsDecryptionConfig: - enableDecryption: "false" - decryptionKey: - -proxyConfig: - host: - port: - username: - password: - -# Global metrics config for all accounts -metricsConfig: - - # By default, all metrics retrieved from cloudwatch are 'Average' values. - # This option allows you to override the metric type. - # - # metricName supports regex - # Allowed statTypes are: ave, max, min, sum, samplecount - # - # Note: Irrespective of the metric type, value will still be reported as - # Observed value to the Controller - metricTypes: - - metricName: "" - statType: "" - - excludeMetrics: [ ] - - metricsTimeRange: - startTimeInMinsBeforeNow: 5 - endTimeInMinsBeforeNow: 0 - - # - # The max number of retry attempts for failed retryable requests - # (ex: 5xx error responses from a service) or throttling errors - # - maxErrorRetrySize: 0 - -concurrencyConfig: - noOfNamespaceThreads: 3 - noOfAccountThreads: 3 - noOfRegionThreadsPerAccount: 3 - noOfMetricThreadsPerRegion: 3 - -#prefix used to show up metrics in AppDynamics -#metricPrefix: "Custom Metrics|Amazon Custom Namespace|" - -#This will create it in specific Tier/Component. Make sure to replace with the appropriate one from your environment. -#To find the in your environment, please follow the screenshot https://docs.appdynamics.com/display/PRO42/Build+a+Monitoring+Extension+Using+Java -metricPrefix: "Server|Component:|Custom Metrics|Amazon Custom Namespace|" \ No newline at end of file diff --git a/src/main/resources/conf/config.yml b/src/main/resources/conf/config.yml new file mode 100644 index 0000000..c0c2825 --- /dev/null +++ b/src/main/resources/conf/config.yml @@ -0,0 +1,115 @@ +#This will create this metric in all the tiers, under this path. Please make sure to have a trailing | +#metricPrefix: "Custom Metrics|Amazon Custom Namespace|" + +#This will create it in specific Tier/Component. Make sure to replace with the appropriate one from your environment. +#To find the in your environment, please follow the screenshot https://docs.appdynamics.com/display/PRO42/Build+a+Monitoring+Extension+Using+Java +metricPrefix: "Server|Component:|Custom Metrics|Amazon Custom Namespace|" + +accounts: + - awsAccessKey: "" + awsSecretKey: "" + displayAccountName: "AppD Custom" + + # Regions where customnamespace is registered + # Allowed values: Please refer to regionEndPoints configuration in this file. + regions: ["us-west-2", "us-east-1"] + + +# Please specify the namespace that you would like to monitor. AWS/ElasticBeanstalk is just an example. Please change this to your need. +namespace: "AWS/DynamoDB" + +# Password Encryption +credentialsDecryptionConfig: + enableDecryption: "false" + encryptionKey: "" + +# configure proxy if any +proxyConfig: + host: + port: + username: + password: + + +dimensions: + - name: "TableName" + displayName: "Table Name" + values: ["ABC"] + - name: "AvailabilityZone" + displayName: "Availability Zone" + values: [] + +# Global metrics config for all accounts +metricsConfig: + + # By default, all metrics retrieved from cloudwatch are 'Average' values. + # This option allows you to override the metric type. + # + # Allowed statTypes are: ave, max, min, sum, samplecount + # + # Note: Irrespective of the metric type, value will still be reported as + # Observed value to the Controller + includeMetrics: + - name: "ProvisionedReadCapacityUnits" + alias: "Provisioned Read Capacity Units" + statType: "sum" + aggregationType: "AVERAGE" + timeRollUpType: "AVERAGE" + clusterRollUpType: "INDIVIDUAL" + delta: false + multiplier: 1 + - name: "ConsumedReadCapacityUnits" + alias: "Consumed Read Capacity Units" + statType: "ave" + aggregationType: "AVERAGE" + timeRollUpType: "AVERAGE" + clusterRollUpType: "INDIVIDUAL" + delta: false + multiplier: 1 + + metricsTimeRange: + startTimeInMinsBeforeNow: 5 + endTimeInMinsBeforeNow: 0 + + # Rate limit ( per second ) for GetMetricStatistics, default value is 400. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html + getMetricStatisticsRateLimit: 400 + + # + # The max number of retry attempts for failed retryable requests + # (ex: 5xx error responses from a service) or throttling errors + # + maxErrorRetrySize: 0 + +concurrencyConfig: + noOfAccountThreads: 3 + noOfRegionThreadsPerAccount: 3 + noOfMetricThreadsPerRegion: 3 + #Thread timeout in seconds + threadTimeOut: 30 + + +#Allowed values are Basic and Detailed. Refer https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-new.html for more information +# Basic will fire CloudWatch API calls every 5 minutes +# Detailed will fire CloudWatch API calls every 1 minutes +cloudWatchMonitoring: "Basic" + +regionEndPoints: + us-east-1 : monitoring.us-east-1.amazonaws.com + us-east-2 : monitoring.us-east-2.amazonaws.com + us-west-1 : monitoring.us-west-1.amazonaws.com + us-west-2 : monitoring.us-west-2.amazonaws.com + ap-northeast-1 : monitoring.ap-northeast-1.amazonaws.com + ap-northeast-2: monitoring.ap-northeast-2.amazonaws.com + ap-northeast-3 : monitoring.ap-northeast-3.amazonaws.com + ap-south-1 : monitoring.ap-south-1.amazonaws.com + ap-southeast-1 : monitoring.ap-southeast-1.amazonaws.com + ap-southeast-2 : monitoring.ap-southeast-2.amazonaws.com + ca-central-1 : monitoring.ca-central-1.amazonaws.com + cn-north-1 : monitoring.cn-north-1.amazonaws.com.cn + cn-northwest-1 : monitoring.cn-northwest-1.amazonaws.com.cn + eu-central-1 : monitoring.eu-central-1.amazonaws.com + eu-west-1 : monitoring.eu-west-1.amazonaws.com + eu-west-2 : monitoring.eu-west-2.amazonaws.com + eu-west-3 : monitoring.eu-west-3.amazonaws.com + sa-east-1 : monitoring.sa-east-1.amazonaws.com + us-gov-west-1 : monitoring.us-gov-west-1.amazonaws.com \ No newline at end of file diff --git a/src/main/resources/conf/monitor.xml b/src/main/resources/conf/monitor.xml index c655e09..a418718 100755 --- a/src/main/resources/conf/monitor.xml +++ b/src/main/resources/conf/monitor.xml @@ -19,16 +19,12 @@ AWS Custom Namespace Monitor Run task java 60 - - - + - aws-customnamespace-monitoring-extension.jar com.appdynamics.extensions.aws.customnamespace.CustomNamespaceMonitor - diff --git a/src/main/resources/conf/region-endpoints.yaml b/src/main/resources/conf/region-endpoints.yaml deleted file mode 100644 index e0cd943..0000000 --- a/src/main/resources/conf/region-endpoints.yaml +++ /dev/null @@ -1,11 +0,0 @@ -{ - "ap-southeast-1" : "monitoring.ap-southeast-1.amazonaws.com", - "ap-southeast-2" : "monitoring.ap-southeast-2.amazonaws.com", - "ap-northeast-1" : "monitoring.ap-northeast-1.amazonaws.com", - "eu-central-1" : "monitoring.eu-central-1.amazonaws.com", - "eu-west-1" : "monitoring.eu-west-1.amazonaws.com", - "us-east-1" : "monitoring.us-east-1.amazonaws.com", - "us-west-1" : "monitoring.us-west-1.amazonaws.com", - "us-west-2" : "monitoring.us-west-2.amazonaws.com", - "sa-east-1" : "monitoring.sa-east-1.amazonaws.com" -} diff --git a/src/test/java/com/appdynamics/extensions/aws/customnamespace/CustomNamespaceMonitorITest.java b/src/test/java/com/appdynamics/extensions/aws/customnamespace/CustomNamespaceMonitorITest.java deleted file mode 100644 index 812c0c3..0000000 --- a/src/test/java/com/appdynamics/extensions/aws/customnamespace/CustomNamespaceMonitorITest.java +++ /dev/null @@ -1,41 +0,0 @@ -/* - * Copyright 2018. AppDynamics LLC and its affiliates. - * All Rights Reserved. - * This is unpublished proprietary source code of AppDynamics LLC and its affiliates. - * The copyright notice above does not evidence any actual or intended publication of such source code. - * - */ - -package com.appdynamics.extensions.aws.customnamespace; - -import static org.junit.Assert.assertTrue; - -import java.util.Map; - -import org.junit.Test; - -import com.google.common.collect.Maps; -import com.singularity.ee.agent.systemagent.api.TaskOutput; - -public class CustomNamespaceMonitorITest { - - private CustomNamespaceMonitor classUnderTest = new CustomNamespaceMonitor(); - - @Test - public void testMetricsCollectionCredentialsEncrypted() throws Exception { - Map args = Maps.newHashMap(); - args.put("config-file","src/test/resources/conf/itest-encrypted-config.yaml"); - - TaskOutput result = classUnderTest.execute(args, null); - assertTrue(result.getStatusMessage().contains("successfully completed")); - } - - @Test - public void testMetricsCoyllectionWithProxy() throws Exception { - Map args = Maps.newHashMap(); - args.put("config-file","src/test/resources/conf/itest-proxy-config.yaml"); - - TaskOutput result = classUnderTest.execute(args, null); - assertTrue(result.getStatusMessage().contains("successfully completed")); - } -} diff --git a/src/test/resources/conf/itest-encrypted-config.yaml b/src/test/resources/conf/itest-encrypted-config.yaml deleted file mode 100644 index 7aeed2c..0000000 --- a/src/test/resources/conf/itest-encrypted-config.yaml +++ /dev/null @@ -1,37 +0,0 @@ -accounts: - - awsAccessKey: "" - awsSecretKey: "" - displayAccountName: "TestAccount" - regions: ["us-east-1", "us-west-1", "us-west-2"] - namespaces: ["AWS/EC2", "AWS/AutoScaling"] - -credentialsDecryptionConfig: - enableDecryption: "true" - decryptionKey: "test" - -proxyConfig: - host: - port: - username: - password: - -metricsConfig: - metricTypes: - - metricName: "" - statType: "" - - excludeMetrics: [ ] - - metricsTimeRange: - startTimeInMinsBeforeNow: 5 - endTimeInMinsBeforeNow: 0 - - maxErrorRetrySize: 0 - -concurrencyConfig: - noOfNamespaceThreads: 3 - noOfAccountThreads: 3 - noOfRegionThreadsPerAccount: 3 - noOfMetricThreadsPerRegion: 3 - -metricPrefix: "Custom Metrics|Amazon Custom Namespace|" \ No newline at end of file diff --git a/src/test/resources/conf/itest-encrypted-config.yml b/src/test/resources/conf/itest-encrypted-config.yml new file mode 100644 index 0000000..37bbd6e --- /dev/null +++ b/src/test/resources/conf/itest-encrypted-config.yml @@ -0,0 +1,112 @@ +#This will create this metric in all the tiers, under this path. Please make sure to have a trailing | +#metricPrefix: "Custom Metrics|Amazon Custom Namespace|" + +#This will create it in specific Tier/Component. Make sure to replace with the appropriate one from your environment. +#To find the in your environment, please follow the screenshot https://docs.appdynamics.com/display/PRO42/Build+a+Monitoring+Extension+Using+Java +metricPrefix: "Server|Component:|Custom Metrics|Amazon Custom Namespace|" + +accounts: + - awsAccessKey: "" + awsSecretKey: "" + displayAccountName: "AppD_AWS_Account" + + # Regions where customnamespace is registered + # Allowed values: Please refer to regionEndPoints configuration in this file. + regions: ["us-east-2"] + + +# Please specify the namespace that you would like to monitor. AWS/ElasticBeanstalk is just an example. Please change this to your need. +namespace: "AWS/SQS" + +# Password Encryption +credentialsDecryptionConfig: + enableDecryption: "true" + encryptionKey: "encryptionKey" + +# configure proxy if any +proxyConfig: + host: + port: + username: + password: + + +dimensions: + - name: "QueueName" + displayName: "Queue Name" + values: ["MetricLoadEventQueue"] + +# Global metrics config for all accounts +metricsConfig: + + # By default, all metrics retrieved from cloudwatch are 'Average' values. + # This option allows you to override the metric type. + # + # Allowed statTypes are: ave, max, min, sum, samplecount + # + # Note: Irrespective of the metric type, value will still be reported as + # Observed value to the Controller + includeMetrics: + - name: "ApproximateNumberOfMessagesDelayed" + alias: "Approximate Number Of Messages Delayed" + statType: "sum" + aggregationType: "AVERAGE" + timeRollUpType: "AVERAGE" + clusterRollUpType: "INDIVIDUAL" + delta: false + multiplier: 1 + - name: "NumberOfMessagesDelayed" + alias: "Number Of Messages Delayed" + statType: "ave" + aggregationType: "AVERAGE" + timeRollUpType: "AVERAGE" + clusterRollUpType: "INDIVIDUAL" + delta: false + multiplier: 1 + + metricsTimeRange: + startTimeInMinsBeforeNow: 5 + endTimeInMinsBeforeNow: 0 + + # Rate limit ( per second ) for GetMetricStatistics, default value is 400. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html + getMetricStatisticsRateLimit: 400 + + # + # The max number of retry attempts for failed retryable requests + # (ex: 5xx error responses from a service) or throttling errors + # + maxErrorRetrySize: 0 + +concurrencyConfig: + noOfAccountThreads: 3 + noOfRegionThreadsPerAccount: 3 + noOfMetricThreadsPerRegion: 3 + #Thread timeout in seconds + threadTimeOut: 30 + + +#Allowed values are Basic and Detailed. Refer https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-new.html for more information +# Basic will fire CloudWatch API calls every 5 minutes +# Detailed will fire CloudWatch API calls every 1 minutes +cloudWatchMonitoring: "Basic" + +regionEndPoints: + us-east-1 : monitoring.us-east-1.amazonaws.com + us-east-2 : monitoring.us-east-2.amazonaws.com + us-west-1 : monitoring.us-west-1.amazonaws.com + us-west-2 : monitoring.us-west-2.amazonaws.com + ap-northeast-1 : monitoring.ap-northeast-1.amazonaws.com + ap-northeast-2: monitoring.ap-northeast-2.amazonaws.com + ap-northeast-3 : monitoring.ap-northeast-3.amazonaws.com + ap-south-1 : monitoring.ap-south-1.amazonaws.com + ap-southeast-1 : monitoring.ap-southeast-1.amazonaws.com + ap-southeast-2 : monitoring.ap-southeast-2.amazonaws.com + ca-central-1 : monitoring.ca-central-1.amazonaws.com + cn-north-1 : monitoring.cn-north-1.amazonaws.com.cn + cn-northwest-1 : monitoring.cn-northwest-1.amazonaws.com.cn + eu-central-1 : monitoring.eu-central-1.amazonaws.com + eu-west-1 : monitoring.eu-west-1.amazonaws.com + eu-west-2 : monitoring.eu-west-2.amazonaws.com + eu-west-3 : monitoring.eu-west-3.amazonaws.com + sa-east-1 : monitoring.sa-east-1.amazonaws.com + us-gov-west-1 : monitoring.us-gov-west-1.amazonaws.com \ No newline at end of file diff --git a/src/test/resources/conf/itest-proxy-config.yaml b/src/test/resources/conf/itest-proxy-config.yml similarity index 100% rename from src/test/resources/conf/itest-proxy-config.yaml rename to src/test/resources/conf/itest-proxy-config.yml diff --git a/src/test/resources/conf/test-dynamodb-config.yml b/src/test/resources/conf/test-dynamodb-config.yml new file mode 100644 index 0000000..95b90cd --- /dev/null +++ b/src/test/resources/conf/test-dynamodb-config.yml @@ -0,0 +1,112 @@ +#This will create this metric in all the tiers, under this path. Please make sure to have a trailing | +#metricPrefix: "Custom Metrics|Amazon Custom Namespace|" + +#This will create it in specific Tier/Component. Make sure to replace with the appropriate one from your environment. +#To find the in your environment, please follow the screenshot https://docs.appdynamics.com/display/PRO42/Build+a+Monitoring+Extension+Using+Java +metricPrefix: "Server|Component:|Custom Metrics|Amazon Custom Namespace|" + +accounts: + - awsAccessKey: "" + awsSecretKey: "" + displayAccountName: "AppD Custom" + + # Regions where customnamespace is registered + # Allowed values: Please refer to regionEndPoints configuration in this file. + regions: ["us-west-2", "us-east-1"] + + +# Please specify the namespace that you would like to monitor. AWS/ElasticBeanstalk is just an example. Please change this to your need. +namespace: "AWS/DynamoDB" + +# Password Encryption +credentialsDecryptionConfig: + enableDecryption: "false" + encryptionKey: "" + +# configure proxy if any +proxyConfig: + host: "" + port: "" + username: "" + password: "" + + +dimensions: + - name: "TableName" + displayName: "Table Name" + values: ["ABC"] + +# Global metrics config for all accounts +metricsConfig: + + # By default, all metrics retrieved from cloudwatch are 'Average' values. + # This option allows you to override the metric type. + # + # Allowed statTypes are: ave, max, min, sum, samplecount + # + # Note: Irrespective of the metric type, value will still be reported as + # Observed value to the Controller + includeMetrics: + - name: "ProvisionedReadCapacityUnits" + alias: "Provisioned Read Capacity Units" + statType: "ave" + aggregationType: "AVERAGE" + timeRollUpType: "AVERAGE" + clusterRollUpType: "INDIVIDUAL" + delta: false + multiplier: 1 + - name: "ConsumedReadCapacityUnits" + alias: "Consumed Read Capacity Units" + statType: "ave" + aggregationType: "AVERAGE" + timeRollUpType: "AVERAGE" + clusterRollUpType: "INDIVIDUAL" + delta: false + multiplier: 1 + + metricsTimeRange: + startTimeInMinsBeforeNow: 5 + endTimeInMinsBeforeNow: 0 + + # Rate limit ( per second ) for GetMetricStatistics, default value is 400. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html + getMetricStatisticsRateLimit: 400 + + # + # The max number of retry attempts for failed retryable requests + # (ex: 5xx error responses from a service) or throttling errors + # + maxErrorRetrySize: 0 + +concurrencyConfig: + noOfAccountThreads: 3 + noOfRegionThreadsPerAccount: 3 + noOfMetricThreadsPerRegion: 3 + #Thread timeout in seconds + threadTimeOut: 30 + + +#Allowed values are Basic and Detailed. Refer https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-new.html for more information +# Basic will fire CloudWatch API calls every 5 minutes +# Detailed will fire CloudWatch API calls every 1 minutes +cloudWatchMonitoring: "Basic" + +regionEndPoints: + us-east-1 : monitoring.us-east-1.amazonaws.com + us-east-2 : monitoring.us-east-2.amazonaws.com + us-west-1 : monitoring.us-west-1.amazonaws.com + us-west-2 : monitoring.us-west-2.amazonaws.com + ap-northeast-1 : monitoring.ap-northeast-1.amazonaws.com + ap-northeast-2: monitoring.ap-northeast-2.amazonaws.com + ap-northeast-3 : monitoring.ap-northeast-3.amazonaws.com + ap-south-1 : monitoring.ap-south-1.amazonaws.com + ap-southeast-1 : monitoring.ap-southeast-1.amazonaws.com + ap-southeast-2 : monitoring.ap-southeast-2.amazonaws.com + ca-central-1 : monitoring.ca-central-1.amazonaws.com + cn-north-1 : monitoring.cn-north-1.amazonaws.com.cn + cn-northwest-1 : monitoring.cn-northwest-1.amazonaws.com.cn + eu-central-1 : monitoring.eu-central-1.amazonaws.com + eu-west-1 : monitoring.eu-west-1.amazonaws.com + eu-west-2 : monitoring.eu-west-2.amazonaws.com + eu-west-3 : monitoring.eu-west-3.amazonaws.com + sa-east-1 : monitoring.sa-east-1.amazonaws.com + us-gov-west-1 : monitoring.us-gov-west-1.amazonaws.com \ No newline at end of file diff --git a/src/test/resources/conf/test-sqs-config.yml b/src/test/resources/conf/test-sqs-config.yml new file mode 100644 index 0000000..7dff891 --- /dev/null +++ b/src/test/resources/conf/test-sqs-config.yml @@ -0,0 +1,112 @@ +#This will create this metric in all the tiers, under this path. Please make sure to have a trailing | +#metricPrefix: "Custom Metrics|Amazon Custom Namespace|" + +#This will create it in specific Tier/Component. Make sure to replace with the appropriate one from your environment. +#To find the in your environment, please follow the screenshot https://docs.appdynamics.com/display/PRO42/Build+a+Monitoring+Extension+Using+Java +metricPrefix: "Server|Component:|Custom Metrics|Amazon Custom Namespace|" + +accounts: + - awsAccessKey: "" + awsSecretKey: "" + displayAccountName: "AppD_AWS_Account" + + # Regions where customnamespace is registered + # Allowed values: Please refer to regionEndPoints configuration in this file. + regions: ["us-east-2"] + + +# Please specify the namespace that you would like to monitor. AWS/ElasticBeanstalk is just an example. Please change this to your need. +namespace: "AWS/SQS" + +# Password Encryption +credentialsDecryptionConfig: + enableDecryption: "false" + encryptionKey: "" + +# configure proxy if any +proxyConfig: + host: + port: + username: + password: + + +dimensions: + - name: "QueueName" + displayName: "Queue Name" + values: ["MetricLoadEventQueue"] + +# Global metrics config for all accounts +metricsConfig: + + # By default, all metrics retrieved from cloudwatch are 'Average' values. + # This option allows you to override the metric type. + # + # Allowed statTypes are: ave, max, min, sum, samplecount + # + # Note: Irrespective of the metric type, value will still be reported as + # Observed value to the Controller + includeMetrics: + - name: "ApproximateNumberOfMessagesDelayed" + alias: "Approximate Number Of Messages Delayed" + statType: "sum" + aggregationType: "AVERAGE" + timeRollUpType: "AVERAGE" + clusterRollUpType: "INDIVIDUAL" + delta: false + multiplier: 1 + - name: "NumberOfMessagesDeleted" + alias: "Number Of Messages Deleted" + statType: "ave" + aggregationType: "AVERAGE" + timeRollUpType: "AVERAGE" + clusterRollUpType: "INDIVIDUAL" + delta: false + multiplier: 1 + + metricsTimeRange: + startTimeInMinsBeforeNow: 5 + endTimeInMinsBeforeNow: 0 + + # Rate limit ( per second ) for GetMetricStatistics, default value is 400. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html + getMetricStatisticsRateLimit: 400 + + # + # The max number of retry attempts for failed retryable requests + # (ex: 5xx error responses from a service) or throttling errors + # + maxErrorRetrySize: 0 + +concurrencyConfig: + noOfAccountThreads: 3 + noOfRegionThreadsPerAccount: 3 + noOfMetricThreadsPerRegion: 3 + #Thread timeout in seconds + threadTimeOut: 30 + + +#Allowed values are Basic and Detailed. Refer https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-new.html for more information +# Basic will fire CloudWatch API calls every 5 minutes +# Detailed will fire CloudWatch API calls every 1 minutes +cloudWatchMonitoring: "Basic" + +regionEndPoints: + us-east-1 : monitoring.us-east-1.amazonaws.com + us-east-2 : monitoring.us-east-2.amazonaws.com + us-west-1 : monitoring.us-west-1.amazonaws.com + us-west-2 : monitoring.us-west-2.amazonaws.com + ap-northeast-1 : monitoring.ap-northeast-1.amazonaws.com + ap-northeast-2: monitoring.ap-northeast-2.amazonaws.com + ap-northeast-3 : monitoring.ap-northeast-3.amazonaws.com + ap-south-1 : monitoring.ap-south-1.amazonaws.com + ap-southeast-1 : monitoring.ap-southeast-1.amazonaws.com + ap-southeast-2 : monitoring.ap-southeast-2.amazonaws.com + ca-central-1 : monitoring.ca-central-1.amazonaws.com + cn-north-1 : monitoring.cn-north-1.amazonaws.com.cn + cn-northwest-1 : monitoring.cn-northwest-1.amazonaws.com.cn + eu-central-1 : monitoring.eu-central-1.amazonaws.com + eu-west-1 : monitoring.eu-west-1.amazonaws.com + eu-west-2 : monitoring.eu-west-2.amazonaws.com + eu-west-3 : monitoring.eu-west-3.amazonaws.com + sa-east-1 : monitoring.sa-east-1.amazonaws.com + us-gov-west-1 : monitoring.us-gov-west-1.amazonaws.com \ No newline at end of file