Skip to content

Commit 8fdd48b

Browse files
author
Srikanth
committed
First Version of Proxy Coverage Tool
1 parent cd2c8b1 commit 8fdd48b

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

52 files changed

+2759
-3
lines changed

‎.gitignore

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
target/
2+
pom.xml.tag
3+
pom.xml.releaseBackup
4+
pom.xml.next
5+
release.properties
6+
.idea
7+
*.iml
8+
out
9+
profile_test*
10+
cov_report

‎LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
The MIT License (MIT)
2+
3+
Copyright (c) 2014 Srikanth Seshadri
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎README.md

Lines changed: 91 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,92 @@
1-
apigee-proxy-coverage
2-
=====================
1+
Apigee Proxy Coverage
2+
==================
3+
4+
This is a tool to determine test case coverage for [Apigee](www.apigee.com) proxies.
5+
6+
Given a fully functional [API proxy](http://apigee.com/docs/), the tool produces an instrumented proxy file as output. The instrumented proxy file should be deployed to Apigee Edge against which the test cases should be run. Upon completion, the execution data is retrieved by the tool from Apigee Edge to produce the coverage report.
7+
8+
The tool determines coverage of single API proxy under test.
9+
10+
Coverage in case of Apigee proxies is the percentage of policies executed with respect to the total number of policies. Unlike, programming languages where the practical acceptable coverage is around 80%, in case of Apigee proxies the coverage must be 100% - there are no policies that need not be tested or have a remote possibility of execution.
11+
12+
Usage
13+
--------
14+
15+
**Pre-requisties**
16+
17+
The [distribution](https://github.com/sriki77/apigee-proxy-coverage/tree/master/dist) provides shell scripts, a executable `Java 8` jar file and `python` scripts - which makes` bash, python, Java SE 8` as pre-requisites.
18+
19+
---
20+
21+
**Steps:** *(Short Version)*
22+
23+
1. Configure environment settings in `config_vars`
24+
2. Run `instr_prepare.sh`
25+
3. Run your test cases against the instrumented proxy deployed by previous script
26+
4. Run `instr_report.sh` to obtain the coverage report.
27+
28+
---
29+
30+
**Steps:** *(A bit more detail)*
31+
32+
1. Enter your environment details in the file `config_vars`. The details of Apigee `org`, `env`, `api` proxy for coverage and the `revision` of the proxy to be used for instrumentation. Credentials to access the `org` is essential.
33+
2. Run `instr_prepare.sh`. Following are steps performed by this script.
34+
+ Download specified revision of proxy bundle from the `org`.
35+
+ Instrument the downloaded bundle.
36+
+ Deploy the instrumented bundle into the `org` and activate it.
37+
+ Delete any instrumented data recorded against the proxy in the org.
38+
3. Run the all tests against the deployed instrumented bundle. The instrument bundle has the same base path as the original bundle
39+
4. Run `instr_report.sh` to obtain the coverage results. Following are the steps performed by this script.
40+
+ Download the instrumented data recorded in the [Key Value map](http://apigee.com/docs/api-services/content/persist-data-using-keyvaluemap) in Apigee edge.
41+
+ Generate the coverage report. The report has XML format - a `summary.xml` and one XML corresponding to every proxy and target endpoints are generated.
42+
+ Custom stylesheets can be used convert the XMLs into desired report format.
43+
+ A HTML version of the report is also generated by applying the default stylesheet.
44+
+ Open the `summary.html` in the default browser.
45+
46+
Sample Report Screenshots
47+
--------------------------------------
48+
49+
**Summary**
50+
![summary](https://raw.githubusercontent.com/sriki77/apigee-proxy-coverage/master/sample_summary.png)
51+
52+
**Policy Execution Drilldown**
53+
54+
![policy execution](https://raw.githubusercontent.com/sriki77/apigee-proxy-coverage/master/policy_exec.png)
55+
56+
Coverage Numbers
57+
----------------
58+
59+
Coverage is percentage of policies executed against the total number of policies that exists in the API proxy. With this definition it's possible to have overall coverage at 100%, while the flow level coverage be less than 100%. This is because all the policies would have been executed - but not on all paths.
60+
61+
It's best to achieve 100% flow level coverage in effect attain 100% overall coverage, than vice versa.
62+
63+
Limitations
64+
--------------
65+
All proxy endpoints are expected to have **only one**
66+
67+
+ `PreFlow` tag
68+
+ `PostFlow` tag
69+
+ `FaultRules` tag
70+
71+
Apigee Edge allows multiple of the above tags, though the behaviour in certain cases is un-documented and not supported for instrumentation.
72+
73+
74+
Internals
75+
------------
76+
77+
The tool has to determine the policies that executed in each flow to arrive at the final coverage number.
78+
79+
The tool instruments a given proxy using the following approach.
80+
81+
1. For each policy a new key-value map policy is added to record the execution.
82+
2. The policy is added prior to the actual policy with the same condition as the one that exists on the actual.
83+
3. The rationale is that if the key-value map policy executes so will the actual policy since the same condition is satisfied.
84+
4. Further, its added prior to the actual policy to handle the scenario of fault policies in which case the policy following the fault will not be executed.
85+
2. The generated policies have random unique names to avoid conflict. They log the information into a Key Value Map named instrument at the api proxy level.
86+
3. After test execution, the tool uses the content of the key value map to reconstruct the execution information and calculate coverage.
87+
88+
About
89+
--------
90+
The initial version is implemented by Srikanth Seshadri. [Interesting feedback](https://github.com/sriki77/apigee-proxy-coverage/issues?state=open) has been provided by the Apigee CS architects for enhancement. Please feel free to contribute.
91+
392

4-
apigee-proxy-coverage

‎dist/apc-1.0.jar

1.1 MB
Binary file not shown.

‎dist/config_vars

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#!/bin/bash
2+
3+
#===========================================
4+
# Edit this section only
5+
6+
# Proxy Env Configuration
7+
export org= #Org name
8+
export api= #API proxy name
9+
export env= # Env name
10+
export rev= # API Proxy revision for instrumentation
11+
12+
#Credentials
13+
export username= #Username
14+
export password= #password
15+
16+
#Report directory
17+
export report_dir=coverage_report # report directory
18+
#===========================================

0 commit comments

Comments
 (0)