[DX-1780]Generate test for tyk gateway swagger #6827
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
A JIRA Issue ID is missing from your branch name, PR title and PR description! 🦄 Your branch: generate-test-for-tyk-gateway-open-api-spec-improved Your PR title: add swagger file empty string Your PR description: DescriptionRelated IssueMotivation and ContextHow This Has Been TestedScreenshots (if appropriate)Types of changes
Checklist
If this is your first time contributing to this repository - welcome! Please refer to jira-lint to get started. Without the JIRA Issue ID in your branch name you would lose out on automatic updates to JIRA via SCM; some GitHub status checks might fail. Valid sample branch names:‣ feature/shiny-new-feature--mojo-10' |
yurisasuke
changed the title
|
Swagger Changes
_ __ __
nullable: true
_| |_ _ / _|/ _| between swagger-prev.yml
+ one list entry added:
+ one map entry added:
- one list entry removed: + one list entry added:
± value change
/ _' | | | | |_| |_ and swagger-current.yml
\__,_|\__, |_| |_| returned four differences
components.schemas.GraphQLConfig.properties.execution_mode.enum
components.schemas.GraphQLConfig.properties.version.enum
components.schemas.RateLimitSmoothing
paths./tyk/certs/{certID}.get.tags.0
| (_| | |_| | _| _| |
|
API Changes no api changes detected |
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Explore these optional code suggestions:
|
|
I'm a bot and I 👍 this PR title. 🤖 |
…f github.com:TykTechnologies/tyk into generate-test-for-tyk-gateway-open-api-spec-improved
Co-authored-by: Tit Petric <[email protected]>
titpetric
approved these changes
Feb 7, 2025
…f github.com:TykTechnologies/tyk into generate-test-for-tyk-gateway-open-api-spec-improved
|
titpetric
deleted the
generate-test-for-tyk-gateway-open-api-spec-improved
branch
edsonmichaque
pushed a commit
that referenced
this pull request
Feb 14, 2025
# OpenAPI Specification Contract Testing with Portman To ensure that the schema in our gateway's OpenAPI specification matches our API, we have implemented contract tests using Portman. Portman leverages Postman collections to automatically generate contract tests from our OpenAPI specifications. ### What the Contract Tests Are Testing: 1. **Request Body Validation**: Verifying that the request body defined in our Swagger schema matches the request body expected by our gateway API. 2. **Response Body Validation**: Ensuring that the response returned by our gateway API matches the response body defined in our Swagger schema. 3. **Content-Type Validation**: Confirming that the content types defined in the request and response schemas in our Swagger documentation match those expected and returned by our gateway API. 4. **Header Validation**: Validating that the headers sent to or returned by the gateway API are as described in our Swagger documentation. Full documentation [can be found here](https://github.com/TykTechnologies/tyk/blob/5540baeb205a9466d661b49ecf19792fa6491f99/ci/tests/schema/specs/README.md) <details open> <summary><a href="https://tyktech.atlassian.net/browse/DX-1780" title="DX-1780" target="_blank">DX-1780</a></summary> <br /> <table> <tr> <th>Summary</th> <td>Create contract tests for the tyk gateway</td> </tr> <tr> <th>Type</th> <td> <img alt="Story" src="https://tyktech.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10315?size=medium" /> Story </td> </tr> <tr> <th>Status</th> <td>In Progress</td> </tr> <tr> <th>Points</th> <td>N/A</td> </tr> <tr> <th>Labels</th> <td>-</td> </tr> </table> </details> <!-- do not remove this marker as it will break jira-lint's functionality. added_by_jira_lint --> --- ### **User description** [DX-1780] <!-- Provide a general summary of your changes in the Title above --> ## Description <!-- Describe your changes in detail --> ## Related Issue <!-- This project only accepts pull requests related to open issues. --> <!-- If suggesting a new feature or change, please discuss it in an issue first. --> <!-- If fixing a bug, there should be an issue describing it with steps to reproduce. --> <!-- OSS: Please link to the issue here. Tyk: please create/link the JIRA ticket. --> ## Motivation and Context <!-- Why is this change required? What problem does it solve? --> ## How This Has Been Tested <!-- Please describe in detail how you tested your changes --> <!-- Include details of your testing environment, and the tests --> <!-- you ran to see how your change affects other areas of the code, etc. --> <!-- This information is helpful for reviewers and QA. --> ## Screenshots (if appropriate) ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If there are no documentation updates required, mark the item as checked. --> <!-- Raise up any additional concerns not covered by the checklist. --> - [ ] I ensured that the documentation is up to date - [ ] I explained why this PR updates go.mod in detail with reasoning why it's required - [ ] I would like a code coverage CI quality gate exception and have explained why ___ ### **PR Type** Tests, Enhancement ___ ### **Description** - Added a pre-request script for Tyk Gateway reload. - Configured Portman CLI for OpenAPI specification testing. - Introduced test cases for API and policy management. - Updated Swagger schema to allow empty string values. ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table> <tr> <td> <details> <summary><strong>prescript.js</strong><dd><code>Add pre-request script for Tyk Gateway reload</code> </dd></summary> <hr> ci/tests/schema/specs/config/prescript.js <li>Added a script to reload Tyk Gateway using API key.<br> <li> Included error handling for missing API key.<br> <li> Logged responses from the Tyk Gateway reload endpoint. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-d5d32e1394f9dd0559de91dabec5b31c9a90d8e4a3f9e5db2aff9b05f82bed43">+27/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>swagger.yml</strong><dd><code>Update Swagger schema to allow empty strings</code> </dd></summary> <hr> swagger.yml <li>Allowed empty string values for execution mode.<br> <li> Updated version enum to include empty string. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-8f3c4cb253eee09ae2401daa7279a8bbfbfd4168bb579c3ac0ee5c672d63bb2c">+2/-1</a> </td> </tr> </table></td></tr><tr><td><strong>Configuration changes</strong></td><td><table> <tr> <td> <details> <summary><strong>Taskfile.yml</strong><dd><code>Configure tasks for OpenAPI specification tests</code> </dd></summary> <hr> ci/tests/schema/specs/Taskfile.yml <li>Added tasks for running OpenAPI specification tests.<br> <li> Configured Venom and npm commands for testing. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-51702000b45f913d32d2ae4db64c31e5cbf915e241182975f938c2ad0b21ea1f">+9/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>portman-cli-options.json</strong><dd><code>Add Portman CLI configuration for testing</code> </dd></summary> <hr> ci/tests/schema/specs/config/portman-cli-options.json <li>Configured Portman CLI options for OpenAPI testing.<br> <li> Enabled test inclusion and Newman execution.<br> <li> Set up environment file and output paths. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-d8baf6a4141e957d52ea1f3e8589c3744e2210973ba8af02556f9d82502c1ae8">+12/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>portmanconfig.json</strong><dd><code>Define Portman configuration for OpenAPI tests</code> </dd></summary> <hr> ci/tests/schema/specs/config/portmanconfig.json <li>Defined contract tests for OpenAPI operations.<br> <li> Added variable assignments for API operations.<br> <li> Configured pre-request scripts and request overwrites.<br> <li> Included global settings for test execution order. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-129f9b9eb8a9bbbf3d6519a2ecee6f5d4d65e97ff4618ba99a7e8e30a6f29263">+144/-0</a> </td> </tr> </table></td></tr><tr><td><strong>Dependencies</strong></td><td><table> <tr> <td> <details> <summary><strong>package.json</strong><dd><code>Add Portman dependency and npm script</code> </dd></summary> <hr> ci/tests/schema/specs/package.json <li>Added Portman dependency for OpenAPI testing.<br> <li> Configured npm start script for Portman execution. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-bfba39305243a3cfcb0dd9e2c659f32326dbdca20e8f342f74f85725fd8d328e">+16/-0</a> </td> </tr> </table></td></tr><tr><td><strong>Tests</strong></td><td><table> <tr> <td> <details> <summary><strong>populate_gateway_test_data.yaml</strong><dd><code>Add test cases for API and Gateway management</code> </dd></summary> <hr> ci/tests/schema/specs/testdata/populate_gateway_test_data.yaml <li>Added test cases for API creation and deletion.<br> <li> Included a test case for Tyk Gateway reload.<br> <li> Configured assertions for HTTP responses. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-e3a75bb64613130bff9f2ba7fa867ab0f8d2fb1a4a95403c9e2a5f92802ffd60">+61/-0</a> </td> </tr> </table></td></tr></tr></tbody></table> ___ > 💡 **PR-Agent usage**: Comment `/help "your question"` on any pull request to receive relevant information [DX-1780]: https://tyktech.atlassian.net/browse/DX-1780?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: itachi sasuke <[email protected]> Co-authored-by: Tit Petric <[email protected]>
edsonmichaque
pushed a commit
that referenced
this pull request
Feb 14, 2025
# OpenAPI Specification Contract Testing with Portman To ensure that the schema in our gateway's OpenAPI specification matches our API, we have implemented contract tests using Portman. Portman leverages Postman collections to automatically generate contract tests from our OpenAPI specifications. ### What the Contract Tests Are Testing: 1. **Request Body Validation**: Verifying that the request body defined in our Swagger schema matches the request body expected by our gateway API. 2. **Response Body Validation**: Ensuring that the response returned by our gateway API matches the response body defined in our Swagger schema. 3. **Content-Type Validation**: Confirming that the content types defined in the request and response schemas in our Swagger documentation match those expected and returned by our gateway API. 4. **Header Validation**: Validating that the headers sent to or returned by the gateway API are as described in our Swagger documentation. Full documentation [can be found here](https://github.com/TykTechnologies/tyk/blob/5540baeb205a9466d661b49ecf19792fa6491f99/ci/tests/schema/specs/README.md) <details open> <summary><a href="https://tyktech.atlassian.net/browse/DX-1780" title="DX-1780" target="_blank">DX-1780</a></summary> <br /> <table> <tr> <th>Summary</th> <td>Create contract tests for the tyk gateway</td> </tr> <tr> <th>Type</th> <td> <img alt="Story" src="https://tyktech.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10315?size=medium" /> Story </td> </tr> <tr> <th>Status</th> <td>In Progress</td> </tr> <tr> <th>Points</th> <td>N/A</td> </tr> <tr> <th>Labels</th> <td>-</td> </tr> </table> </details> <!-- do not remove this marker as it will break jira-lint's functionality. added_by_jira_lint --> --- ### **User description** [DX-1780] <!-- Provide a general summary of your changes in the Title above --> ## Description <!-- Describe your changes in detail --> ## Related Issue <!-- This project only accepts pull requests related to open issues. --> <!-- If suggesting a new feature or change, please discuss it in an issue first. --> <!-- If fixing a bug, there should be an issue describing it with steps to reproduce. --> <!-- OSS: Please link to the issue here. Tyk: please create/link the JIRA ticket. --> ## Motivation and Context <!-- Why is this change required? What problem does it solve? --> ## How This Has Been Tested <!-- Please describe in detail how you tested your changes --> <!-- Include details of your testing environment, and the tests --> <!-- you ran to see how your change affects other areas of the code, etc. --> <!-- This information is helpful for reviewers and QA. --> ## Screenshots (if appropriate) ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If there are no documentation updates required, mark the item as checked. --> <!-- Raise up any additional concerns not covered by the checklist. --> - [ ] I ensured that the documentation is up to date - [ ] I explained why this PR updates go.mod in detail with reasoning why it's required - [ ] I would like a code coverage CI quality gate exception and have explained why ___ ### **PR Type** Tests, Enhancement ___ ### **Description** - Added a pre-request script for Tyk Gateway reload. - Configured Portman CLI for OpenAPI specification testing. - Introduced test cases for API and policy management. - Updated Swagger schema to allow empty string values. ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table> <tr> <td> <details> <summary><strong>prescript.js</strong><dd><code>Add pre-request script for Tyk Gateway reload</code> </dd></summary> <hr> ci/tests/schema/specs/config/prescript.js <li>Added a script to reload Tyk Gateway using API key.<br> <li> Included error handling for missing API key.<br> <li> Logged responses from the Tyk Gateway reload endpoint. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-d5d32e1394f9dd0559de91dabec5b31c9a90d8e4a3f9e5db2aff9b05f82bed43">+27/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>swagger.yml</strong><dd><code>Update Swagger schema to allow empty strings</code> </dd></summary> <hr> swagger.yml <li>Allowed empty string values for execution mode.<br> <li> Updated version enum to include empty string. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-8f3c4cb253eee09ae2401daa7279a8bbfbfd4168bb579c3ac0ee5c672d63bb2c">+2/-1</a> </td> </tr> </table></td></tr><tr><td><strong>Configuration changes</strong></td><td><table> <tr> <td> <details> <summary><strong>Taskfile.yml</strong><dd><code>Configure tasks for OpenAPI specification tests</code> </dd></summary> <hr> ci/tests/schema/specs/Taskfile.yml <li>Added tasks for running OpenAPI specification tests.<br> <li> Configured Venom and npm commands for testing. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-51702000b45f913d32d2ae4db64c31e5cbf915e241182975f938c2ad0b21ea1f">+9/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>portman-cli-options.json</strong><dd><code>Add Portman CLI configuration for testing</code> </dd></summary> <hr> ci/tests/schema/specs/config/portman-cli-options.json <li>Configured Portman CLI options for OpenAPI testing.<br> <li> Enabled test inclusion and Newman execution.<br> <li> Set up environment file and output paths. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-d8baf6a4141e957d52ea1f3e8589c3744e2210973ba8af02556f9d82502c1ae8">+12/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>portmanconfig.json</strong><dd><code>Define Portman configuration for OpenAPI tests</code> </dd></summary> <hr> ci/tests/schema/specs/config/portmanconfig.json <li>Defined contract tests for OpenAPI operations.<br> <li> Added variable assignments for API operations.<br> <li> Configured pre-request scripts and request overwrites.<br> <li> Included global settings for test execution order. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-129f9b9eb8a9bbbf3d6519a2ecee6f5d4d65e97ff4618ba99a7e8e30a6f29263">+144/-0</a> </td> </tr> </table></td></tr><tr><td><strong>Dependencies</strong></td><td><table> <tr> <td> <details> <summary><strong>package.json</strong><dd><code>Add Portman dependency and npm script</code> </dd></summary> <hr> ci/tests/schema/specs/package.json <li>Added Portman dependency for OpenAPI testing.<br> <li> Configured npm start script for Portman execution. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-bfba39305243a3cfcb0dd9e2c659f32326dbdca20e8f342f74f85725fd8d328e">+16/-0</a> </td> </tr> </table></td></tr><tr><td><strong>Tests</strong></td><td><table> <tr> <td> <details> <summary><strong>populate_gateway_test_data.yaml</strong><dd><code>Add test cases for API and Gateway management</code> </dd></summary> <hr> ci/tests/schema/specs/testdata/populate_gateway_test_data.yaml <li>Added test cases for API creation and deletion.<br> <li> Included a test case for Tyk Gateway reload.<br> <li> Configured assertions for HTTP responses. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-e3a75bb64613130bff9f2ba7fa867ab0f8d2fb1a4a95403c9e2a5f92802ffd60">+61/-0</a> </td> </tr> </table></td></tr></tr></tbody></table> ___ > 💡 **PR-Agent usage**: Comment `/help "your question"` on any pull request to receive relevant information [DX-1780]: https://tyktech.atlassian.net/browse/DX-1780?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: itachi sasuke <[email protected]> Co-authored-by: Tit Petric <[email protected]>
edsonmichaque
pushed a commit
to edsonmichaque/tyk-1
that referenced
this pull request
Feb 18, 2025
# OpenAPI Specification Contract Testing with Portman To ensure that the schema in our gateway's OpenAPI specification matches our API, we have implemented contract tests using Portman. Portman leverages Postman collections to automatically generate contract tests from our OpenAPI specifications. ### What the Contract Tests Are Testing: 1. **Request Body Validation**: Verifying that the request body defined in our Swagger schema matches the request body expected by our gateway API. 2. **Response Body Validation**: Ensuring that the response returned by our gateway API matches the response body defined in our Swagger schema. 3. **Content-Type Validation**: Confirming that the content types defined in the request and response schemas in our Swagger documentation match those expected and returned by our gateway API. 4. **Header Validation**: Validating that the headers sent to or returned by the gateway API are as described in our Swagger documentation. Full documentation [can be found here](https://github.com/TykTechnologies/tyk/blob/5540baeb205a9466d661b49ecf19792fa6491f99/ci/tests/schema/specs/README.md) <details open> <summary><a href="https://tyktech.atlassian.net/browse/DX-1780" title="DX-1780" target="_blank">DX-1780</a></summary> <br /> <table> <tr> <th>Summary</th> <td>Create contract tests for the tyk gateway</td> </tr> <tr> <th>Type</th> <td> <img alt="Story" src="https://tyktech.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10315?size=medium" /> Story </td> </tr> <tr> <th>Status</th> <td>In Progress</td> </tr> <tr> <th>Points</th> <td>N/A</td> </tr> <tr> <th>Labels</th> <td>-</td> </tr> </table> </details> <!-- do not remove this marker as it will break jira-lint's functionality. added_by_jira_lint --> --- ### **User description** [DX-1780] <!-- Provide a general summary of your changes in the Title above --> ## Description <!-- Describe your changes in detail --> ## Related Issue <!-- This project only accepts pull requests related to open issues. --> <!-- If suggesting a new feature or change, please discuss it in an issue first. --> <!-- If fixing a bug, there should be an issue describing it with steps to reproduce. --> <!-- OSS: Please link to the issue here. Tyk: please create/link the JIRA ticket. --> ## Motivation and Context <!-- Why is this change required? What problem does it solve? --> ## How This Has Been Tested <!-- Please describe in detail how you tested your changes --> <!-- Include details of your testing environment, and the tests --> <!-- you ran to see how your change affects other areas of the code, etc. --> <!-- This information is helpful for reviewers and QA. --> ## Screenshots (if appropriate) ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If there are no documentation updates required, mark the item as checked. --> <!-- Raise up any additional concerns not covered by the checklist. --> - [ ] I ensured that the documentation is up to date - [ ] I explained why this PR updates go.mod in detail with reasoning why it's required - [ ] I would like a code coverage CI quality gate exception and have explained why ___ ### **PR Type** Tests, Enhancement ___ ### **Description** - Added a pre-request script for Tyk Gateway reload. - Configured Portman CLI for OpenAPI specification testing. - Introduced test cases for API and policy management. - Updated Swagger schema to allow empty string values. ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table> <tr> <td> <details> <summary><strong>prescript.js</strong><dd><code>Add pre-request script for Tyk Gateway reload</code> </dd></summary> <hr> ci/tests/schema/specs/config/prescript.js <li>Added a script to reload Tyk Gateway using API key.<br> <li> Included error handling for missing API key.<br> <li> Logged responses from the Tyk Gateway reload endpoint. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-d5d32e1394f9dd0559de91dabec5b31c9a90d8e4a3f9e5db2aff9b05f82bed43">+27/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>swagger.yml</strong><dd><code>Update Swagger schema to allow empty strings</code> </dd></summary> <hr> swagger.yml <li>Allowed empty string values for execution mode.<br> <li> Updated version enum to include empty string. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-8f3c4cb253eee09ae2401daa7279a8bbfbfd4168bb579c3ac0ee5c672d63bb2c">+2/-1</a> </td> </tr> </table></td></tr><tr><td><strong>Configuration changes</strong></td><td><table> <tr> <td> <details> <summary><strong>Taskfile.yml</strong><dd><code>Configure tasks for OpenAPI specification tests</code> </dd></summary> <hr> ci/tests/schema/specs/Taskfile.yml <li>Added tasks for running OpenAPI specification tests.<br> <li> Configured Venom and npm commands for testing. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-51702000b45f913d32d2ae4db64c31e5cbf915e241182975f938c2ad0b21ea1f">+9/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>portman-cli-options.json</strong><dd><code>Add Portman CLI configuration for testing</code> </dd></summary> <hr> ci/tests/schema/specs/config/portman-cli-options.json <li>Configured Portman CLI options for OpenAPI testing.<br> <li> Enabled test inclusion and Newman execution.<br> <li> Set up environment file and output paths. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-d8baf6a4141e957d52ea1f3e8589c3744e2210973ba8af02556f9d82502c1ae8">+12/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>portmanconfig.json</strong><dd><code>Define Portman configuration for OpenAPI tests</code> </dd></summary> <hr> ci/tests/schema/specs/config/portmanconfig.json <li>Defined contract tests for OpenAPI operations.<br> <li> Added variable assignments for API operations.<br> <li> Configured pre-request scripts and request overwrites.<br> <li> Included global settings for test execution order. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-129f9b9eb8a9bbbf3d6519a2ecee6f5d4d65e97ff4618ba99a7e8e30a6f29263">+144/-0</a> </td> </tr> </table></td></tr><tr><td><strong>Dependencies</strong></td><td><table> <tr> <td> <details> <summary><strong>package.json</strong><dd><code>Add Portman dependency and npm script</code> </dd></summary> <hr> ci/tests/schema/specs/package.json <li>Added Portman dependency for OpenAPI testing.<br> <li> Configured npm start script for Portman execution. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-bfba39305243a3cfcb0dd9e2c659f32326dbdca20e8f342f74f85725fd8d328e">+16/-0</a> </td> </tr> </table></td></tr><tr><td><strong>Tests</strong></td><td><table> <tr> <td> <details> <summary><strong>populate_gateway_test_data.yaml</strong><dd><code>Add test cases for API and Gateway management</code> </dd></summary> <hr> ci/tests/schema/specs/testdata/populate_gateway_test_data.yaml <li>Added test cases for API creation and deletion.<br> <li> Included a test case for Tyk Gateway reload.<br> <li> Configured assertions for HTTP responses. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-e3a75bb64613130bff9f2ba7fa867ab0f8d2fb1a4a95403c9e2a5f92802ffd60">+61/-0</a> </td> </tr> </table></td></tr></tr></tbody></table> ___ > 💡 **PR-Agent usage**: Comment `/help "your question"` on any pull request to receive relevant information [DX-1780]: https://tyktech.atlassian.net/browse/DX-1780?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: itachi sasuke <[email protected]> Co-authored-by: Tit Petric <[email protected]>
edsonmichaque
pushed a commit
to edsonmichaque/tyk-1
that referenced
this pull request
Feb 18, 2025
# OpenAPI Specification Contract Testing with Portman To ensure that the schema in our gateway's OpenAPI specification matches our API, we have implemented contract tests using Portman. Portman leverages Postman collections to automatically generate contract tests from our OpenAPI specifications. ### What the Contract Tests Are Testing: 1. **Request Body Validation**: Verifying that the request body defined in our Swagger schema matches the request body expected by our gateway API. 2. **Response Body Validation**: Ensuring that the response returned by our gateway API matches the response body defined in our Swagger schema. 3. **Content-Type Validation**: Confirming that the content types defined in the request and response schemas in our Swagger documentation match those expected and returned by our gateway API. 4. **Header Validation**: Validating that the headers sent to or returned by the gateway API are as described in our Swagger documentation. Full documentation [can be found here](https://github.com/TykTechnologies/tyk/blob/5540baeb205a9466d661b49ecf19792fa6491f99/ci/tests/schema/specs/README.md) <details open> <summary><a href="https://tyktech.atlassian.net/browse/DX-1780" title="DX-1780" target="_blank">DX-1780</a></summary> <br /> <table> <tr> <th>Summary</th> <td>Create contract tests for the tyk gateway</td> </tr> <tr> <th>Type</th> <td> <img alt="Story" src="https://tyktech.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10315?size=medium" /> Story </td> </tr> <tr> <th>Status</th> <td>In Progress</td> </tr> <tr> <th>Points</th> <td>N/A</td> </tr> <tr> <th>Labels</th> <td>-</td> </tr> </table> </details> <!-- do not remove this marker as it will break jira-lint's functionality. added_by_jira_lint --> --- ### **User description** [DX-1780] <!-- Provide a general summary of your changes in the Title above --> ## Description <!-- Describe your changes in detail --> ## Related Issue <!-- This project only accepts pull requests related to open issues. --> <!-- If suggesting a new feature or change, please discuss it in an issue first. --> <!-- If fixing a bug, there should be an issue describing it with steps to reproduce. --> <!-- OSS: Please link to the issue here. Tyk: please create/link the JIRA ticket. --> ## Motivation and Context <!-- Why is this change required? What problem does it solve? --> ## How This Has Been Tested <!-- Please describe in detail how you tested your changes --> <!-- Include details of your testing environment, and the tests --> <!-- you ran to see how your change affects other areas of the code, etc. --> <!-- This information is helpful for reviewers and QA. --> ## Screenshots (if appropriate) ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If there are no documentation updates required, mark the item as checked. --> <!-- Raise up any additional concerns not covered by the checklist. --> - [ ] I ensured that the documentation is up to date - [ ] I explained why this PR updates go.mod in detail with reasoning why it's required - [ ] I would like a code coverage CI quality gate exception and have explained why ___ ### **PR Type** Tests, Enhancement ___ ### **Description** - Added a pre-request script for Tyk Gateway reload. - Configured Portman CLI for OpenAPI specification testing. - Introduced test cases for API and policy management. - Updated Swagger schema to allow empty string values. ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table> <tr> <td> <details> <summary><strong>prescript.js</strong><dd><code>Add pre-request script for Tyk Gateway reload</code> </dd></summary> <hr> ci/tests/schema/specs/config/prescript.js <li>Added a script to reload Tyk Gateway using API key.<br> <li> Included error handling for missing API key.<br> <li> Logged responses from the Tyk Gateway reload endpoint. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-d5d32e1394f9dd0559de91dabec5b31c9a90d8e4a3f9e5db2aff9b05f82bed43">+27/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>swagger.yml</strong><dd><code>Update Swagger schema to allow empty strings</code> </dd></summary> <hr> swagger.yml <li>Allowed empty string values for execution mode.<br> <li> Updated version enum to include empty string. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-8f3c4cb253eee09ae2401daa7279a8bbfbfd4168bb579c3ac0ee5c672d63bb2c">+2/-1</a> </td> </tr> </table></td></tr><tr><td><strong>Configuration changes</strong></td><td><table> <tr> <td> <details> <summary><strong>Taskfile.yml</strong><dd><code>Configure tasks for OpenAPI specification tests</code> </dd></summary> <hr> ci/tests/schema/specs/Taskfile.yml <li>Added tasks for running OpenAPI specification tests.<br> <li> Configured Venom and npm commands for testing. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-51702000b45f913d32d2ae4db64c31e5cbf915e241182975f938c2ad0b21ea1f">+9/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>portman-cli-options.json</strong><dd><code>Add Portman CLI configuration for testing</code> </dd></summary> <hr> ci/tests/schema/specs/config/portman-cli-options.json <li>Configured Portman CLI options for OpenAPI testing.<br> <li> Enabled test inclusion and Newman execution.<br> <li> Set up environment file and output paths. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-d8baf6a4141e957d52ea1f3e8589c3744e2210973ba8af02556f9d82502c1ae8">+12/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>portmanconfig.json</strong><dd><code>Define Portman configuration for OpenAPI tests</code> </dd></summary> <hr> ci/tests/schema/specs/config/portmanconfig.json <li>Defined contract tests for OpenAPI operations.<br> <li> Added variable assignments for API operations.<br> <li> Configured pre-request scripts and request overwrites.<br> <li> Included global settings for test execution order. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-129f9b9eb8a9bbbf3d6519a2ecee6f5d4d65e97ff4618ba99a7e8e30a6f29263">+144/-0</a> </td> </tr> </table></td></tr><tr><td><strong>Dependencies</strong></td><td><table> <tr> <td> <details> <summary><strong>package.json</strong><dd><code>Add Portman dependency and npm script</code> </dd></summary> <hr> ci/tests/schema/specs/package.json <li>Added Portman dependency for OpenAPI testing.<br> <li> Configured npm start script for Portman execution. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-bfba39305243a3cfcb0dd9e2c659f32326dbdca20e8f342f74f85725fd8d328e">+16/-0</a> </td> </tr> </table></td></tr><tr><td><strong>Tests</strong></td><td><table> <tr> <td> <details> <summary><strong>populate_gateway_test_data.yaml</strong><dd><code>Add test cases for API and Gateway management</code> </dd></summary> <hr> ci/tests/schema/specs/testdata/populate_gateway_test_data.yaml <li>Added test cases for API creation and deletion.<br> <li> Included a test case for Tyk Gateway reload.<br> <li> Configured assertions for HTTP responses. </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6827/files#diff-e3a75bb64613130bff9f2ba7fa867ab0f8d2fb1a4a95403c9e2a5f92802ffd60">+61/-0</a> </td> </tr> </table></td></tr></tr></tbody></table> ___ > 💡 **PR-Agent usage**: Comment `/help "your question"` on any pull request to receive relevant information [DX-1780]: https://tyktech.atlassian.net/browse/DX-1780?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: itachi sasuke <[email protected]> Co-authored-by: Tit Petric <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
OpenAPI Specification Contract Testing with Portman
To ensure that the schema in our gateway's OpenAPI specification matches our API, we have implemented contract tests using Portman. Portman leverages Postman collections to automatically generate contract tests from our OpenAPI specifications.
What the Contract Tests Are Testing:
Request Body Validation: Verifying that the request body defined in our Swagger schema matches the request body expected by our gateway API.
Response Body Validation: Ensuring that the response returned by our gateway API matches the response body defined in our Swagger schema.
Content-Type Validation: Confirming that the content types defined in the request and response schemas in our Swagger documentation match those expected and returned by our gateway API.
Header Validation: Validating that the headers sent to or returned by the gateway API are as described in our Swagger documentation.
Full documentation can be found here
DX-1780
User description
DX-1780
Description
Related Issue
Motivation and Context
How This Has Been Tested
Screenshots (if appropriate)
Types of changes
Checklist
PR Type
Tests, Enhancement
Description
Added a pre-request script for Tyk Gateway reload.
Configured Portman CLI for OpenAPI specification testing.
Introduced test cases for API and policy management.
Updated Swagger schema to allow empty string values.
Changes walkthrough 📝
prescript.js
Add pre-request script for Tyk Gateway reloadci/tests/schema/specs/config/prescript.js
swagger.yml
Update Swagger schema to allow empty stringsswagger.yml
Taskfile.yml
Configure tasks for OpenAPI specification testsci/tests/schema/specs/Taskfile.yml
portman-cli-options.json
Add Portman CLI configuration for testingci/tests/schema/specs/config/portman-cli-options.json
portmanconfig.json
Define Portman configuration for OpenAPI testsci/tests/schema/specs/config/portmanconfig.json
package.json
Add Portman dependency and npm scriptci/tests/schema/specs/package.json
populate_gateway_test_data.yaml
Add test cases for API and Gateway managementci/tests/schema/specs/testdata/populate_gateway_test_data.yaml