Allow multiple scripts to be given to "artillery run"

[hassy]

Use case: when testing a complex application (an ecommerce website backend in my case), it would be nice to keep scenario definitions in separate files for readability.

It's already possible to extract parts of a flow (e.g. checking out) into their own "subflows" using YAML anchors/references to reduce repetition. The ability to separate scenarios into different files would be another step towards making large test definitions easier to read and maintain.

Mechanism:

The run command can accept a list of files:

artillery run main.yaml scenario1.yaml scenario2.yaml scenario3.yaml...

Where scenario1.yaml etc have the following structure:

config:
  # optional payload definition - gets added to the global payload definitions
  payload:
    path: "./somefile.csv"
    fields: ["field1", "field2"]
  # optional variable definition - as above
  variables:
    somevar: ["value1", "value2"]
  # optional processor definition - as above
  processor: "./functionsForTheScenarios.js"
# Scenarios defined here get added to the global list
scenarios:
  - name: "Select a product and add it to cart"
     flow:
       # flow definition goes here

All paths are to be resolved relative to the path to the scenario file.
Any fields other than defined above will be ignored (e.g. config.environments - those should be set in main.yaml).

[steveschnepp]

Letting the main.ymlhaving special rights is counterintuitive. A much better way would be to have an #include mechanism, which then will just fit in.

[Offtree]

Have you considered allowing glob patterns instead of having to list out scenarios? Seems like part of the goal with this enhancement would be to allow for easier integration with a full team and not really have to think about what tests your testing pipeline is running. This would align with common practices with other testing frameworks.

Ex.

artillery run scenario.*.yaml
artillery run ./tests/**/*.scenario.yaml

[vichybaby]

Is anyone having a working example?

For me, only the first scenario got executed successfully.
artillery run scenario1.yaml scenario2.yaml...

[sultanzarif]

Hi,

I have many yaml test files, each including its own config and scenarios. I would like to run all tests at the same time, and using artillery run test1.yml test2.yml test3.yaml... (each file contains both config and scenarios, there is no separate config file); however, it is only picking up the last file in my case. Is there a command to run all files at one go?

[hassy]

@sultanzarif and anyone else looking for a solution - I have created an example you can run here: https://github.com/artilleryio/artillery/tree/master/examples/multiple-scenario-specs

This lets you run each scenario individually, or run all scenarios one after another.

I will create a separate issue for adding the ability to combine multiple scenarios into one, and run those in a single Artillery command. The feature that the original post talks about here is already supported by Artillery.

[hassy]

Tracking in #1582

[eliezercazarestravelport]

@sultanzarif and anyone else looking for a solution - I have created an example you can run here: https://github.com/artilleryio/artillery/tree/master/examples/multiple-scenario-specs

This lets you run each scenario individually, or run all scenarios one after another.

I will create a separate issue for adding the ability to combine multiple scenarios into one, and run those in a single Artillery command. The feature that the original post talks about here is already supported by Artillery.

Hello Hassy, hope you're doing well. That's a nice approach but how can we get it all in the same report?

[nithinr19]

Hi Team,
i have multiple yaml files to execute. how do i put these yaml files in single yaml and execute.