Tests
Most of the metrics presented in the paper are the same as some of the basic ones, so tests are pretty straightforward. We used the model showed in the paper for our tests (00 Test).
Its metric values are:
- TNSE: 1
- TNIE: 4
- TNEE: 1
- TNE: 6
- TNT: 2
- TNCS: 4
- TNA: 6
- TNG: 4
- TNDO: 0
- CLA: 3
- CLP: 0
- PDOPIn: 0
- PDOPOut: 0
- PDOTOut: 0
- PLT: 0
Our tool extracts exactly the same values, except for the TNT and the TNCS. Probably this is caused by an issue with the original model, as it shows only 2 collapsed sub-processes and 4 normal task, despite the fact that the values reported in the paper are reversed.
The only metric present here is the Control-Flow Complexity. No example is reported in the paper, so we made one that included every type of relevant gateway (01 Test).
The result is correct and it is 7:
-
- 2 from the Exclusive Gateway
-
- 3 from the Inclusive Gateway
-
- 1 from the Parallel Gateway
-
- 1 from the Uncontrolled Split
For the first four metrics, we designed a process that has multiple type of activities and some gateways.
The results are:
- NOF: 11 (this is the same as TNSF)
- NOA: 5 (normal task, collapsed sub-processe, call activity, user task, manual task)
- NOAC: 8 (activities + 3 gateways)
- NOAJS: 8 (activities + 2 split gateways + 1 join gateway)
For the IC and the IC-related metrics, we use the same test model, as it already has several types of activities. The results are:
- NoI: 2
- NoO: 1
- Length: 5
- IC: 20 {Lenght * (NoI * NoO)^2 -> 5 * (2 * 1)^2}
The paper reports 3 models as examples, that we reproduced in the test's model directory (Test 04a, Test 04b, Test 04c).
Model 04a:
Model 04b:
Model 04c:
- First model's CC: 0,09131
- Second model's CC: in the paper -> 0.12486 // with BPMN Metadata Extractor -> 0.11056
- Third model's CC: 0.08503
For the ICP and ECP, we created a model that presented both subprocess with tasks and flows inside and outside it (Test 05), so that we would have the types of sequence flows that can cover every possible scenario for this metrics.
The results are:
- ICP: 5
- ECP: 5
For this metric we created a model containing all the possibile case that can be recognized by it.
- W: 25
that is the sum of:
- Sequence: + 1
- Exclusive split Gateways with 2 branches: + 2
- Exclusive split Gateways with more than 2 branches: + 3
- Parallel Gateways: + 4
- Inclusive Gateways: + 7
- Subprocesses: + 2
- Multiple Instance Characteristics: + 6
- Cancel Events: + 1
To test the Maxmimum and the Mean Nesting Depth we used the model used for the previous metrics, as it presents more than one Exclusive Gateway.
The results are:
- MaxND: 2
- MeanND: 1
The exclusive split gateways are sequential, and they are 2, therefore the value of the MaxND. The sum of the ND of every node is 22, and the node themselves are 22, so the Mean is 1.
To test the Coupling Metrics, we used a process that we specially created to have several types of activities, linked between them.
The result is:
- CP: 0.114
We tested the following metrics with a model reported as an example in the paper. This is the model:
The results are:
- Sn: 27
- Sequentiality: 0.344
- Diameter: 14
- Depth: 1
- GM: 8
- CH: 0.819
- CYC: 0.259
- TS: 2
- Density: 0.0413
We also tested the following metrics with the previous model.
- MCD: 3
- ACD: 3
We also tested the following metrics with the model for "Metric for Process Models".
- ECaM: 8
- ECyM: 23
We tested the following metrics with a model reported as an example in the paper. This is the model:
- DSM: 3
- PSM: 4
We also tested the following metrics with the model for "Metric for Process Models".
- Layout_Complexity: -32.28
We tested this metric on nonRectilinearFlows, shapesPosition and various models that we modified with Signavio's modeler, to juxtapose the shapes, as it is not permitted in bpmn.io's modeler.