From 64f1957ca3a4c6ee14d79fa496f20af831fb8650 Mon Sep 17 00:00:00 2001 From: atestini Date: Fri, 22 Jul 2022 14:53:40 -0400 Subject: [PATCH 1/5] Improve readibility and wording --- docs/aetest/parameters.rst | 277 ++++++++++++++++++------------------- 1 file changed, 135 insertions(+), 142 deletions(-) diff --git a/docs/aetest/parameters.rst b/docs/aetest/parameters.rst index a5b93e8..96a52b8 100644 --- a/docs/aetest/parameters.rst +++ b/docs/aetest/parameters.rst @@ -7,27 +7,27 @@ Test Parameters - `Function Arguments`_ - `Data-Driven Programming`_ - - `Mutable/Immuntable Objects`_ + - `Mutable/Immutable Objects`_ - `Variable Scoping`_ .. _Data-Driven Programming: http://en.wikipedia.org/wiki/Data-driven_programming .. _Function Arguments: https://docs.python.org/3.4/tutorial/controlflow.html#more-on-defining-functions -.. _Mutable/Immuntable Objects: http://en.wikibooks.org/wiki/Python_Programming/Data_Types#Mutable_vs_Immutable_Objects +.. _Mutable/Immutable Objects: http://en.wikibooks.org/wiki/Python_Programming/Data_Types#Mutable_vs_Immutable_Objects .. _Variable Scoping: https://docs.python.org/3.4/reference/executionmodel.html -``aetest`` is a data-driven test infrastructure. Its scripts and testcases are +``aetest`` is a data-driven test infrastructure. Its scripts and test cases are intended to be driven dynamically by: -- data in the form of input arguments to the testscript -- dynamic data generated during runtime +- Data in the form of input arguments to the test script +- Dynamic data generated during runtime The collection of dynamic data that artificially affects the behavior of -testscripts and testcases in ``aetest`` is called **parameters**. It adheres to -a pre-defined set of parent/child propagation relationships, and may be used as +test scripts and test cases in ``aetest`` is called **parameters**. It adheres to +a pre-defined set of parent/child propagation relationships and may be used as `Function Arguments`_ to each test section. -This feature is a supplement to static testcase data (attribute values stored -within each testcases). +This feature is a supplement to static test case data (attribute values stored +within each test case). .. tip:: @@ -43,10 +43,10 @@ Before delving deeper into the concept and mechanism of **parameters**, let's first spend some time on what it is, why they are needed, and how they may benefit the end user. -Parameters are a special kind of variable, used within functions and methods to -refer and access its input data (arguments). If you consider the function and -methods as the "doer", then parameters are the "what to do with". Eg, an -``add`` function would require 2 or more parameters to be "added together". +Parameters are a special kind of variable used within functions and methods to +refer to and access its input data (arguments). If you consider the function and +methods as the "doer," then parameters are the "what to do with." E.g., an +``add`` function would require two or more parameters to be "added together". .. code-block:: python @@ -74,33 +74,33 @@ methods as the "doer", then parameters are the "what to do with". Eg, an add_to_c(1, 2) # 103 -In a data-driven testing, testscripts are the *doers*, performing the act -of testing a facet of some product. Its arguments and parameters are thus the -input data that influences the specific acts of testing being carried out. Here +In data-driven testing, test scripts are the *doers*, performing the act +of testing a facet of some product. Test script arguments and parameters are thus the +input data that influences the specific actions of testing. Here are some possible use cases: - - ``testbed`` argument to a script tells it which testbed to run on, and + - The ``testbed`` argument to a script tells it which testbed to run on, and what the topology is like. The script can then connect to the testbed, and decide how to perform the testing best suited for this topology. - - ``vlan`` argument to ``layer2_traffic`` script can dynamically control the - vlan to be be configured for traffic testing. + - The ``vlan`` argument to the ``layer2_traffic`` script can dynamically control the + VLAN to be configured for traffic testing. - - other toggle arguments that dynamically turns on/off certain testcases, - and/or combination of features to be configured & tested + - Other toggle arguments that dynamically turn on/off certain test cases, + and a combination of features to be configured & tested - - etc. + - Etc. Of course, the parameters feature in ``aetest`` is much more than just script -arguments. It enables users to write testcases and testscripts that are capable +arguments. It enables users to write test cases and test scripts that are capable of being driven by inputs, varying the degree of testing, etc. Relationship Model ------------------ In ``aetest``, parameters are **relative**: parameters corresponding to each -object is the combination of its local specific parameters, and all of its -parent object's parameters. Eg: +object are the combination of its local specific parameters and all of its +parent object's parameters. E.g.: - ``Testcase`` parameters = local parameters + ``TestScript`` parameters @@ -113,9 +113,9 @@ runtime behaviors section: each object has a parent, and the parameters seen at each object level is an aggregation of its and all of its parent's parameters. -In case when an object and its parent (or its parent's parent, etc) have the same +In case when an object and its parent (or its parent's parent, etc.) have the same parameter names defined, then the parameter value closest to the current scope -is used/prefered. +is used/preferred. .. figure:: parameter_relations.png :align: center @@ -123,7 +123,7 @@ is used/prefered. *Parameter Relationship Model In a Nutshell* Below is a behavior demonstration of this relationship model. In actual script -execution, this happens behind-the-scenes automatically. +execution, this happens behind the scenes automatically. .. code-block:: python @@ -153,23 +153,23 @@ execution, this happens behind-the-scenes automatically. } # during runtime, the combined parameters seen at the - # testcase level, would be equivalent to the following: - # - take the testscript parameters as basis + # testcase level would be equivalent to the following: + # - take the testscript parameters as the basis # - and add to it, testcase parameters # - # eg: + #, e.g.: new_testcase_parameters = testscript.parameters.copy() new_testcase_parameters.update(testcase.parameters) testcase.parameters = new_testcase_parameters - # so that the new parameters seen at the testcase + # so that the new parameters seen in the test case # level, is: testcase.parameters # {'param_A': 100, 'param_B': 2, 'param_C': 3} .. hint:: - in other words, childs inherits but shadows parent parameters. This is + In other words, children inherit but shadow parent parameters. This is similar to Python `Variable Scoping`_ concept. Parameters Property @@ -178,7 +178,7 @@ Parameters Property Every top-level object in ``aetest`` comes with the special ``parameters`` property: a dictionary containing the key/value data pairs relative to this object (:ref:`object_model`). Its default values can be set/updated by the user -within the testscript. +within the test script. .. code-block:: python @@ -200,10 +200,10 @@ within the testscript. } # using Testcase to demonstrate TestContainer-type parameters definitions - # note that this also is applicable to CommonSetup & CommonCleanup + # note that this also applies to CommonSetup & CommonCleanup class Testcase(aetest.Testcase): - # all default parameters specific to this testcase is declared + # all default parameters specific to this test case are declared # in its own parameters dictionary. parameters = { 'generic_param_A': 200 @@ -212,7 +212,7 @@ within the testscript. # etc ... During runtime, these dictionaries form the baseline ``parameters`` properties -of their corresponding section. Eg: +of their corresponding section. E.g.: - script-level ``parameters`` dictionary is used to create ``TestScript`` object parameters. @@ -221,7 +221,7 @@ of their corresponding section. Eg: One exception to the above is method local parameters for sections such as ``subsection``, ``setup``, ``test`` and ``cleanup``. Even though their -corresponding classes (``Subection``, ``SetupSection``, ``TestSection``, +corresponding classes (``Subsection``, ``SetupSection``, ``TestSection``, ``CleanupSection``) also have the parameters property, these class instances only exists briefly during runtime (see :ref:`aetest_function_classes`), so their attributes are mostly only dynamic in nature, set & controlled by the @@ -235,9 +235,9 @@ dynamically access & update parameters. .. important:: - even though parameters seen at each object level also includes its parent's + Even though parameters seen at each object level also include its parent's parameters, setting & updating the parameters dictionary is only reflected - locally, and does not propagate to the parent. This is also inline with how + locally, and does not propagate to the parent. This is also in line with how Python `Variable Scoping`_ works. .. code-block:: python @@ -247,7 +247,7 @@ dynamically access & update parameters. # # continuing from the above - # re-defining the testcase for the sake of code-continuity + # re-defining the test case for the sake of code-continuity class Testcase(aetest.Testcase): # local parameters defaults, same as above @@ -258,7 +258,7 @@ dynamically access & update parameters. # within any sections, the parent container parameters are directly # accessible (applicable to setup/test/cleanup and subsections) - # here we'll do a combination access & updating of parameters + # here, we'll do a combination of access & updating of parameters @aetest.setup def setup(self): # add to the parameters dict @@ -272,7 +272,7 @@ dynamically access & update parameters. @aetest.test def test(self): - # access & print parent testscript parameters + # access & print parent test script parameters # (following the parent model) print(self.parent.parameters) # {'generic_param_A': 100, @@ -288,12 +288,11 @@ dynamically access & update parameters. # 'testscript_param_B': [], # 'testscript_param_A': 'another value'} - -Consider the above example: parameters can be set and acccessed as the script -runs, opening up the opportunity for scripts to dynamically discover the runtime -environment and modify test behavior (parameters) as required. Eg, ``setup`` -section of modifying testcase parameters based on current testbed state, and -altering the behavior of ensuiting ``test`` sections, etc. +Consider the above example: parameters can be set and accessed as the script +runs, opening the opportunity for scripts to dynamically discover the runtime +environment and modify test behavior (parameters) as required. E.g., the ``setup`` +section of modifying test case parameters based on the current testbed state, and +altering the behavior of ensuing ``test`` sections, etc. .. tip:: @@ -302,13 +301,12 @@ altering the behavior of ensuiting ``test`` sections, etc. .. _Collections.ChainMap: https://docs.python.org/3/library/collections.html#collections.ChainMap - .. _script_args: Script Arguments ---------------- -In short, any arguments passed to the testscript before startup becomes part of +In short, any arguments passed to the test script before startup becomes part of the ``TestScript`` parameter. This includes all the arguments passed through the :ref:`easypy_jobfile` during :ref:`aetest_jobfile_execution`, and/or any command line arguments parsed and passed to ``aetest.main()`` during @@ -320,12 +318,12 @@ line arguments parsed and passed to ``aetest.main()`` during # ------- # # script parameters and how it works - # (pseudo code, for demonstration only) + # (pseudo-code, for demonstration only) - # without going into details of how script parameters/arguments are + # without going into details about how script parameters/arguments are # passed in (covered under Running Scripts section) - # assuming that the testscript was called with the following inputs + # assuming that the test script was called with the following inputs script_arguments = { 'arg_a': 100, 'arg_c': 3, @@ -347,15 +345,15 @@ line arguments parsed and passed to ``aetest.main()`` during # 'arg_b': 2, # 'arg_c': 3} -As demonstrated in the above example, script arguments/parameters are actually -added on top of ``TestScript`` parameters defined within the script. In +As demonstrated in the above example, script arguments/parameters are +added on top of the ``TestScript`` parameters defined within the script. In other words, script arguments are added dynamically to the current running -script's base parameters, and overwrites any existing ones. +script's base parameters and overwrite existing ones. .. tip:: - define your default parameters in the script, and change the behavior of - the testscript by overwriting specific ones using script arguments. + Define your default parameters in the script, and change the behavior of + the test script by overwriting specific ones using script arguments. .. _parameters_as_funcargs: @@ -364,15 +362,15 @@ Parameters as Function Arguments ``parameters`` property & functionality provides a means for objects within ``aetest`` to follow the :ref:`parent` model and aggregate data together in a -clean, accessible format. It serves also as the basis for providing section -methods their `Function Arguments`_. This is the main mechanism behind the +clean, accessible format. It also serves as the basis for providing section +methods their `Function Arguments`_. This is the primary mechanism behind the data-driven concept of ``aetest``: function/methods are **driven** by input parameters. During runtime, all section method function arguments are filled by their corresponding parameter value, matching the argument name vs. the parameter key/name. The following table describes the types of function arguments and -their supports. +their support. .. csv-table:: Function Argument Types & Parameters Support :header: "Type", "Example", "Comments" @@ -404,9 +402,9 @@ their supports. class Testcase(aetest.Testcase): - # this setup section definition identifies "param_B" as - # as a input requirement. as this parameter is available at this - # testcase level (aggregated from parent testscript), it + # this setup section definition identifies "param_B" + # as an input requirement. As this parameter is available at this + # test case level (aggregated from the parent test script), it # is passed in as input @aetest.setup def setup(self, param_B): @@ -442,32 +440,31 @@ their supports. This is the preferred method of accessing parameters: by passing each in explicitly as function arguments. It is more pythonic: - - explictly passing in parameters makes the code (and its dependencies) + - Explicitly passing in parameters makes the code (and its dependencies) easier to read and understand - - allows the infrastructure to handle error scenarios (such as missing + - Allows the infrastructure to handle error scenarios (such as missing parameters) - - allows users to easily define defaults (without dealing with dictionary + - Allows users to easily define defaults (without dealing with dictionary operations) - - maintaining the ability to call each section as a function with various + - Maintaining the ability to call each section as a function with various arguments during test/debugging situations. - - etc... + - Etc. .. tip:: - once a parameter is passed into a section as a function argument, it becomes + Once a parameter is passed into a section as a function argument, it becomes a local variable. All rules of `Variable Scoping`_ apply. - Callable Parameters ------------------- -A callable parameter is a one that evaluates to ``True`` using callable_. When +A callable parameter is one that evaluates to ``True`` using callable_. When a callable parameter is filled as a function argument to test sections, the -infrastructure "calls" it, and uses its return value as the actual argument +infrastructure "calls" it and uses its return value as the actual argument parameter. .. code-block:: python @@ -490,65 +487,63 @@ parameter. class Testcase(aetest.Testcase): - # as the "number" parameter's value is the callable function + # As the "number" parameter's value is the callable function # random.random, this function is evaluated right before the # execution of this test method, and the call result is then used # as the actual argument input @aetest.test def test(self, number): - # test whether the generated number is greater than 0.5 + # Test whether the generated number is greater than 0.5 assert number > 0.5 - # if you run this test enough times + # If you run this test enough times # you will find that it passes exactly 50% of the time # assuming that random.random() generate truly random numbers :) - # note that callable parameters are only evaluated if used - # as function arguments. they are still objects if viewed - # through the parameters property + # Note that callable parameters are only evaluated if used + # as function arguments. Callable parameters are still objects + # if viewed through the parameters property self.parameters['number'] # - -Callable parameters still shows up as their original function object when -accessed through ``parameters`` property. They are only evaluated (called) when +Callable parameters still appear as their original function object when +accessed through the ``parameters`` property. They are only evaluated (called) when used as function arguments to test methods. This evaluation occurs "on demand": - - the evalution takes place right before method execution. + - The evaluation takes place right before method execution. - - each method gets its own indepedent evaluated result. + - Each method gets its independent evaluated result. The only limitation with callable parameters is that they cannot have arguments. ``aetest`` would not know how to fulfill them during runtime. .. tip:: - eliminate function arguments (by pre-setting them) with + Eliminate function arguments (by pre-setting them) with `partial functions`_. .. _callable: https://docs.python.org/3.4/library/functions.html#callable .. _partial functions: https://docs.python.org/3.4/library/functools.html#functools.partial - Parametrizing Functions ----------------------- Parametrized functions are a special breed of "smart" callable parameters. They -support arguments, are capable of identifying the current execution context, and -act accordingly. +support arguments, can identify the current execution context, and +can act accordingly. A parametrized function is declared when the ``@parameters.parametrize`` -decorator is used on a function in your testscript. This also adds the newly +decorator is used on a function within a test script. This also adds the newly created parametrized function automatically as part of ``TestScript`` parameters, using the function name as the parameter name. -During runtime, the behavior of these parametrized functions is exactly +During runtime, the behavior of these parametrized functions are identical to its callable parameter sibling, with the following additions: - - any arguments to the ``@parameters.parametrize`` decorator are stored + - Any arguments to the ``@parameters.parametrize`` decorator are stored and used as function arguments during the evaluation. - - if an argument named ``section`` is defined for a parametrized function, + - If an argument named ``section`` is defined for a parametrized function, the current section object is passed to the function. .. code-block:: python @@ -556,28 +551,27 @@ identical to its callable parameter sibling, with the following additions: # Example # ------- # - # parametrized function example + # Parametrized function example import random from pyats import aetest - # defining a parametrized function called "number" + # Defining a parametrized function called "number" # ------------------------------------------------ - # this function accepts a lower and an upper bound, and - # uses the random.randint() api to do the actual work. - # as part of this parametrization declaration, notice that - # a lower_bound and an upper_bound was provided. these values + # This function accepts a lower and an upper bound, and + # uses the random.randint() API to do the actual work. + # As part of this parametrization declaration, notice that + # a lower_bound and an upper_bound were provided. These values # are used as function arguments when the function is evaluated @aetest.parameters.parametrize(lower_bound=1, upper_bound=100) def number(lower_bound, upper_bound): return random.randint(lower_bound, upper_bound) - - # defining a parametrized function named "expectation" + # Defining a parametrized function named "expectation" # ---------------------------------------------------- - # this is a smart function: it can decide what to return + # This is a smart function: it can decide what to return # based on the current section object information. - # it accepts the current section as input, and + # The fucntion accepts the current section as input, and # returns 999 when the section uid is 'expected_to_pass', or 0 otherwise. @aetest.parameters.parametrize def expectation(section): @@ -586,49 +580,48 @@ identical to its callable parameter sibling, with the following additions: else return 0 - # as previously stated, there's no need to add parametrized functions + # As previously stated, there's no need to add parametrized functions # to the parameters dict(). They are automatically discovered and added. - # defining two tests under this testcase + # Defining two tests under this test case # ---------------------------------------------- - # similar to callable parameters, the above parameters + # Similar to callable parameters, the above parameters # are evaluated when used as function arguments to section # the only difference is the support for parameter function arguments. class Testcase(aetest.Testcase): - # this section is expected to pass + # This section is expected to pass # the generated number is between 1 and 100, and the # expectation is 9999 (as section uid is "expected_to_pass") @aetest.test def expected_to_pass(self, number, expectation): - # test whether expectation is > than generated number + # Test whether expectation is > than generated number assert expectation > number - # this section is expected to fail + # This section is expected to fail # the generated number is still between 1 and 100, but the # expectation is 0 (as section uid is not "expected_to_pass") @aetest.test def expected_to_fail(self, number, expectation): - # test whether expectation is > than generated number + # Test whether expectation is > than generated number assert expectation > number -Essentially, parametrized functions allows users to create smart, dynamic +Essentially, parametrized functions allow users to create smart, dynamic parameter values that can vary based on the current state of execution: by leveraging the :ref:`object_model` and :ref:`parent` relationship, the use cases are endless: - - return values based on current or parent section uid/type/result + - Return values based on current or parent section uid/type/result - - return values based on a combination of parameters available to the curent + - Return values based on a combination of parameters available to the current section - - etc. + - Etc. .. warning:: - when using ``section`` argument in parametrized function, the provided - section object is the same as the internal parameter described in the next - topic below. Try not to break stuff. + When using the ``section`` argument in the parametrized function, the provided + section object is the same as the internal parameter described in the following section. **Try not to break stuff.** .. _reserved_parameters: @@ -636,9 +629,9 @@ Reserved Parameters ------------------- Reserved parameters are those that are generated by the test infrastructure -during runtime. They are not normally seen when accessing the ``parameters`` +during runtime. They are generally not seen when accessing the ``parameters`` dictionary property, but are extremely useful when you need to refer to -``aetest`` internal objects that are normally unaccessible, and are needed when +``aetest`` internal objects that are normally inaccessible and are needed when using certain ``aetest`` optional features, such as :ref:`aetest_steps`. .. csv-table:: Current Reserved Parameters @@ -660,13 +653,12 @@ using certain ``aetest`` optional features, such as :ref:`aetest_steps`. internal: parameter offering access to AEtest internal objects feature: optional feature, enabled only when parameter is used as funcargs - Reserved parameters are special: they are only accessible if their name is provided as a keyword argument to test methods (or in the case of parametrized functions, ``section`` as a function input argument). They remain hidden in all other cases. -They are *reserved*: eg, they are resolved first and takes precedence over +They are *reserved*: e.g., they are resolved first and take precedence over normal parameters. In the case where a normal parameter is created with the same name, that parameter is only accessible using the ``parameters`` property, and is not useable as a function argument. @@ -676,72 +668,73 @@ property, and is not useable as a function argument. # Example # ------- # - # accessing reserved parameters + # Accessing reserved parameters from pyats import aetest - # using CommonSetup as an example + # Using CommonSetup as an example # also applicable to other TestContainer classes class CommonSetup(aetest.CommonSetup): - # create a local parameter with the same name + # Create a local parameter with the same name # as the reserved parameter parameters = { 'steps': object(), } - # access reserved parameters by providing their + # Access reserved parameters by providing their # names as keyword arguments to methods @aetest.subsection def subsection_one(self, testscript, section, steps): - # testscript object has an attribute called module - # which is this testscript's module + # Testscript object has an attribute called module + # which is this test script's module print (testscript.module) # - # current section object is Subsection + # Current section object is Subsection # and subsections have a unique uid print(section.uid) # subsection_one - # steps object enables the usages of steps + # The steps object enables the usage of steps with steps.start('a new demo step'): pass - # reserved parameters do not show up in **kwargs + # Reserved parameters do not show up in **kwargs @aetest.subsection def subsection_two(self, **kwargs): - # only the locally defined steps parameter show up + # Only the locally defined steps parameter shows up print(kwargs) # {'steps': } - # reserved paramters takes precedence when resolved. + # Reserved parameters take precedence when resolved. @aetest.subsection def subsection_three(self, steps): - # test steps is not the same as local paramter + # Test steps are not the same as local parameter steps is not self.parameters['steps'] # True - -Reserved parameters provides ``aetest`` a mechanism to offer optional features +Reserved parameters provide ``aetest`` a mechanism to offer optional features without polluting the :ref:`object_model` with additional attributes. It also -allows users to write testscripts that delve deeper and interact with the -internals of ``aetest`` using a supported method, instead of hacking around. +allows users to write test scripts that delve deeper and interact with the +internals of ``aetest`` using a supported method instead of hacking around. - *With great power, comes great responsibilities* - use them wisely. + *With great power comes great responsibilities* - use them wisely. .. tip:: - there is no reserved parameter for the current ``TestContainer``, as the + There is no reserved parameter for the current ``TestContainer``, as the class instance is naturally provided to bound methods as the first - argument (eg, ``self``). + argument (e.g., ``self``). .. warning:: - modifying internal parameters without knowing what you're doing may result - in catastrophic failures, and/or inexplicable script behaviors. + Modifying internal parameters without knowing what you're doing may result + in catastrophic failures and inexplicable script behaviors. Monkey patching internals is strictly prohibited. Doing so will void your warranty: **no further support will be provided.** + + From 76bffeb53651ff5c985c56f4843006cd64f2dfc2 Mon Sep 17 00:00:00 2001 From: atestini Date: Fri, 22 Jul 2022 16:13:51 -0400 Subject: [PATCH 2/5] Improve readability and wording for Looping section --- docs/aetest/loop.rst | 335 +++++++++++++++++++++---------------------- 1 file changed, 165 insertions(+), 170 deletions(-) diff --git a/docs/aetest/loop.rst b/docs/aetest/loop.rst index 9f751f3..0a5ded3 100644 --- a/docs/aetest/loop.rst +++ b/docs/aetest/loop.rst @@ -12,7 +12,6 @@ Looping Sections - `Yield Expressions`_ - `Factory Design`_ - .. _Decorators: https://wiki.python.org/moin/PythonDecorators .. _Classes Tutorial: https://docs.python.org/3.4/tutorial/classes.html .. _Scopes and Namespaces: https://docs.python.org/3.4/tutorial/classes.html#python-scopes-and-namespaces @@ -22,102 +21,104 @@ Looping Sections *What are loops? Refer to the end of this section.* -As an integral extension of :ref:`test_parameters` data-driven testing concept, -``aetest`` also supports section looping: reusing section code body by providing +As an integral extension of the :ref:`test_parameters` data-driven testing concept, +``aetest`` also supports section looping. Section looping means reusing a section's code body by providing it with different parameters during each loop iteration. The following describes each section and their loop capability and behaviors: ``CommonSetup``/``CommonCleanup`` - Common setup and cleanup sections are unique within each testscript. They - are run only once per testscript execution, and are not loopable. + Common setup and cleanup sections are unique within each test script. They + are run only once per test script execution, and are not loopable. ``subsection`` Subsections within ``CommonSetup`` and ``CommonCleanup`` are loopable. - When a ``subsection`` is marked for looping, each of its iterations is + When a ``subsection`` is marked for looping, each iteration is reported as a new subsection. ``Testcase`` Testcases are loopable. Each iteration of a looping ``Testcase`` is reported - individually as new testcase instances with different ``uid``. When a - ``Testcase`` is looped, all of its contents (setup, tests and cleanup) are + individually as new test case instances with a different ``uid``. When a + ``Testcase`` is looped, and all of its contents (setup, tests, and cleanup) are run fully per each iteration. ``setup``/``cleanup`` - Setup and cleanup sections within each testcase is unique, and are run + Setup and cleanup sections within each test case is unique, and are run only once per ``Testcase``. They cannot be looped individually, but if their parent ``Testcase`` is looped, then they are run once per ``Testcase`` iteration. ``test`` Test sections within ``Testcase`` are loopable individually. Each - iteration has its own unique id, and is reported as a new test - section. When a looping ``test`` section's parent ``Testcase`` is also - looped, the resulting loops are multiplicative. Eg: if a testcase is - looped ``2x``, and contains a test that is also looped ``2x``, that - test would loop ``2x`` per testcase loop iteration. + iteration has its own unique id and is reported as a new test + section. When a looping ``test`` section's parent ``Testcase`` is + looped, the resulting loops are multiplicative. E.g.: if a test case is + looped ``2x``and contains a test that is also looped ``2x``, that + test would loop ``2x`` per test case loop iteration. .. hint:: - in other words, ``subsection``, ``Testcase`` and ``test`` sections are the + In other words, ``subsection``, ``Testcase`` and ``test`` sections are the only loopable sections. - Defining Loops -------------- -Sections are marked for looping when they are decorated with ``@loop``, and its -looping parameters provided as decorator arguments. During runtime, when +Sections are marked for looping when decorated with ``@loop``, and its +looping parameters are provided as decorator arguments. During runtime, when ``aetest`` infrastructure detects looped section code, their corresponding -section object is then instantiated once for each of its iterations. +section object is then instantiated once for each iteration. .. code-block:: python # Example # ------- # - # defining loops on sections + # Defining loops on sections from pyats import aetest - # defining a common setup section - # contains a subsection that is looped twice. + # Defining a common setup section + # The CommonSetup section contains a + # subsection that is looped twice. class CommonSetup(aetest.CommonSetup): - # defining a subsection - # this subsection is marked to be looped twice - # the first time having a uid of "subsection_one", and + # Defining a subsection + # ------------------------ + # This subsection is marked to be looped twice. + # The first time having a uid of "subsection_one", and # the second time having a uid of "subsection_two" @aetest.loop(uids=['subsection_one', 'subsection_two']) @aetest.subsection def looped_subsection(self): pass - # defining a testcase that loops - # this testcase also contains a test section that is looped twice + # Defining a test case that loops + # ---------------------------------- + # This test case also contains a test section that is looped twice @aetest.loop(uids=['testcase_one', 'testcase_two']) class Testcase(aetest.Testcase): - # setup section of this testcase is run once - # every time the testcase is looped. + # Setup section of this test case is run once + # every time the test case is looped. @aetest.setup def setup(self): pass - # looped test section - # both iterations are run per testcase iteration + # Looped test section + # both iterations are run per test case iteration @aetest.loop(uids=['test_one', 'test_two']) @aetest.test def test(self): pass - # cleanup section of this testcase is run once - # every time the testcase is looped. + # Cleanup section of this test case is run once + # every time the test case is looped. @aetest.cleanup def cleanup(self): pass - # this testscript's resulting sections would look like this below + # This test script's resulting sections would look like this below # # SECTIONS/TESTCASES RESULT # ---------------------------------------------------------------------- @@ -136,22 +137,22 @@ section object is then instantiated once for each of its iterations. # |-- test_two PASSED # `-- cleanup PASSED -As shown above, the minimum requirement to loop a section (eg, to run its code -1+ times) is to decorate the section with ``@loop``, and provide a list of loop -iteration uids using ``uids`` argument. This controls the number of iterations -this section is looped: each unique item in the ``uids`` list generates to +As shown above, the minimum requirement to loop a section (e.g., to run its code +1+ times) is to decorate the section with ``@loop``and provide a list of loop +iteration uids using the ``uids`` argument. This controls the number of iterations +this section is looped: Each unique item in the ``uids`` list generates a new section with that uid. -When ``@loop`` is used on a ``@subsection`` or ``@test``, the section method +When the ``@loop`` decorator is used on a ``@subsection`` or ``@test``, the section method is effectively decorated twice, and even though the order does not matter, it make more sense to use ``@loop`` as the outermost decorator, signifying that -this method is first marked as a section, then this section is looped. +this method is first marked as a section; then this section is looped. .. tip:: - decorators are executed in the order of "innermost" to "outermost". + Decorators are executed from "innermost" to "outermost." -In addition, in an effort to make the script more aesthetically pleasing, +Additionally, to make the script more aesthetically pleasing, ``aetest`` also features a shortcut to avoid the double decorators: ``@subsection.loop`` and ``@test.loop``. @@ -160,20 +161,20 @@ In addition, in an effort to make the script more aesthetically pleasing, # Example # ------- # - # demonstration the double decorator shortcut for test and subsections + # Demonstrating the double decorator shortcut for tests and subsections from pyats import aetest class CommonSetup(aetest.CommonSetup): - # marking this as both a subsection, and being looped + # Marking this as both a subsection and being looped @aetest.subsection.loop(uids=['subsection_one', 'subsection_two']) def looped_subsection(self): pass class Testcase(aetest.Testcase): - # marking this as both a test section and being looped + # Marking this as both a test section and being looped @aetest.test.loop(uids =['test_one', 'test_two']) def test(self): pass @@ -187,24 +188,23 @@ In addition, in an effort to make the script more aesthetically pleasing, .. tip:: - python ``@decorators`` are evaluated at import time. Thus, decorator - arguments may only be static. If you need to reference runtime and/or - dynamic information information as part of your loop declaration, eg, - accessing parameters & etc, refer to :ref:`dynamic_looping`. - + Python ``@decorators`` are evaluated at import time. Thus, decorator + arguments may only be static. If you need to reference runtime and + dynamic information information as part of your loop declaration, e.g., + accessing parameters, etc., refer to :ref:`dynamic_looping`. Loop Parameters --------------- -Looping the same section again and again is not very useful. Even if each -section has a unique uid as demonstrated above, the usefulness of a test -that repeatedly perform the same actions is questionable. This is where **loop +Looping the same section, again and again, is not very useful. Even if each +section has a unique uid, as demonstrated above, the usefulness of a test +that repeatedly performs the same actions is questionable. This is where **loop parameters** comes in. -Loop parameters feature allows each loop iteration to receive new, distinct +The loop parameters feature allows each loop iteration to receive new, distinct :ref:`test_parameters`. These parameters are specified as part of the ``@loop`` decorator, processed and propagated to each section instance as their -*local parameters*. Combined with :ref:`parameters_as_funcargs` feature, each +*local parameters*. Combined with the :ref:`parameters_as_funcargs` feature, each looped section is then driven to potentially do something different. .. code-block:: python @@ -212,25 +212,24 @@ looped section is then driven to potentially do something different. # Example # ------- # - # loop parameters demonstration + # Loop parameters demonstration from pyats import aetest - # loop this testcast with a loop parameter named "a" + # Loop this test case with a loop parameter named "a" # and set it to value 2 for the first iteration, # and 3 for the second iteration @aetest.loop(a=[2, 3]) class Testcase(aetest.Testcase): - # loop this test with loop parameter named "b" - # and set it to 8 for the first iteration, 9 for the second. + # Loop this test with a loop parameter named "b" + # and set it to 8 for the first iteration and 9 for the second. @aetest.test.loop(b=[8, 9]) def test(self, a, b): # this test prints the exponential of two inputs, a and b print("%s ^ %s = %s" % (a, b, a**b)) - - # the output of the testcase would look like this: + # The output of the test case would look like this: # 2 ^ 8 = 256 # 2 ^ 9 = 512 # 3 ^ 8 = 6561 @@ -249,22 +248,22 @@ looped section is then driven to potentially do something different. # |-- test[b=8] PASSED # `-- test[b=9] PASSED -In effect, loop parameters allows users to create and/or modify the looped -section's local parameters on the fly, per iteration. It is an extension of the -dynamic parameter concept, where section parameters are being generated and fed +In effect, loop parameters allow users to create and modify the looped +section's local parameters on the fly per iteration. It is an extension of the +dynamic parameter concept, where section parameters are generated and fed to each section during runtime. -The use of loop parameters also makes ``uids`` argument optional: if ``uids`` +The use of loop parameters also makes the ``uids`` argument optional: If the ``uids`` arguments are not provided, the infrastructure generates unique section uids by combining the original section name with each of its current loop parameters as postfix -in square backets. Otherwise, the provided ``uids`` are used as section uids. +in square brackets. Otherwise, the provided ``uids`` are used as section uids. There are two methods of providing loop parameters to the ``@loop`` decorator: - - by providing a list of parameters, and a list of parameter values for + - By providing a list of parameters, and a list of parameter values for each iteration (eg, using ``args`` and ``argvs``) - - by providing each parameter as a keyword argument, and a list of its + - By providing each parameter as a keyword argument, and a list of its corresponding argument values. (eg, ``a=[1, 2, 3], b=[4, 5, 6]``) .. code-block:: python @@ -272,22 +271,22 @@ There are two methods of providing loop parameters to the ``@loop`` decorator: # Example # ------- # - # providing loop parameters + # Providing loop parameters from pyats import aetest class Testcase(aetest.Testcase): - # loop this test with arguments "a", "b", and "c" - # provide all of its iteration arguments together using method one - # the positions of each value in argvs corresponds to its args name + # Loop this test with arguments "a", "b", and "c". + # Provide all of its iteration arguments together using method one. + # The positions of each value in argvs correspond to its args name. @aetest.test.loop(args=('a', 'b', 'c'), argvs=((1, 2, 3), (4, 5, 6))) def test_one(self, a, b, c): print("a=%s, b=%s, c=%s" % (a, b, c)) - # loop this test with the same arguments as above, but + # Loop this test with the same arguments as above, but # provide each of its iteration arguments independently using method two @aetest.test.loop(a=(1,4), b=(2,5), @@ -296,7 +295,7 @@ There are two methods of providing loop parameters to the ``@loop`` decorator: print("a=%s, b=%s, c=%s" % (a, b, c)) - # testcase output: + # Testcase output: # a=1, b=2, c=3 # a=4, b=5, c=6 # a=1, b=2, c=3 @@ -311,25 +310,25 @@ There are two methods of providing loop parameters to the ``@loop`` decorator: # |-- test_two[a=1,b=2,c=3] PASSED # `-- test_two[a=4,b=5,c=6] PASSED -As shown above, there were no difference in the outcome of the results. The only +As shown above, there were no differences in the outcome of the results. The only difference was how the loop parameters were provided. One method may be superior -to the other depending on the situation, the number of arguments, etc. +to the other depending on the use case, the number of arguments, etc. -When using loop parameters, the following rules determines the actual number +When using loop parameters, the following rules determine the actual number of iterations: - - if ``uids`` were provided, the number of iterations is equal to the number + - If ``uids`` arguments were provided, the number of iterations is equal to the number of ``uids`` provided. - - if the number of parameter values exceeds the number of ``uids``, all + - If the number of parameter values exceeds the number of ``uids``, all extra values are discarded. - - if ``uids`` are not provided, the number of iterations is equal to the + - If ``uids`` arguments are not provided, the number of iterations equals the number of loop parameter values. Eg, if ``@loop(a=[1,2,3])``, then there would be 3 loop instances, each taking on one distinct value: ``a=1``, ``a=2``, ``a=3``. - - if there are multiple parameters and the number of their values do not + - If there are multiple parameters and the number of their values do not agree, or if the number of parameter values is less than the number of provided ``uids``, a ``filler`` is used to fill empty spots. ``filler`` defaults to ``None``, and only 1 filler can be provided. @@ -339,26 +338,26 @@ of iterations: # Example # ------- # - # loop parameter combinations - # (pseudo code for demonstration only) + # Loop parameter combinations + # (pseudo-code for demonstration only) from pyats.aetest import loop - # loop with 2 iterations using uids argument + # Loop with 2 iterations using uids argument # ------------------------------------------ # iteration 1: uid='id_one' # iteration 2: uid='id_two' @loop(uids=['id_one', 'id_two']) - # loop with 2 iterations using parameters argument + # Loop with 2 iterations using parameters argument # ------------------------------------------------ # iteration 1: a=1, b=4 # iteration 2: a=2, b=5 @loop(a = [1, 2], b = [4, 5]) - # same as above, using args and argvs + # Same as above, using args and argvs @loop(args=['a', 'b'], argvs=[(1, 4), (2, 5)]) - # loop with 2 iterations, and extra arguments are discarded due to uids + # Loop with 2 iterations, and extra arguments are discarded due to uids # --------------------------------------------------------------------- # iteration 1: uid='id_one', a=1, b=2 # iteration 2: uid='id_two', a=3, b=4 @@ -368,23 +367,23 @@ of iterations: argvs=[(1, 2), (3, 4), (5, 6)]) - # same example as above but using per-parameter values + # Same example as above but using per-parameter values @loop(uids=['id_one', 'id_two'], a=[1, 3, 5], b=[2, 4, 6]) - # loop with 3 iterations, and their number of parameters values do not agree + # Loop with 3 iterations, and their number of parameters values do not agree # -------------------------------------------------------------------------- # iteration 1: a=1, b=4 # iteration 2: a=2, b=5 # iteration 3: a=3, b=None ---> default filler comes in to fill the blanks @loop(a=[1, 2, 3], b=[4, 5]) - # same as above, using args and argvs + # Same as above, using args and argvs @loop(args=['a', 'b'], argvs=[(1, 4), (2, 5), (3, )]) - # loop with more uids than parameters, and custom filler + # Loop with more uids than parameters, and custom filler # ------------------------------------------------------ # iteration 1: uid='id_one', a=1, b=3 # iteration 2: uid='id_two', a=2, b=4 @@ -399,13 +398,13 @@ Advanced Loop Usages -------------------- Arguments to the ``@loop`` decorator may also be `callable`_, `iterable`_, or a -`generator`_. The infrastructure is able to distinguish and treat each as you +`generator`_. The infrastructure can distinguish and treat each as you would normally expect it to: - - if an argument value is a `callable`_, it is called, and its returns + - If an argument value is a `callable`_, it is called, and its returns are then used as the actual loop argument value. - - if an argument value is an `iterable`_ or a `generator`_, the loop engine + - If an argument value is an `iterable`_ or a `generator`_, the loop engine picks only one element from it at a time to build the next iteration, until it is exhausted. @@ -418,18 +417,18 @@ would normally expect it to: # Example # ------- # - # demonstrating advanced loop parameter behaviors + # Demonstrating advanced loop parameter behaviors from pyats import aetest - # defining a function - # functions are callable + # Defining a function + # Functions are callable def my_function(): value = [1, 2, 3] print("returning %s" % value) return value - # defining a generator + # Defining a generator def my_generator(): for i in [4, 5, 6]: print('generating %s' % i) @@ -437,20 +436,20 @@ would normally expect it to: class Testcase(aetest.Testcase): - # creating test section with parameter "a" as a function - # note that the function object is passed, not its values + # Creating test section with parameter "a" as a function + # Note that the function object is passed, not its values @aetest.test.loop(a=my_function) def test_one(self, a): print("a = %s" % a) - # creating a test section with parameter "b" as a generator - # note that the generator is a result of calling my_generator(), not + # Creating a test section with parameter "b" as a generator + # Note that the generator is a result of calling my_generator(), not # the function itself. @aetest.test.loop(b=my_generator()) def test_two(self, b): print('b = %s' % b) - # the output of the testcase would be: + # The output of the test case would be: # returning [1, 2, 3] # a = 1 # a = 2 @@ -470,31 +469,30 @@ In the above example, pay close attention to the output lines: - Iterators and generators are only queried before the next section needs to be created. -This behavior enables the use of custom generator as input values to your loop +This behavior enables using a custom generator as input values to your loop parameters. For example, a generator state machine that queries the current testbed device status and creates iterations based on that information. Since the generator is not polled until right before the next iteration, your custom function is only run in-between test sections, thus dynamically generating the -loop iterations based current test environments. - +loop iterations based on current test environments. .. _dynamic_looping: Dynamic Loop Marking -------------------- -So far, all loop examples focused on defining ``@loop`` directly within the -testscripts. Eg, the ``@loop`` decorators are coded as part of the testscript. -In addition, it is also possible to dynamically mark sections for looping during -runtime, eg, creating loops based on information that is only available during -a script run. To do this, use the ``loop.mark()`` function. +So far, all loop examples focus on defining the ``@loop`` decorator directly within the +test scripts. E.g., the ``@loop`` decorators are coded as part of the test script. +However, it is also possible to dynamically mark sections for looping during +runtime, e.g., creating loops based on information that is only available during +a script’s run. To do this, use the ``loop.mark()`` function. .. code-block:: python # Example # ------- # - # dynamically marking sections for looping + # Dynamically marking sections for looping from pyats import aetest @@ -502,23 +500,22 @@ a script run. To do this, use the ``loop.mark()`` function. @aetest.setup def setup(self): - # mark the next test for looping - # provide it with two unique test uids. + # Mark the next test for looping + # Provide it with two unique test uids. # (self.simple_test is the next test method) aetest.loop.mark(self.simple_test, uids=['test_one', 'test_two']) - # note: the simple_test section is not directly marked for looping + # Note: the simple_test section is not directly marked for looping # instead, during runtime, its testcase's setup section marks it for # looping dynamically. @aetest.test def simple_test(self, section): - # print the current section uid + # Print the current section uid # by using the internal parameter "section" print("current section: %s" % section.uid) - - # output of this testcase + # Output of this test case # current section: test_one # current section: test_two # @@ -530,20 +527,19 @@ a script run. To do this, use the ``loop.mark()`` function. # |-- test_one PASSED # `-- test_two PASSED -``loop.mark()`` arguments & behaviors (including loop parameters & etc) are -exactly identical to its sibling ``@loop`` decorator, with the only exception -that its first input argument must be the target section method/class. Eg: +``loop.mark()`` arguments and behaviors (including loop parameters, etc.) are +exactly identical to its sibling, the ``@loop`` decorator, with the only exception +that its first input argument must be the target section method/class. E.g.: ``loop.mark(Testcase_Two, a=[1,2,3])``. -The benefit of this approach is simple: dynamic information, parameters and -variables such as :ref:`script_args`, :ref:`parent` etc, are only available +The benefit of this approach is simple: Dynamic information, parameters and +variables such as :ref:`script_args`, :ref:`parent` etc., are only available during runtime. This information and its corresponding variables are not available when the script is written, and delaying variable references (while -using ``@loop`` decorator) in Python is very difficult, if not impossible. +using the ``@loop`` decorator) in Python is very difficult, if not impossible. -------------------------------------------------------------------------------- - Loop Internals -------------- @@ -552,61 +548,60 @@ Loop Internals The information here onwards is for users interested in ``aetest`` internals & extensions only. - If you are new to this, do not read on. These advanced topics may - further fuel your confusion. + **If you are new to this, do not read on. These advanced topics may + further fuel your confusion.** The previous sections focused on the "how to use" aspect of ``aetest`` looping functionality. From here onwards, we'll dig deeper into loop internals, look at how it functions, and how to deviate from its default behaviors. -The ``aetest`` looping behavior & how its arguments are processed is actually +The ``aetest`` looping behavior and how its arguments are processed are highly customizable. This was not highlighted in previous sections for the sake of serializing the training & simplifying the learning curve. -In reality, consider ``@loop`` decorator and ``loop.mark()`` function as only -markers: they only mark the given section for looping. The details (parameters) -of each iterations is actually generated from **loop generators**, where all -arguments to ``@loop`` and ``loop.mark()`` propagates to. Eg: +In reality, consider the ``@loop`` decorator and ``loop.mark()`` function as only +markers: They only mark the given section for looping. The details (parameters) +Each iteration is generated from **loop generators**, where all +arguments to ``@loop`` and ``loop.mark()``propagate to. E.g.: .. code-block:: python # Example # ------- # - # pseudo code demonstrating @loop decorator functionality + # Pseudo-code demonstrating @loop decorator functionality - # what the loop decorator definition sort-of looks like - # note where the generator defaults to "DefaultLooper" + # What the loop decorator definition sort of looks like + # Note where the generator defaults to "DefaultLooper" def loop(generator=DefaultLooper, *args, **kwargs): - # the actual loop generator gets called with all of the arguments + # The actual loop generator gets called with all of the arguments # to loop decorator, and generates each section iteration return generator(*args, **kwargs) - # pseudo code here onwards, demonstrating internals + # Pseudo-code here onwards, demonstrating internals # ------------------------------------------------- # - # during runtime, the looped is expanded to create each iteration + # During runtime, the looped is expanded to create each iteration for iteration in loop(*args, **kwargs): - # create a section from iteration information and run it + # Create a section from iteration information and run it # ... - # eg, instantiate Subsection + #E.g., instantiate Subsection subsection = Subsection(uid=iteration.uid, parameters=iteration.parameters) # and add to common setup's subsections list common_setup.subsections.append(subsection) - # etc ... - + # etc. -Behind the scenes, **loop generators** are the actual classes that does the -heavy lifting: creating each iteration based on ``@loop`` and ``loop.mark()`` +Behind the scenes, **loop generators** are the actual classes that do the +heavy lifting: Creating each iteration based on the ``@loop`` and ``loop.mark()`` decorator arguments. Loop generators are `iterable`_. Each of its returned -member is an instance of ``Iteration`` class, containing the uid & parameters -information unique to this loop, and used by the infrastructure to create the +members is an instance of the ``Iteration`` class, containing the uid & parameters +information unique to this loop and used by the infrastructure to create the next section instance. .. csv-table:: Iteration class (collections.namedtuple) @@ -619,61 +614,61 @@ next section instance. the next looped section" In other words, **loop generator** is the object that ultimately controls how -loops are generated, and what parameters each iteration is associated with. The -looping behavior and arguments described in topics above are actually that of +loops are generated and what parameters each iteration is associated with. The +looping behavior and arguments described in the topics above are that of ``DefaultLooper``, the default **loop generator** provided by ``aetest`` loop infrastructure. Its features are sufficient for most use cases. However, if you -wish to customize loop behavior, it is possible to extend and/or override it. +wish to customize loop behavior, it is possible to extend and override it. .. code-block:: python # Example # ------- # - # demonstrating how to write and pass your own loop generator + # Demonstrating how to write and pass your own loop generator - # loop generators must return Iterations + # Loop generators must return Iterations from pyats import aetest from pyats.aetest.loop import Iteration - # let's write a custom loop generator - # it generates integers between a and b as loop iterations - # and pass the integer as "number" parameter of the executed section. - # each iteration uid is named "iteration_uid" + number + # Let's write a custom loop generator + # It generates integers between a and b as loop iterations + # and pass the integer as the "number" parameter of the executed section. + # Each iteration uid is named "iteration_uid" + number class DemoGenerator(object): - # at minimum, the loop generator needs to accept an argument called + # At a minimum, the loop generator needs to accept an argument called # "loopee", which is the actual object being looped. This allows the - # loop generator to know what it is looping on, and build information + # loop generator to know what it is looping on and build information # based on it. - # in this example, we're ignoring that argument, as our loop genrator - # is simple and straightforward. + # In this example, we're ignoring that argument, as our loop generator + # is straightforward. def __init__(self, loopee, a, b): self.numbers = list(range(a, b)) def __iter__(self): for i in self.numbers: - # each generated member is an instance of Iteration - # each Iteration must have a unique id + # Each generated member is an instance of Iteration + # Each Iteration must have a unique id # and all of its parameters stored in a dictionary yield Iteration(uid='iteration_uid_%s' % i, parameters={'number': i}) - # this loop generator can be used as @loop and loop.mark() argument. - # let's define a looped testcase with it. + # This loop generator can be used as the @loop and loop.mark() argument. + # Let's define a looped test case with it. - # looping this testcase with custom generator, and a=1, b=5 + # Looping this test case with a custom generator, and a=1, b=5 @aetest.loop(generator=DemoGenerator, a=1, b=5) class Testcase(aetest.Testcase): - # since our generator generates a parameter named "number" - # let's print it in this simple test. + # Since our generator generates a parameter named "number" + # Let's print it in this simple test. @aetest.test def test(self, number): print('current number: %s' % number) - # output of this testcase + # Output of this test case # current number: 1 # current number: 2 # current number: 3 @@ -693,14 +688,14 @@ wish to customize loop behavior, it is possible to extend and/or override it. .. hint:: - the above examples may be simple, the demonstrated underlying principles + The above examples may be simple, but the demonstrated underlying principles are not. *"Do not try and bend the spoon. That's impossible. Instead... only try to realize the truth..."* -And voila. Custom **loop generators** like above is immensely powerful: by -extending and/or overriding the default loop generation behavior, and defining +And voila. Custom **loop generators** like the above are immensely powerful: By +extending and overriding the default loop generation behavior, and defining custom test sections entirely driven by parameter inputs, users can effectively overload the loop functionality into a dynamic generator of highly abstracted test executor. From e1a31ce9cb31fd311649db6e2e22877ad3bc3141 Mon Sep 17 00:00:00 2001 From: atestini Date: Fri, 22 Jul 2022 14:53:40 -0400 Subject: [PATCH 3/5] Improve readibility and wording --- docs/aetest/parameters.rst | 277 ++++++++++++++++++------------------- 1 file changed, 135 insertions(+), 142 deletions(-) diff --git a/docs/aetest/parameters.rst b/docs/aetest/parameters.rst index a5b93e8..96a52b8 100644 --- a/docs/aetest/parameters.rst +++ b/docs/aetest/parameters.rst @@ -7,27 +7,27 @@ Test Parameters - `Function Arguments`_ - `Data-Driven Programming`_ - - `Mutable/Immuntable Objects`_ + - `Mutable/Immutable Objects`_ - `Variable Scoping`_ .. _Data-Driven Programming: http://en.wikipedia.org/wiki/Data-driven_programming .. _Function Arguments: https://docs.python.org/3.4/tutorial/controlflow.html#more-on-defining-functions -.. _Mutable/Immuntable Objects: http://en.wikibooks.org/wiki/Python_Programming/Data_Types#Mutable_vs_Immutable_Objects +.. _Mutable/Immutable Objects: http://en.wikibooks.org/wiki/Python_Programming/Data_Types#Mutable_vs_Immutable_Objects .. _Variable Scoping: https://docs.python.org/3.4/reference/executionmodel.html -``aetest`` is a data-driven test infrastructure. Its scripts and testcases are +``aetest`` is a data-driven test infrastructure. Its scripts and test cases are intended to be driven dynamically by: -- data in the form of input arguments to the testscript -- dynamic data generated during runtime +- Data in the form of input arguments to the test script +- Dynamic data generated during runtime The collection of dynamic data that artificially affects the behavior of -testscripts and testcases in ``aetest`` is called **parameters**. It adheres to -a pre-defined set of parent/child propagation relationships, and may be used as +test scripts and test cases in ``aetest`` is called **parameters**. It adheres to +a pre-defined set of parent/child propagation relationships and may be used as `Function Arguments`_ to each test section. -This feature is a supplement to static testcase data (attribute values stored -within each testcases). +This feature is a supplement to static test case data (attribute values stored +within each test case). .. tip:: @@ -43,10 +43,10 @@ Before delving deeper into the concept and mechanism of **parameters**, let's first spend some time on what it is, why they are needed, and how they may benefit the end user. -Parameters are a special kind of variable, used within functions and methods to -refer and access its input data (arguments). If you consider the function and -methods as the "doer", then parameters are the "what to do with". Eg, an -``add`` function would require 2 or more parameters to be "added together". +Parameters are a special kind of variable used within functions and methods to +refer to and access its input data (arguments). If you consider the function and +methods as the "doer," then parameters are the "what to do with." E.g., an +``add`` function would require two or more parameters to be "added together". .. code-block:: python @@ -74,33 +74,33 @@ methods as the "doer", then parameters are the "what to do with". Eg, an add_to_c(1, 2) # 103 -In a data-driven testing, testscripts are the *doers*, performing the act -of testing a facet of some product. Its arguments and parameters are thus the -input data that influences the specific acts of testing being carried out. Here +In data-driven testing, test scripts are the *doers*, performing the act +of testing a facet of some product. Test script arguments and parameters are thus the +input data that influences the specific actions of testing. Here are some possible use cases: - - ``testbed`` argument to a script tells it which testbed to run on, and + - The ``testbed`` argument to a script tells it which testbed to run on, and what the topology is like. The script can then connect to the testbed, and decide how to perform the testing best suited for this topology. - - ``vlan`` argument to ``layer2_traffic`` script can dynamically control the - vlan to be be configured for traffic testing. + - The ``vlan`` argument to the ``layer2_traffic`` script can dynamically control the + VLAN to be configured for traffic testing. - - other toggle arguments that dynamically turns on/off certain testcases, - and/or combination of features to be configured & tested + - Other toggle arguments that dynamically turn on/off certain test cases, + and a combination of features to be configured & tested - - etc. + - Etc. Of course, the parameters feature in ``aetest`` is much more than just script -arguments. It enables users to write testcases and testscripts that are capable +arguments. It enables users to write test cases and test scripts that are capable of being driven by inputs, varying the degree of testing, etc. Relationship Model ------------------ In ``aetest``, parameters are **relative**: parameters corresponding to each -object is the combination of its local specific parameters, and all of its -parent object's parameters. Eg: +object are the combination of its local specific parameters and all of its +parent object's parameters. E.g.: - ``Testcase`` parameters = local parameters + ``TestScript`` parameters @@ -113,9 +113,9 @@ runtime behaviors section: each object has a parent, and the parameters seen at each object level is an aggregation of its and all of its parent's parameters. -In case when an object and its parent (or its parent's parent, etc) have the same +In case when an object and its parent (or its parent's parent, etc.) have the same parameter names defined, then the parameter value closest to the current scope -is used/prefered. +is used/preferred. .. figure:: parameter_relations.png :align: center @@ -123,7 +123,7 @@ is used/prefered. *Parameter Relationship Model In a Nutshell* Below is a behavior demonstration of this relationship model. In actual script -execution, this happens behind-the-scenes automatically. +execution, this happens behind the scenes automatically. .. code-block:: python @@ -153,23 +153,23 @@ execution, this happens behind-the-scenes automatically. } # during runtime, the combined parameters seen at the - # testcase level, would be equivalent to the following: - # - take the testscript parameters as basis + # testcase level would be equivalent to the following: + # - take the testscript parameters as the basis # - and add to it, testcase parameters # - # eg: + #, e.g.: new_testcase_parameters = testscript.parameters.copy() new_testcase_parameters.update(testcase.parameters) testcase.parameters = new_testcase_parameters - # so that the new parameters seen at the testcase + # so that the new parameters seen in the test case # level, is: testcase.parameters # {'param_A': 100, 'param_B': 2, 'param_C': 3} .. hint:: - in other words, childs inherits but shadows parent parameters. This is + In other words, children inherit but shadow parent parameters. This is similar to Python `Variable Scoping`_ concept. Parameters Property @@ -178,7 +178,7 @@ Parameters Property Every top-level object in ``aetest`` comes with the special ``parameters`` property: a dictionary containing the key/value data pairs relative to this object (:ref:`object_model`). Its default values can be set/updated by the user -within the testscript. +within the test script. .. code-block:: python @@ -200,10 +200,10 @@ within the testscript. } # using Testcase to demonstrate TestContainer-type parameters definitions - # note that this also is applicable to CommonSetup & CommonCleanup + # note that this also applies to CommonSetup & CommonCleanup class Testcase(aetest.Testcase): - # all default parameters specific to this testcase is declared + # all default parameters specific to this test case are declared # in its own parameters dictionary. parameters = { 'generic_param_A': 200 @@ -212,7 +212,7 @@ within the testscript. # etc ... During runtime, these dictionaries form the baseline ``parameters`` properties -of their corresponding section. Eg: +of their corresponding section. E.g.: - script-level ``parameters`` dictionary is used to create ``TestScript`` object parameters. @@ -221,7 +221,7 @@ of their corresponding section. Eg: One exception to the above is method local parameters for sections such as ``subsection``, ``setup``, ``test`` and ``cleanup``. Even though their -corresponding classes (``Subection``, ``SetupSection``, ``TestSection``, +corresponding classes (``Subsection``, ``SetupSection``, ``TestSection``, ``CleanupSection``) also have the parameters property, these class instances only exists briefly during runtime (see :ref:`aetest_function_classes`), so their attributes are mostly only dynamic in nature, set & controlled by the @@ -235,9 +235,9 @@ dynamically access & update parameters. .. important:: - even though parameters seen at each object level also includes its parent's + Even though parameters seen at each object level also include its parent's parameters, setting & updating the parameters dictionary is only reflected - locally, and does not propagate to the parent. This is also inline with how + locally, and does not propagate to the parent. This is also in line with how Python `Variable Scoping`_ works. .. code-block:: python @@ -247,7 +247,7 @@ dynamically access & update parameters. # # continuing from the above - # re-defining the testcase for the sake of code-continuity + # re-defining the test case for the sake of code-continuity class Testcase(aetest.Testcase): # local parameters defaults, same as above @@ -258,7 +258,7 @@ dynamically access & update parameters. # within any sections, the parent container parameters are directly # accessible (applicable to setup/test/cleanup and subsections) - # here we'll do a combination access & updating of parameters + # here, we'll do a combination of access & updating of parameters @aetest.setup def setup(self): # add to the parameters dict @@ -272,7 +272,7 @@ dynamically access & update parameters. @aetest.test def test(self): - # access & print parent testscript parameters + # access & print parent test script parameters # (following the parent model) print(self.parent.parameters) # {'generic_param_A': 100, @@ -288,12 +288,11 @@ dynamically access & update parameters. # 'testscript_param_B': [], # 'testscript_param_A': 'another value'} - -Consider the above example: parameters can be set and acccessed as the script -runs, opening up the opportunity for scripts to dynamically discover the runtime -environment and modify test behavior (parameters) as required. Eg, ``setup`` -section of modifying testcase parameters based on current testbed state, and -altering the behavior of ensuiting ``test`` sections, etc. +Consider the above example: parameters can be set and accessed as the script +runs, opening the opportunity for scripts to dynamically discover the runtime +environment and modify test behavior (parameters) as required. E.g., the ``setup`` +section of modifying test case parameters based on the current testbed state, and +altering the behavior of ensuing ``test`` sections, etc. .. tip:: @@ -302,13 +301,12 @@ altering the behavior of ensuiting ``test`` sections, etc. .. _Collections.ChainMap: https://docs.python.org/3/library/collections.html#collections.ChainMap - .. _script_args: Script Arguments ---------------- -In short, any arguments passed to the testscript before startup becomes part of +In short, any arguments passed to the test script before startup becomes part of the ``TestScript`` parameter. This includes all the arguments passed through the :ref:`easypy_jobfile` during :ref:`aetest_jobfile_execution`, and/or any command line arguments parsed and passed to ``aetest.main()`` during @@ -320,12 +318,12 @@ line arguments parsed and passed to ``aetest.main()`` during # ------- # # script parameters and how it works - # (pseudo code, for demonstration only) + # (pseudo-code, for demonstration only) - # without going into details of how script parameters/arguments are + # without going into details about how script parameters/arguments are # passed in (covered under Running Scripts section) - # assuming that the testscript was called with the following inputs + # assuming that the test script was called with the following inputs script_arguments = { 'arg_a': 100, 'arg_c': 3, @@ -347,15 +345,15 @@ line arguments parsed and passed to ``aetest.main()`` during # 'arg_b': 2, # 'arg_c': 3} -As demonstrated in the above example, script arguments/parameters are actually -added on top of ``TestScript`` parameters defined within the script. In +As demonstrated in the above example, script arguments/parameters are +added on top of the ``TestScript`` parameters defined within the script. In other words, script arguments are added dynamically to the current running -script's base parameters, and overwrites any existing ones. +script's base parameters and overwrite existing ones. .. tip:: - define your default parameters in the script, and change the behavior of - the testscript by overwriting specific ones using script arguments. + Define your default parameters in the script, and change the behavior of + the test script by overwriting specific ones using script arguments. .. _parameters_as_funcargs: @@ -364,15 +362,15 @@ Parameters as Function Arguments ``parameters`` property & functionality provides a means for objects within ``aetest`` to follow the :ref:`parent` model and aggregate data together in a -clean, accessible format. It serves also as the basis for providing section -methods their `Function Arguments`_. This is the main mechanism behind the +clean, accessible format. It also serves as the basis for providing section +methods their `Function Arguments`_. This is the primary mechanism behind the data-driven concept of ``aetest``: function/methods are **driven** by input parameters. During runtime, all section method function arguments are filled by their corresponding parameter value, matching the argument name vs. the parameter key/name. The following table describes the types of function arguments and -their supports. +their support. .. csv-table:: Function Argument Types & Parameters Support :header: "Type", "Example", "Comments" @@ -404,9 +402,9 @@ their supports. class Testcase(aetest.Testcase): - # this setup section definition identifies "param_B" as - # as a input requirement. as this parameter is available at this - # testcase level (aggregated from parent testscript), it + # this setup section definition identifies "param_B" + # as an input requirement. As this parameter is available at this + # test case level (aggregated from the parent test script), it # is passed in as input @aetest.setup def setup(self, param_B): @@ -442,32 +440,31 @@ their supports. This is the preferred method of accessing parameters: by passing each in explicitly as function arguments. It is more pythonic: - - explictly passing in parameters makes the code (and its dependencies) + - Explicitly passing in parameters makes the code (and its dependencies) easier to read and understand - - allows the infrastructure to handle error scenarios (such as missing + - Allows the infrastructure to handle error scenarios (such as missing parameters) - - allows users to easily define defaults (without dealing with dictionary + - Allows users to easily define defaults (without dealing with dictionary operations) - - maintaining the ability to call each section as a function with various + - Maintaining the ability to call each section as a function with various arguments during test/debugging situations. - - etc... + - Etc. .. tip:: - once a parameter is passed into a section as a function argument, it becomes + Once a parameter is passed into a section as a function argument, it becomes a local variable. All rules of `Variable Scoping`_ apply. - Callable Parameters ------------------- -A callable parameter is a one that evaluates to ``True`` using callable_. When +A callable parameter is one that evaluates to ``True`` using callable_. When a callable parameter is filled as a function argument to test sections, the -infrastructure "calls" it, and uses its return value as the actual argument +infrastructure "calls" it and uses its return value as the actual argument parameter. .. code-block:: python @@ -490,65 +487,63 @@ parameter. class Testcase(aetest.Testcase): - # as the "number" parameter's value is the callable function + # As the "number" parameter's value is the callable function # random.random, this function is evaluated right before the # execution of this test method, and the call result is then used # as the actual argument input @aetest.test def test(self, number): - # test whether the generated number is greater than 0.5 + # Test whether the generated number is greater than 0.5 assert number > 0.5 - # if you run this test enough times + # If you run this test enough times # you will find that it passes exactly 50% of the time # assuming that random.random() generate truly random numbers :) - # note that callable parameters are only evaluated if used - # as function arguments. they are still objects if viewed - # through the parameters property + # Note that callable parameters are only evaluated if used + # as function arguments. Callable parameters are still objects + # if viewed through the parameters property self.parameters['number'] # - -Callable parameters still shows up as their original function object when -accessed through ``parameters`` property. They are only evaluated (called) when +Callable parameters still appear as their original function object when +accessed through the ``parameters`` property. They are only evaluated (called) when used as function arguments to test methods. This evaluation occurs "on demand": - - the evalution takes place right before method execution. + - The evaluation takes place right before method execution. - - each method gets its own indepedent evaluated result. + - Each method gets its independent evaluated result. The only limitation with callable parameters is that they cannot have arguments. ``aetest`` would not know how to fulfill them during runtime. .. tip:: - eliminate function arguments (by pre-setting them) with + Eliminate function arguments (by pre-setting them) with `partial functions`_. .. _callable: https://docs.python.org/3.4/library/functions.html#callable .. _partial functions: https://docs.python.org/3.4/library/functools.html#functools.partial - Parametrizing Functions ----------------------- Parametrized functions are a special breed of "smart" callable parameters. They -support arguments, are capable of identifying the current execution context, and -act accordingly. +support arguments, can identify the current execution context, and +can act accordingly. A parametrized function is declared when the ``@parameters.parametrize`` -decorator is used on a function in your testscript. This also adds the newly +decorator is used on a function within a test script. This also adds the newly created parametrized function automatically as part of ``TestScript`` parameters, using the function name as the parameter name. -During runtime, the behavior of these parametrized functions is exactly +During runtime, the behavior of these parametrized functions are identical to its callable parameter sibling, with the following additions: - - any arguments to the ``@parameters.parametrize`` decorator are stored + - Any arguments to the ``@parameters.parametrize`` decorator are stored and used as function arguments during the evaluation. - - if an argument named ``section`` is defined for a parametrized function, + - If an argument named ``section`` is defined for a parametrized function, the current section object is passed to the function. .. code-block:: python @@ -556,28 +551,27 @@ identical to its callable parameter sibling, with the following additions: # Example # ------- # - # parametrized function example + # Parametrized function example import random from pyats import aetest - # defining a parametrized function called "number" + # Defining a parametrized function called "number" # ------------------------------------------------ - # this function accepts a lower and an upper bound, and - # uses the random.randint() api to do the actual work. - # as part of this parametrization declaration, notice that - # a lower_bound and an upper_bound was provided. these values + # This function accepts a lower and an upper bound, and + # uses the random.randint() API to do the actual work. + # As part of this parametrization declaration, notice that + # a lower_bound and an upper_bound were provided. These values # are used as function arguments when the function is evaluated @aetest.parameters.parametrize(lower_bound=1, upper_bound=100) def number(lower_bound, upper_bound): return random.randint(lower_bound, upper_bound) - - # defining a parametrized function named "expectation" + # Defining a parametrized function named "expectation" # ---------------------------------------------------- - # this is a smart function: it can decide what to return + # This is a smart function: it can decide what to return # based on the current section object information. - # it accepts the current section as input, and + # The fucntion accepts the current section as input, and # returns 999 when the section uid is 'expected_to_pass', or 0 otherwise. @aetest.parameters.parametrize def expectation(section): @@ -586,49 +580,48 @@ identical to its callable parameter sibling, with the following additions: else return 0 - # as previously stated, there's no need to add parametrized functions + # As previously stated, there's no need to add parametrized functions # to the parameters dict(). They are automatically discovered and added. - # defining two tests under this testcase + # Defining two tests under this test case # ---------------------------------------------- - # similar to callable parameters, the above parameters + # Similar to callable parameters, the above parameters # are evaluated when used as function arguments to section # the only difference is the support for parameter function arguments. class Testcase(aetest.Testcase): - # this section is expected to pass + # This section is expected to pass # the generated number is between 1 and 100, and the # expectation is 9999 (as section uid is "expected_to_pass") @aetest.test def expected_to_pass(self, number, expectation): - # test whether expectation is > than generated number + # Test whether expectation is > than generated number assert expectation > number - # this section is expected to fail + # This section is expected to fail # the generated number is still between 1 and 100, but the # expectation is 0 (as section uid is not "expected_to_pass") @aetest.test def expected_to_fail(self, number, expectation): - # test whether expectation is > than generated number + # Test whether expectation is > than generated number assert expectation > number -Essentially, parametrized functions allows users to create smart, dynamic +Essentially, parametrized functions allow users to create smart, dynamic parameter values that can vary based on the current state of execution: by leveraging the :ref:`object_model` and :ref:`parent` relationship, the use cases are endless: - - return values based on current or parent section uid/type/result + - Return values based on current or parent section uid/type/result - - return values based on a combination of parameters available to the curent + - Return values based on a combination of parameters available to the current section - - etc. + - Etc. .. warning:: - when using ``section`` argument in parametrized function, the provided - section object is the same as the internal parameter described in the next - topic below. Try not to break stuff. + When using the ``section`` argument in the parametrized function, the provided + section object is the same as the internal parameter described in the following section. **Try not to break stuff.** .. _reserved_parameters: @@ -636,9 +629,9 @@ Reserved Parameters ------------------- Reserved parameters are those that are generated by the test infrastructure -during runtime. They are not normally seen when accessing the ``parameters`` +during runtime. They are generally not seen when accessing the ``parameters`` dictionary property, but are extremely useful when you need to refer to -``aetest`` internal objects that are normally unaccessible, and are needed when +``aetest`` internal objects that are normally inaccessible and are needed when using certain ``aetest`` optional features, such as :ref:`aetest_steps`. .. csv-table:: Current Reserved Parameters @@ -660,13 +653,12 @@ using certain ``aetest`` optional features, such as :ref:`aetest_steps`. internal: parameter offering access to AEtest internal objects feature: optional feature, enabled only when parameter is used as funcargs - Reserved parameters are special: they are only accessible if their name is provided as a keyword argument to test methods (or in the case of parametrized functions, ``section`` as a function input argument). They remain hidden in all other cases. -They are *reserved*: eg, they are resolved first and takes precedence over +They are *reserved*: e.g., they are resolved first and take precedence over normal parameters. In the case where a normal parameter is created with the same name, that parameter is only accessible using the ``parameters`` property, and is not useable as a function argument. @@ -676,72 +668,73 @@ property, and is not useable as a function argument. # Example # ------- # - # accessing reserved parameters + # Accessing reserved parameters from pyats import aetest - # using CommonSetup as an example + # Using CommonSetup as an example # also applicable to other TestContainer classes class CommonSetup(aetest.CommonSetup): - # create a local parameter with the same name + # Create a local parameter with the same name # as the reserved parameter parameters = { 'steps': object(), } - # access reserved parameters by providing their + # Access reserved parameters by providing their # names as keyword arguments to methods @aetest.subsection def subsection_one(self, testscript, section, steps): - # testscript object has an attribute called module - # which is this testscript's module + # Testscript object has an attribute called module + # which is this test script's module print (testscript.module) # - # current section object is Subsection + # Current section object is Subsection # and subsections have a unique uid print(section.uid) # subsection_one - # steps object enables the usages of steps + # The steps object enables the usage of steps with steps.start('a new demo step'): pass - # reserved parameters do not show up in **kwargs + # Reserved parameters do not show up in **kwargs @aetest.subsection def subsection_two(self, **kwargs): - # only the locally defined steps parameter show up + # Only the locally defined steps parameter shows up print(kwargs) # {'steps': } - # reserved paramters takes precedence when resolved. + # Reserved parameters take precedence when resolved. @aetest.subsection def subsection_three(self, steps): - # test steps is not the same as local paramter + # Test steps are not the same as local parameter steps is not self.parameters['steps'] # True - -Reserved parameters provides ``aetest`` a mechanism to offer optional features +Reserved parameters provide ``aetest`` a mechanism to offer optional features without polluting the :ref:`object_model` with additional attributes. It also -allows users to write testscripts that delve deeper and interact with the -internals of ``aetest`` using a supported method, instead of hacking around. +allows users to write test scripts that delve deeper and interact with the +internals of ``aetest`` using a supported method instead of hacking around. - *With great power, comes great responsibilities* - use them wisely. + *With great power comes great responsibilities* - use them wisely. .. tip:: - there is no reserved parameter for the current ``TestContainer``, as the + There is no reserved parameter for the current ``TestContainer``, as the class instance is naturally provided to bound methods as the first - argument (eg, ``self``). + argument (e.g., ``self``). .. warning:: - modifying internal parameters without knowing what you're doing may result - in catastrophic failures, and/or inexplicable script behaviors. + Modifying internal parameters without knowing what you're doing may result + in catastrophic failures and inexplicable script behaviors. Monkey patching internals is strictly prohibited. Doing so will void your warranty: **no further support will be provided.** + + From c1460a319e140f7c67854d59e8cbd738dec105df Mon Sep 17 00:00:00 2001 From: atestini Date: Fri, 22 Jul 2022 16:13:51 -0400 Subject: [PATCH 4/5] Improve readability and wording for Looping section --- docs/aetest/loop.rst | 335 +++++++++++++++++++++---------------------- 1 file changed, 165 insertions(+), 170 deletions(-) diff --git a/docs/aetest/loop.rst b/docs/aetest/loop.rst index 9f751f3..0a5ded3 100644 --- a/docs/aetest/loop.rst +++ b/docs/aetest/loop.rst @@ -12,7 +12,6 @@ Looping Sections - `Yield Expressions`_ - `Factory Design`_ - .. _Decorators: https://wiki.python.org/moin/PythonDecorators .. _Classes Tutorial: https://docs.python.org/3.4/tutorial/classes.html .. _Scopes and Namespaces: https://docs.python.org/3.4/tutorial/classes.html#python-scopes-and-namespaces @@ -22,102 +21,104 @@ Looping Sections *What are loops? Refer to the end of this section.* -As an integral extension of :ref:`test_parameters` data-driven testing concept, -``aetest`` also supports section looping: reusing section code body by providing +As an integral extension of the :ref:`test_parameters` data-driven testing concept, +``aetest`` also supports section looping. Section looping means reusing a section's code body by providing it with different parameters during each loop iteration. The following describes each section and their loop capability and behaviors: ``CommonSetup``/``CommonCleanup`` - Common setup and cleanup sections are unique within each testscript. They - are run only once per testscript execution, and are not loopable. + Common setup and cleanup sections are unique within each test script. They + are run only once per test script execution, and are not loopable. ``subsection`` Subsections within ``CommonSetup`` and ``CommonCleanup`` are loopable. - When a ``subsection`` is marked for looping, each of its iterations is + When a ``subsection`` is marked for looping, each iteration is reported as a new subsection. ``Testcase`` Testcases are loopable. Each iteration of a looping ``Testcase`` is reported - individually as new testcase instances with different ``uid``. When a - ``Testcase`` is looped, all of its contents (setup, tests and cleanup) are + individually as new test case instances with a different ``uid``. When a + ``Testcase`` is looped, and all of its contents (setup, tests, and cleanup) are run fully per each iteration. ``setup``/``cleanup`` - Setup and cleanup sections within each testcase is unique, and are run + Setup and cleanup sections within each test case is unique, and are run only once per ``Testcase``. They cannot be looped individually, but if their parent ``Testcase`` is looped, then they are run once per ``Testcase`` iteration. ``test`` Test sections within ``Testcase`` are loopable individually. Each - iteration has its own unique id, and is reported as a new test - section. When a looping ``test`` section's parent ``Testcase`` is also - looped, the resulting loops are multiplicative. Eg: if a testcase is - looped ``2x``, and contains a test that is also looped ``2x``, that - test would loop ``2x`` per testcase loop iteration. + iteration has its own unique id and is reported as a new test + section. When a looping ``test`` section's parent ``Testcase`` is + looped, the resulting loops are multiplicative. E.g.: if a test case is + looped ``2x``and contains a test that is also looped ``2x``, that + test would loop ``2x`` per test case loop iteration. .. hint:: - in other words, ``subsection``, ``Testcase`` and ``test`` sections are the + In other words, ``subsection``, ``Testcase`` and ``test`` sections are the only loopable sections. - Defining Loops -------------- -Sections are marked for looping when they are decorated with ``@loop``, and its -looping parameters provided as decorator arguments. During runtime, when +Sections are marked for looping when decorated with ``@loop``, and its +looping parameters are provided as decorator arguments. During runtime, when ``aetest`` infrastructure detects looped section code, their corresponding -section object is then instantiated once for each of its iterations. +section object is then instantiated once for each iteration. .. code-block:: python # Example # ------- # - # defining loops on sections + # Defining loops on sections from pyats import aetest - # defining a common setup section - # contains a subsection that is looped twice. + # Defining a common setup section + # The CommonSetup section contains a + # subsection that is looped twice. class CommonSetup(aetest.CommonSetup): - # defining a subsection - # this subsection is marked to be looped twice - # the first time having a uid of "subsection_one", and + # Defining a subsection + # ------------------------ + # This subsection is marked to be looped twice. + # The first time having a uid of "subsection_one", and # the second time having a uid of "subsection_two" @aetest.loop(uids=['subsection_one', 'subsection_two']) @aetest.subsection def looped_subsection(self): pass - # defining a testcase that loops - # this testcase also contains a test section that is looped twice + # Defining a test case that loops + # ---------------------------------- + # This test case also contains a test section that is looped twice @aetest.loop(uids=['testcase_one', 'testcase_two']) class Testcase(aetest.Testcase): - # setup section of this testcase is run once - # every time the testcase is looped. + # Setup section of this test case is run once + # every time the test case is looped. @aetest.setup def setup(self): pass - # looped test section - # both iterations are run per testcase iteration + # Looped test section + # both iterations are run per test case iteration @aetest.loop(uids=['test_one', 'test_two']) @aetest.test def test(self): pass - # cleanup section of this testcase is run once - # every time the testcase is looped. + # Cleanup section of this test case is run once + # every time the test case is looped. @aetest.cleanup def cleanup(self): pass - # this testscript's resulting sections would look like this below + # This test script's resulting sections would look like this below # # SECTIONS/TESTCASES RESULT # ---------------------------------------------------------------------- @@ -136,22 +137,22 @@ section object is then instantiated once for each of its iterations. # |-- test_two PASSED # `-- cleanup PASSED -As shown above, the minimum requirement to loop a section (eg, to run its code -1+ times) is to decorate the section with ``@loop``, and provide a list of loop -iteration uids using ``uids`` argument. This controls the number of iterations -this section is looped: each unique item in the ``uids`` list generates to +As shown above, the minimum requirement to loop a section (e.g., to run its code +1+ times) is to decorate the section with ``@loop``and provide a list of loop +iteration uids using the ``uids`` argument. This controls the number of iterations +this section is looped: Each unique item in the ``uids`` list generates a new section with that uid. -When ``@loop`` is used on a ``@subsection`` or ``@test``, the section method +When the ``@loop`` decorator is used on a ``@subsection`` or ``@test``, the section method is effectively decorated twice, and even though the order does not matter, it make more sense to use ``@loop`` as the outermost decorator, signifying that -this method is first marked as a section, then this section is looped. +this method is first marked as a section; then this section is looped. .. tip:: - decorators are executed in the order of "innermost" to "outermost". + Decorators are executed from "innermost" to "outermost." -In addition, in an effort to make the script more aesthetically pleasing, +Additionally, to make the script more aesthetically pleasing, ``aetest`` also features a shortcut to avoid the double decorators: ``@subsection.loop`` and ``@test.loop``. @@ -160,20 +161,20 @@ In addition, in an effort to make the script more aesthetically pleasing, # Example # ------- # - # demonstration the double decorator shortcut for test and subsections + # Demonstrating the double decorator shortcut for tests and subsections from pyats import aetest class CommonSetup(aetest.CommonSetup): - # marking this as both a subsection, and being looped + # Marking this as both a subsection and being looped @aetest.subsection.loop(uids=['subsection_one', 'subsection_two']) def looped_subsection(self): pass class Testcase(aetest.Testcase): - # marking this as both a test section and being looped + # Marking this as both a test section and being looped @aetest.test.loop(uids =['test_one', 'test_two']) def test(self): pass @@ -187,24 +188,23 @@ In addition, in an effort to make the script more aesthetically pleasing, .. tip:: - python ``@decorators`` are evaluated at import time. Thus, decorator - arguments may only be static. If you need to reference runtime and/or - dynamic information information as part of your loop declaration, eg, - accessing parameters & etc, refer to :ref:`dynamic_looping`. - + Python ``@decorators`` are evaluated at import time. Thus, decorator + arguments may only be static. If you need to reference runtime and + dynamic information information as part of your loop declaration, e.g., + accessing parameters, etc., refer to :ref:`dynamic_looping`. Loop Parameters --------------- -Looping the same section again and again is not very useful. Even if each -section has a unique uid as demonstrated above, the usefulness of a test -that repeatedly perform the same actions is questionable. This is where **loop +Looping the same section, again and again, is not very useful. Even if each +section has a unique uid, as demonstrated above, the usefulness of a test +that repeatedly performs the same actions is questionable. This is where **loop parameters** comes in. -Loop parameters feature allows each loop iteration to receive new, distinct +The loop parameters feature allows each loop iteration to receive new, distinct :ref:`test_parameters`. These parameters are specified as part of the ``@loop`` decorator, processed and propagated to each section instance as their -*local parameters*. Combined with :ref:`parameters_as_funcargs` feature, each +*local parameters*. Combined with the :ref:`parameters_as_funcargs` feature, each looped section is then driven to potentially do something different. .. code-block:: python @@ -212,25 +212,24 @@ looped section is then driven to potentially do something different. # Example # ------- # - # loop parameters demonstration + # Loop parameters demonstration from pyats import aetest - # loop this testcast with a loop parameter named "a" + # Loop this test case with a loop parameter named "a" # and set it to value 2 for the first iteration, # and 3 for the second iteration @aetest.loop(a=[2, 3]) class Testcase(aetest.Testcase): - # loop this test with loop parameter named "b" - # and set it to 8 for the first iteration, 9 for the second. + # Loop this test with a loop parameter named "b" + # and set it to 8 for the first iteration and 9 for the second. @aetest.test.loop(b=[8, 9]) def test(self, a, b): # this test prints the exponential of two inputs, a and b print("%s ^ %s = %s" % (a, b, a**b)) - - # the output of the testcase would look like this: + # The output of the test case would look like this: # 2 ^ 8 = 256 # 2 ^ 9 = 512 # 3 ^ 8 = 6561 @@ -249,22 +248,22 @@ looped section is then driven to potentially do something different. # |-- test[b=8] PASSED # `-- test[b=9] PASSED -In effect, loop parameters allows users to create and/or modify the looped -section's local parameters on the fly, per iteration. It is an extension of the -dynamic parameter concept, where section parameters are being generated and fed +In effect, loop parameters allow users to create and modify the looped +section's local parameters on the fly per iteration. It is an extension of the +dynamic parameter concept, where section parameters are generated and fed to each section during runtime. -The use of loop parameters also makes ``uids`` argument optional: if ``uids`` +The use of loop parameters also makes the ``uids`` argument optional: If the ``uids`` arguments are not provided, the infrastructure generates unique section uids by combining the original section name with each of its current loop parameters as postfix -in square backets. Otherwise, the provided ``uids`` are used as section uids. +in square brackets. Otherwise, the provided ``uids`` are used as section uids. There are two methods of providing loop parameters to the ``@loop`` decorator: - - by providing a list of parameters, and a list of parameter values for + - By providing a list of parameters, and a list of parameter values for each iteration (eg, using ``args`` and ``argvs``) - - by providing each parameter as a keyword argument, and a list of its + - By providing each parameter as a keyword argument, and a list of its corresponding argument values. (eg, ``a=[1, 2, 3], b=[4, 5, 6]``) .. code-block:: python @@ -272,22 +271,22 @@ There are two methods of providing loop parameters to the ``@loop`` decorator: # Example # ------- # - # providing loop parameters + # Providing loop parameters from pyats import aetest class Testcase(aetest.Testcase): - # loop this test with arguments "a", "b", and "c" - # provide all of its iteration arguments together using method one - # the positions of each value in argvs corresponds to its args name + # Loop this test with arguments "a", "b", and "c". + # Provide all of its iteration arguments together using method one. + # The positions of each value in argvs correspond to its args name. @aetest.test.loop(args=('a', 'b', 'c'), argvs=((1, 2, 3), (4, 5, 6))) def test_one(self, a, b, c): print("a=%s, b=%s, c=%s" % (a, b, c)) - # loop this test with the same arguments as above, but + # Loop this test with the same arguments as above, but # provide each of its iteration arguments independently using method two @aetest.test.loop(a=(1,4), b=(2,5), @@ -296,7 +295,7 @@ There are two methods of providing loop parameters to the ``@loop`` decorator: print("a=%s, b=%s, c=%s" % (a, b, c)) - # testcase output: + # Testcase output: # a=1, b=2, c=3 # a=4, b=5, c=6 # a=1, b=2, c=3 @@ -311,25 +310,25 @@ There are two methods of providing loop parameters to the ``@loop`` decorator: # |-- test_two[a=1,b=2,c=3] PASSED # `-- test_two[a=4,b=5,c=6] PASSED -As shown above, there were no difference in the outcome of the results. The only +As shown above, there were no differences in the outcome of the results. The only difference was how the loop parameters were provided. One method may be superior -to the other depending on the situation, the number of arguments, etc. +to the other depending on the use case, the number of arguments, etc. -When using loop parameters, the following rules determines the actual number +When using loop parameters, the following rules determine the actual number of iterations: - - if ``uids`` were provided, the number of iterations is equal to the number + - If ``uids`` arguments were provided, the number of iterations is equal to the number of ``uids`` provided. - - if the number of parameter values exceeds the number of ``uids``, all + - If the number of parameter values exceeds the number of ``uids``, all extra values are discarded. - - if ``uids`` are not provided, the number of iterations is equal to the + - If ``uids`` arguments are not provided, the number of iterations equals the number of loop parameter values. Eg, if ``@loop(a=[1,2,3])``, then there would be 3 loop instances, each taking on one distinct value: ``a=1``, ``a=2``, ``a=3``. - - if there are multiple parameters and the number of their values do not + - If there are multiple parameters and the number of their values do not agree, or if the number of parameter values is less than the number of provided ``uids``, a ``filler`` is used to fill empty spots. ``filler`` defaults to ``None``, and only 1 filler can be provided. @@ -339,26 +338,26 @@ of iterations: # Example # ------- # - # loop parameter combinations - # (pseudo code for demonstration only) + # Loop parameter combinations + # (pseudo-code for demonstration only) from pyats.aetest import loop - # loop with 2 iterations using uids argument + # Loop with 2 iterations using uids argument # ------------------------------------------ # iteration 1: uid='id_one' # iteration 2: uid='id_two' @loop(uids=['id_one', 'id_two']) - # loop with 2 iterations using parameters argument + # Loop with 2 iterations using parameters argument # ------------------------------------------------ # iteration 1: a=1, b=4 # iteration 2: a=2, b=5 @loop(a = [1, 2], b = [4, 5]) - # same as above, using args and argvs + # Same as above, using args and argvs @loop(args=['a', 'b'], argvs=[(1, 4), (2, 5)]) - # loop with 2 iterations, and extra arguments are discarded due to uids + # Loop with 2 iterations, and extra arguments are discarded due to uids # --------------------------------------------------------------------- # iteration 1: uid='id_one', a=1, b=2 # iteration 2: uid='id_two', a=3, b=4 @@ -368,23 +367,23 @@ of iterations: argvs=[(1, 2), (3, 4), (5, 6)]) - # same example as above but using per-parameter values + # Same example as above but using per-parameter values @loop(uids=['id_one', 'id_two'], a=[1, 3, 5], b=[2, 4, 6]) - # loop with 3 iterations, and their number of parameters values do not agree + # Loop with 3 iterations, and their number of parameters values do not agree # -------------------------------------------------------------------------- # iteration 1: a=1, b=4 # iteration 2: a=2, b=5 # iteration 3: a=3, b=None ---> default filler comes in to fill the blanks @loop(a=[1, 2, 3], b=[4, 5]) - # same as above, using args and argvs + # Same as above, using args and argvs @loop(args=['a', 'b'], argvs=[(1, 4), (2, 5), (3, )]) - # loop with more uids than parameters, and custom filler + # Loop with more uids than parameters, and custom filler # ------------------------------------------------------ # iteration 1: uid='id_one', a=1, b=3 # iteration 2: uid='id_two', a=2, b=4 @@ -399,13 +398,13 @@ Advanced Loop Usages -------------------- Arguments to the ``@loop`` decorator may also be `callable`_, `iterable`_, or a -`generator`_. The infrastructure is able to distinguish and treat each as you +`generator`_. The infrastructure can distinguish and treat each as you would normally expect it to: - - if an argument value is a `callable`_, it is called, and its returns + - If an argument value is a `callable`_, it is called, and its returns are then used as the actual loop argument value. - - if an argument value is an `iterable`_ or a `generator`_, the loop engine + - If an argument value is an `iterable`_ or a `generator`_, the loop engine picks only one element from it at a time to build the next iteration, until it is exhausted. @@ -418,18 +417,18 @@ would normally expect it to: # Example # ------- # - # demonstrating advanced loop parameter behaviors + # Demonstrating advanced loop parameter behaviors from pyats import aetest - # defining a function - # functions are callable + # Defining a function + # Functions are callable def my_function(): value = [1, 2, 3] print("returning %s" % value) return value - # defining a generator + # Defining a generator def my_generator(): for i in [4, 5, 6]: print('generating %s' % i) @@ -437,20 +436,20 @@ would normally expect it to: class Testcase(aetest.Testcase): - # creating test section with parameter "a" as a function - # note that the function object is passed, not its values + # Creating test section with parameter "a" as a function + # Note that the function object is passed, not its values @aetest.test.loop(a=my_function) def test_one(self, a): print("a = %s" % a) - # creating a test section with parameter "b" as a generator - # note that the generator is a result of calling my_generator(), not + # Creating a test section with parameter "b" as a generator + # Note that the generator is a result of calling my_generator(), not # the function itself. @aetest.test.loop(b=my_generator()) def test_two(self, b): print('b = %s' % b) - # the output of the testcase would be: + # The output of the test case would be: # returning [1, 2, 3] # a = 1 # a = 2 @@ -470,31 +469,30 @@ In the above example, pay close attention to the output lines: - Iterators and generators are only queried before the next section needs to be created. -This behavior enables the use of custom generator as input values to your loop +This behavior enables using a custom generator as input values to your loop parameters. For example, a generator state machine that queries the current testbed device status and creates iterations based on that information. Since the generator is not polled until right before the next iteration, your custom function is only run in-between test sections, thus dynamically generating the -loop iterations based current test environments. - +loop iterations based on current test environments. .. _dynamic_looping: Dynamic Loop Marking -------------------- -So far, all loop examples focused on defining ``@loop`` directly within the -testscripts. Eg, the ``@loop`` decorators are coded as part of the testscript. -In addition, it is also possible to dynamically mark sections for looping during -runtime, eg, creating loops based on information that is only available during -a script run. To do this, use the ``loop.mark()`` function. +So far, all loop examples focus on defining the ``@loop`` decorator directly within the +test scripts. E.g., the ``@loop`` decorators are coded as part of the test script. +However, it is also possible to dynamically mark sections for looping during +runtime, e.g., creating loops based on information that is only available during +a script’s run. To do this, use the ``loop.mark()`` function. .. code-block:: python # Example # ------- # - # dynamically marking sections for looping + # Dynamically marking sections for looping from pyats import aetest @@ -502,23 +500,22 @@ a script run. To do this, use the ``loop.mark()`` function. @aetest.setup def setup(self): - # mark the next test for looping - # provide it with two unique test uids. + # Mark the next test for looping + # Provide it with two unique test uids. # (self.simple_test is the next test method) aetest.loop.mark(self.simple_test, uids=['test_one', 'test_two']) - # note: the simple_test section is not directly marked for looping + # Note: the simple_test section is not directly marked for looping # instead, during runtime, its testcase's setup section marks it for # looping dynamically. @aetest.test def simple_test(self, section): - # print the current section uid + # Print the current section uid # by using the internal parameter "section" print("current section: %s" % section.uid) - - # output of this testcase + # Output of this test case # current section: test_one # current section: test_two # @@ -530,20 +527,19 @@ a script run. To do this, use the ``loop.mark()`` function. # |-- test_one PASSED # `-- test_two PASSED -``loop.mark()`` arguments & behaviors (including loop parameters & etc) are -exactly identical to its sibling ``@loop`` decorator, with the only exception -that its first input argument must be the target section method/class. Eg: +``loop.mark()`` arguments and behaviors (including loop parameters, etc.) are +exactly identical to its sibling, the ``@loop`` decorator, with the only exception +that its first input argument must be the target section method/class. E.g.: ``loop.mark(Testcase_Two, a=[1,2,3])``. -The benefit of this approach is simple: dynamic information, parameters and -variables such as :ref:`script_args`, :ref:`parent` etc, are only available +The benefit of this approach is simple: Dynamic information, parameters and +variables such as :ref:`script_args`, :ref:`parent` etc., are only available during runtime. This information and its corresponding variables are not available when the script is written, and delaying variable references (while -using ``@loop`` decorator) in Python is very difficult, if not impossible. +using the ``@loop`` decorator) in Python is very difficult, if not impossible. -------------------------------------------------------------------------------- - Loop Internals -------------- @@ -552,61 +548,60 @@ Loop Internals The information here onwards is for users interested in ``aetest`` internals & extensions only. - If you are new to this, do not read on. These advanced topics may - further fuel your confusion. + **If you are new to this, do not read on. These advanced topics may + further fuel your confusion.** The previous sections focused on the "how to use" aspect of ``aetest`` looping functionality. From here onwards, we'll dig deeper into loop internals, look at how it functions, and how to deviate from its default behaviors. -The ``aetest`` looping behavior & how its arguments are processed is actually +The ``aetest`` looping behavior and how its arguments are processed are highly customizable. This was not highlighted in previous sections for the sake of serializing the training & simplifying the learning curve. -In reality, consider ``@loop`` decorator and ``loop.mark()`` function as only -markers: they only mark the given section for looping. The details (parameters) -of each iterations is actually generated from **loop generators**, where all -arguments to ``@loop`` and ``loop.mark()`` propagates to. Eg: +In reality, consider the ``@loop`` decorator and ``loop.mark()`` function as only +markers: They only mark the given section for looping. The details (parameters) +Each iteration is generated from **loop generators**, where all +arguments to ``@loop`` and ``loop.mark()``propagate to. E.g.: .. code-block:: python # Example # ------- # - # pseudo code demonstrating @loop decorator functionality + # Pseudo-code demonstrating @loop decorator functionality - # what the loop decorator definition sort-of looks like - # note where the generator defaults to "DefaultLooper" + # What the loop decorator definition sort of looks like + # Note where the generator defaults to "DefaultLooper" def loop(generator=DefaultLooper, *args, **kwargs): - # the actual loop generator gets called with all of the arguments + # The actual loop generator gets called with all of the arguments # to loop decorator, and generates each section iteration return generator(*args, **kwargs) - # pseudo code here onwards, demonstrating internals + # Pseudo-code here onwards, demonstrating internals # ------------------------------------------------- # - # during runtime, the looped is expanded to create each iteration + # During runtime, the looped is expanded to create each iteration for iteration in loop(*args, **kwargs): - # create a section from iteration information and run it + # Create a section from iteration information and run it # ... - # eg, instantiate Subsection + #E.g., instantiate Subsection subsection = Subsection(uid=iteration.uid, parameters=iteration.parameters) # and add to common setup's subsections list common_setup.subsections.append(subsection) - # etc ... - + # etc. -Behind the scenes, **loop generators** are the actual classes that does the -heavy lifting: creating each iteration based on ``@loop`` and ``loop.mark()`` +Behind the scenes, **loop generators** are the actual classes that do the +heavy lifting: Creating each iteration based on the ``@loop`` and ``loop.mark()`` decorator arguments. Loop generators are `iterable`_. Each of its returned -member is an instance of ``Iteration`` class, containing the uid & parameters -information unique to this loop, and used by the infrastructure to create the +members is an instance of the ``Iteration`` class, containing the uid & parameters +information unique to this loop and used by the infrastructure to create the next section instance. .. csv-table:: Iteration class (collections.namedtuple) @@ -619,61 +614,61 @@ next section instance. the next looped section" In other words, **loop generator** is the object that ultimately controls how -loops are generated, and what parameters each iteration is associated with. The -looping behavior and arguments described in topics above are actually that of +loops are generated and what parameters each iteration is associated with. The +looping behavior and arguments described in the topics above are that of ``DefaultLooper``, the default **loop generator** provided by ``aetest`` loop infrastructure. Its features are sufficient for most use cases. However, if you -wish to customize loop behavior, it is possible to extend and/or override it. +wish to customize loop behavior, it is possible to extend and override it. .. code-block:: python # Example # ------- # - # demonstrating how to write and pass your own loop generator + # Demonstrating how to write and pass your own loop generator - # loop generators must return Iterations + # Loop generators must return Iterations from pyats import aetest from pyats.aetest.loop import Iteration - # let's write a custom loop generator - # it generates integers between a and b as loop iterations - # and pass the integer as "number" parameter of the executed section. - # each iteration uid is named "iteration_uid" + number + # Let's write a custom loop generator + # It generates integers between a and b as loop iterations + # and pass the integer as the "number" parameter of the executed section. + # Each iteration uid is named "iteration_uid" + number class DemoGenerator(object): - # at minimum, the loop generator needs to accept an argument called + # At a minimum, the loop generator needs to accept an argument called # "loopee", which is the actual object being looped. This allows the - # loop generator to know what it is looping on, and build information + # loop generator to know what it is looping on and build information # based on it. - # in this example, we're ignoring that argument, as our loop genrator - # is simple and straightforward. + # In this example, we're ignoring that argument, as our loop generator + # is straightforward. def __init__(self, loopee, a, b): self.numbers = list(range(a, b)) def __iter__(self): for i in self.numbers: - # each generated member is an instance of Iteration - # each Iteration must have a unique id + # Each generated member is an instance of Iteration + # Each Iteration must have a unique id # and all of its parameters stored in a dictionary yield Iteration(uid='iteration_uid_%s' % i, parameters={'number': i}) - # this loop generator can be used as @loop and loop.mark() argument. - # let's define a looped testcase with it. + # This loop generator can be used as the @loop and loop.mark() argument. + # Let's define a looped test case with it. - # looping this testcase with custom generator, and a=1, b=5 + # Looping this test case with a custom generator, and a=1, b=5 @aetest.loop(generator=DemoGenerator, a=1, b=5) class Testcase(aetest.Testcase): - # since our generator generates a parameter named "number" - # let's print it in this simple test. + # Since our generator generates a parameter named "number" + # Let's print it in this simple test. @aetest.test def test(self, number): print('current number: %s' % number) - # output of this testcase + # Output of this test case # current number: 1 # current number: 2 # current number: 3 @@ -693,14 +688,14 @@ wish to customize loop behavior, it is possible to extend and/or override it. .. hint:: - the above examples may be simple, the demonstrated underlying principles + The above examples may be simple, but the demonstrated underlying principles are not. *"Do not try and bend the spoon. That's impossible. Instead... only try to realize the truth..."* -And voila. Custom **loop generators** like above is immensely powerful: by -extending and/or overriding the default loop generation behavior, and defining +And voila. Custom **loop generators** like the above are immensely powerful: By +extending and overriding the default loop generation behavior, and defining custom test sections entirely driven by parameter inputs, users can effectively overload the loop functionality into a dynamic generator of highly abstracted test executor. From 4b152ee1c8009faed467739fb6dfbf42d0f880e5 Mon Sep 17 00:00:00 2001 From: Michael Bear <38406045+mjbear@users.noreply.github.com> Date: Tue, 5 Nov 2024 12:04:48 -0500 Subject: [PATCH 5/5] Update Testcase/TestScript words in loop.rst Where it's clear the words are referring to Python objects: * test case > Testcase * test script > TestScript --- docs/aetest/loop.rst | 46 ++++++++++++++++++++++---------------------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/docs/aetest/loop.rst b/docs/aetest/loop.rst index 0a5ded3..ab4698e 100644 --- a/docs/aetest/loop.rst +++ b/docs/aetest/loop.rst @@ -28,8 +28,8 @@ it with different parameters during each loop iteration. The following describes each section and their loop capability and behaviors: ``CommonSetup``/``CommonCleanup`` - Common setup and cleanup sections are unique within each test script. They - are run only once per test script execution, and are not loopable. + Common setup and cleanup sections are unique within each TestScript. They + are run only once per TestScript execution, and are not loopable. ``subsection`` Subsections within ``CommonSetup`` and ``CommonCleanup`` are loopable. @@ -38,12 +38,12 @@ The following describes each section and their loop capability and behaviors: ``Testcase`` Testcases are loopable. Each iteration of a looping ``Testcase`` is reported - individually as new test case instances with a different ``uid``. When a + individually as new ``Testcase`` instances with a different ``uid``. When a ``Testcase`` is looped, and all of its contents (setup, tests, and cleanup) are run fully per each iteration. ``setup``/``cleanup`` - Setup and cleanup sections within each test case is unique, and are run + Setup and cleanup sections within each Testcase is unique, and are run only once per ``Testcase``. They cannot be looped individually, but if their parent ``Testcase`` is looped, then they are run once per ``Testcase`` iteration. @@ -52,9 +52,9 @@ The following describes each section and their loop capability and behaviors: Test sections within ``Testcase`` are loopable individually. Each iteration has its own unique id and is reported as a new test section. When a looping ``test`` section's parent ``Testcase`` is - looped, the resulting loops are multiplicative. E.g.: if a test case is - looped ``2x``and contains a test that is also looped ``2x``, that - test would loop ``2x`` per test case loop iteration. + looped, the resulting loops are multiplicative. E.g.: if a Testcase is + looped ``2x`` and contains a test that is also looped ``2x``, that + test would loop ``2x`` per Testcase loop iteration. .. hint:: @@ -93,27 +93,27 @@ section object is then instantiated once for each iteration. def looped_subsection(self): pass - # Defining a test case that loops + # Defining a Testcase that loops # ---------------------------------- - # This test case also contains a test section that is looped twice + # This Testcase also contains a test section that is looped twice @aetest.loop(uids=['testcase_one', 'testcase_two']) class Testcase(aetest.Testcase): - # Setup section of this test case is run once - # every time the test case is looped. + # Setup section of this Testcase is run once + # every time the Testcase is looped. @aetest.setup def setup(self): pass # Looped test section - # both iterations are run per test case iteration + # both iterations are run per Testcase iteration @aetest.loop(uids=['test_one', 'test_two']) @aetest.test def test(self): pass - # Cleanup section of this test case is run once - # every time the test case is looped. + # Cleanup section of this Testcase is run once + # every time the Testcase is looped. @aetest.cleanup def cleanup(self): pass @@ -216,7 +216,7 @@ looped section is then driven to potentially do something different. from pyats import aetest - # Loop this test case with a loop parameter named "a" + # Loop this Testcase with a loop parameter named "a" # and set it to value 2 for the first iteration, # and 3 for the second iteration @aetest.loop(a=[2, 3]) @@ -229,7 +229,7 @@ looped section is then driven to potentially do something different. # this test prints the exponential of two inputs, a and b print("%s ^ %s = %s" % (a, b, a**b)) - # The output of the test case would look like this: + # The output of the Testcase would look like this: # 2 ^ 8 = 256 # 2 ^ 9 = 512 # 3 ^ 8 = 6561 @@ -449,7 +449,7 @@ would normally expect it to: def test_two(self, b): print('b = %s' % b) - # The output of the test case would be: + # The output of the Testcase would be: # returning [1, 2, 3] # a = 1 # a = 2 @@ -482,7 +482,7 @@ Dynamic Loop Marking -------------------- So far, all loop examples focus on defining the ``@loop`` decorator directly within the -test scripts. E.g., the ``@loop`` decorators are coded as part of the test script. +TestScripts. E.g., the ``@loop`` decorators are coded as part of the TestScript. However, it is also possible to dynamically mark sections for looping during runtime, e.g., creating loops based on information that is only available during a script’s run. To do this, use the ``loop.mark()`` function. @@ -506,7 +506,7 @@ a script’s run. To do this, use the ``loop.mark()`` function. aetest.loop.mark(self.simple_test, uids=['test_one', 'test_two']) # Note: the simple_test section is not directly marked for looping - # instead, during runtime, its testcase's setup section marks it for + # instead, during runtime, its Testcase's setup section marks it for # looping dynamically. @aetest.test @@ -515,7 +515,7 @@ a script’s run. To do this, use the ``loop.mark()`` function. # by using the internal parameter "section" print("current section: %s" % section.uid) - # Output of this test case + # Output of this Testcase # current section: test_one # current section: test_two # @@ -656,9 +656,9 @@ wish to customize loop behavior, it is possible to extend and override it. # This loop generator can be used as the @loop and loop.mark() argument. - # Let's define a looped test case with it. + # Let's define a looped Testcase with it. - # Looping this test case with a custom generator, and a=1, b=5 + # Looping this Testcase with a custom generator, and a=1, b=5 @aetest.loop(generator=DemoGenerator, a=1, b=5) class Testcase(aetest.Testcase): @@ -668,7 +668,7 @@ wish to customize loop behavior, it is possible to extend and override it. def test(self, number): print('current number: %s' % number) - # Output of this test case + # Output of this Testcase # current number: 1 # current number: 2 # current number: 3