Add mechanism to permit to mock generic methods/functions of the python package

The new `ApiMocker.mockable` is a decorator that can be applied to mock any method.

The decorator will automatically search in the `<function-name-path>.overrides.py` file (if it exists)
a function with the same name of the function.
If the function exists, the decorator will execute it in place of the original one.

The same `slurm_plugin.overrides.py` file is already used by `run_instances` and `create_fleet` methods,
but without the generic decorator mechanism. We can use this new mechanism for them in the future.

I'm using it for `describe_capacity_reservations` to permit to mock it when running e2e tests.
An example of bash script to create overrides file is:
```
node_virtualenv_path=$(sudo find / -iname "site-packages" | grep "node_virtualenv")
# the overrides.py file must be in the same folder of the module of the function to be mocked
cat << EOF | sudo tee -a "${node_virtualenv_path}/aws/overrides.py"
from aws.ec2 import CapacityReservationInfo

def describe_capacity_reservations(_, capacity_reservations_ids):
    return [
        CapacityReservationInfo({
            "CapacityReservationId": "cr-123456",
            "OwnerId": "123456789",
            "CapacityReservationArn": "arn:aws:ec2:us-east-2:123456789:capacity-reservation/cr-123456",
            ...
        })
    ]
EOF
```

### Tests

Tested with:
* not existing overrides.py (`ImportError`)
* empty overrides.py (`AttributeError`)
* overrides.py with other functions defined on it (`AttributeError`)
* overrides.py with the mocked `describe_capacity_reservations` (mocked output)

Signed-off-by: Enrico Usai <[email protected]>

Author: Enrico Usai

src/aws/ec2.py

@@ -10,6 +10,8 @@
 # limitations under the License.
 from typing import List
 
+from common.utils import ApiMocker
+
 from aws.common import AWSExceptionHandler, Boto3Client
 
 
@@ -73,6 +75,7 @@ def __init__(self, config=None, region=None):
         super().__init__("ec2", region=region, config=config)
 
     @AWSExceptionHandler.handle_client_exception
+    @ApiMocker.mockable
     def describe_capacity_reservations(self, capacity_reservation_ids: List[str]) -> List[CapacityReservationInfo]:
         """Accept a space separated list of reservation ids. Return a list of CapacityReservationInfo."""
         result = []

src/common/utils.py

@@ -375,3 +375,39 @@ def filter(self, record: logging.LogRecord) -> bool:
     finally:
         # Remove the custom log filter
         logger.removeFilter(custom_filter)
+
+
+class ApiMocker:
+    """API mocker."""
+
+    @staticmethod
+    def mockable(func):
+        """
+        Try to mock passed function by searching for an overrides.py file in the same path of the given func.
+
+        This function can be used a decorator and applied any method.
+
+        The function will check if a function called with the name of the given function exists
+        in the <function-dir>/overrides.py, and if it does, the function will execute it.
+
+        E.g. if the method with ApiMocker.mockable decorator is defined in Ec2Client class
+        of the ${node_virtualenv_path}/aws/ec2.py module, the mocked function should be defined
+        in the ${node_virtualenv_path}/aws/overrides.py file.
+        """
+
+        def wrapper(*args, **kwargs):
+            try:
+                function_name = func.__name__
+                # retrieve parent module of the given function that has the ApiMocker.mockable decorator
+                func_module = func.__module__
+                func_parent_module = func_module[: func_module.rindex(".")]
+                # try to import overrides.py module in the same folder of the module to mock
+                overrides_module = __import__(f"{func_parent_module}.overrides", fromlist=function_name)
+                overrided_func = getattr(overrides_module, function_name)
+                log.info("Calling %s override with args: %s and kwargs: %s", function_name, args, kwargs)
+                result = overrided_func(*args, **kwargs)
+            except (ImportError, AttributeError):
+                result = func(*args, **kwargs)
+            return result
+
+        return wrapper