Development Tools

Debugging and Development

Debug Mode

Deployments in fabric-cicd can be run in "debug mode" by enabling the following in your deployment script:

change_log_level()

The function will print out logging messages in a grey color to the console, including the API requests and responses called throughout a deployment run. The additional information can be helpful when debugging a deployment. When the function is absent, the console prints minimal logs that show deployment progress.

Debug Scripts

The devtools/ directory contains several scripts to help contributors test and debug their code changes before submitting pull requests.

The folder includes the following scripts:

debug_local.py

Purpose: Test full deployment workflows locally against a Microsoft Fabric workspace.

How To:

• Connect to your own repository directory or use the sample repository directory provided in the repo (sample -> workspace)

Use Cases:

• Validate changes to publish/unpublish logic
• Test deployments against different workspace configurations (item types, folder structures, sizes)
• Debug item type-specific deployment issues
• Verify feature flag behavior
• Test a deployment after introducing changes to the codebase
• Run an ad hoc deployment via user auth or pass in a token credential (e.g., SPN)

Configuration:

# 1. Set your workspace ID, environment, and repository directory path
workspace_id = "your-workspace-id"
environment = "DEV"  # Must match environment in parameter.yml
repository_directory = "root/sample/workspace" # In this example, our workspace content sits within the root/sample/workspace directory

# 2. Configure authentication (optional)
# Uncomment to use Service Principal authentication
# token_credential = ClientSecretCredential(
#     client_id="your-client-id",
#     client_secret="your-client-secret",
#     tenant_id="your-tenant-id"
# )

# 3. Enable debug logging (optional)
# change_log_level()

# 4. Set required feature flags, if any (optional)
# append_feature_flag("feature_flag_1")
# append_feature_flag("feature_flag_2")

# 5. Select item types to deploy (optional, otherwise deploys all supported item types)
# item_type_in_scope = ["Notebook", "DataPipeline", "Environment"]

# 6. Create the FabricWorkspace object
target_workspace = FabricWorkspace(
    workspace_id=workspace_id,
    environment=environment,
    repository_directory=repository_directory,
    # Uncomment to deploy specific item types
    # item_type_in_scope=item_type_in_scope,
    # Uncomment to use SPN auth
    # token_credential=token_credential,
)

# 7. Uncomment publish operation to test
# publish_all_items(target_workspace)

# 8. Uncomment unpublish operation to test
# unpublish_all_orphan_items(target_workspace)

Usage:

cd /path/to/fabric-cicd
uv run python devtools/debug_local.py

debug_local config.py

Purpose: Test configuration-based deployment workflows using a config.yml file. See configuration deployment for more information.

How To:

• Create a config.yml file, add the necessary configurations, and pass in the proper path to the file in the deploy function
• In config.yml, reference your own repository directory or use the sample repository directory provided in the repo (sample -> workspace)

Use Cases:

• Validate the experimental deploy_with_config feature
• Test deployments with custom configurations
• Debug configuration file parsing and validation
• Test a deployment after introducing changes to the codebase
• Run an ad hoc deployment via user auth or another type of auth (e.g., SPN)

Configuration:

# 1. Enable debug logging (optional)
# change_log_level()

# 2. Enable required feature flags
append_feature_flag("enable_experimental_features")
append_feature_flag("enable_config_deploy")

# 3. Point to your config file
config_file = "path/to/config.yml"

# 4. Set environment
environment = "dev"

# 5. Run deployment
deploy_with_config(
    config_file_path=config_file,
    environment=environment
)

Usage:

uv run python "devtools/debug_local config.py"

debug_parameterization.py

Purpose: Validate the parameter.yml file without running actual deployments. See parameterization for more information.

How To:

• Create your own parameter.yml file or use the sample parameter.yml provided in the repo (sample -> workspace)

Use Cases:

• Test parameter file structure and syntax
• Verify parameter values exist for target environment
• Validate parameter configurations
• Debug validation failures with parameters
• Test changes made to parameterization codebase
• Run an ad hoc validation on the parameter file

Configuration:

# 1. Enable debug logging (optional)
# change_log_level()

# 2. Set repository directory containing parameter.yml
repository_directory = "path/to/workspace"

# 3. Define item types to validate against (optional)
#item_type_in_scope = ["DataPipeline", "Notebook", "Environment"]

# 4. Set target environment
environment = "PPE"

# 5. Use custom parameter file location (optional)
# parameter_file_path = "path/to/custom/parameter.yml"

# 6. Run the validation function using the defined input
validate_parameter_file(
    repository_directory=repository_directory,
    # Uncomment to consider specific item types
    # item_type_in_scope=item_type_in_scope,
    # Comment to exclude target environment in validation
    environment=environment,
    # Uncomment to use a different parameter file name within the repository directory (default name: parameter.yml)
    # Assign to the constant in constants.py or pass in a string directly
    # parameter_file_name=constants.PARAMETER_FILE_NAME,
    # Uncomment to use a parameter file from outside the repository (takes precedence over parameter_file_name)
    # parameter_file_path=parameter_file_path,
    # Uncomment to use SPN auth
    # token_credential=token_credential,
)

Usage:

uv run python devtools/debug_parameterization.py

debug_api.py

Purpose: Test Fabric REST API calls directly without going through full deployment workflows.

How To:

• Use the default constant value for the root of the API url, override the value, or disregard entirely (customizable)
• Authenticate via user auth or pass in token credential value for alternate identity auth (e.g., SPN)

Use Cases:

• Debug and validate Fabric API endpoints
• Test API request/response payloads
• Prototype new API integrations
• Verify authentication and authorization
• Troubleshoot API-specific issues
• Make ad hoc API calls (e.g., get item definition for specific items)

Configuration:

# 1. Enable debug logging (optional)
# change_log_level()

# 2. Configure authentication (optional)
# Replace None with credential when using SPN auth
token_credential = None  # Uses DefaultAzureCredential if None

# 3. Set workspace ID if needed
workspace_id = "your-workspace-id"

# 4. Configure the API endpoint
api_url = f"{constants.DEFAULT_API_ROOT_URL}/v1/workspaces/{workspace_id}..."

# 5. Make the API call
response = fe.invoke(
    method="POST",  # GET, POST, PUT, DELETE, PATCH
    url=api_url,
    body={},  # Request payload
)

Usage:

uv run python devtools/debug_api.py

pypi_build_release_dev.ps1

Purpose: Build and publish development versions of the package to TestPyPI for testing.

Use Cases:

• Test package installation and imports before releasing to production PyPI
• Validate packaging configuration changes
• Verify distribution includes all necessary files

Usage:

# From PowerShell terminal
cd devtools
.\pypi_build_release_dev.ps1

Warning: This script requires TestPyPI credentials configured in your environment. It's typically only used by maintainers preparing releases.

Development Process

Specific guidelines on the development process can be found in CONTRIBUTING.md in the repository. Using the debugging tools described above during development is strongly encouraged and provides an effective way to test code changes end to end.

Testing Your Changes

Before submitting a pull request, always test your changes:

1. Run Unit Tests:

   uv run pytest -v

2. Check Code Formatting:

   uv run ruff format
   uv run ruff check

3. Test Import Functionality:

   uv run python -c "from fabric_cicd import FabricWorkspace; print('Import successful')"

4. Use Debug Scripts: Test your changes against real scenarios using the appropriate debug script from devtools/.

Running Tests with Debug Output

To see detailed test output and debug information:

# Run with verbose output
uv run pytest -v -s

# Run specific test file
uv run pytest tests/test_specific.py -v

# Run tests matching a pattern
uv run pytest -k "test_pattern" -v

Additional Resources

• Contribution Guide - Setup instructions and PR requirements
• Feature Flags - Available feature flags for advanced scenarios
• Getting Started - Basic installation and authentication
• Microsoft Fabric API Documentation - Official API reference