Use configurable credentials in reconcile in place of db secret scope

[m-abulazm]

Changes

What does this PR do?

• Move away from hardcoded secrets in reconcile
• use credential manager which enables local, env and databricks

Relevant implementation details

• add load_credentials to DataSource which takes care of loading the credentials

Caveats/things to watch out for when reviewing:

Linked issues

Progresses #1008, #2123, #2157

Functionality

• added relevant user documentation
• added new CLI command
• modified existing command: databricks labs lakebridge reconcile
• ... +add your own

Tests

• manually tested
• added unit tests
• added integration tests

[codecov]

Codecov Report

❌ Patch coverage is 73.14815% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.53%. Comparing base (63a7104) to head (bbf7ad5).
⚠️ Report is 1 commits behind head on refactor/creds-manager.

Files with missing lines	Patch %	Lines
.../labs/lakebridge/reconcile/connectors/snowflake.py	68.75%	7 Missing and 3 partials ⚠️
...cks/labs/lakebridge/reconcile/connectors/oracle.py	80.00%	4 Missing and 1 partial ⚠️
...ricks/labs/lakebridge/reconcile/connectors/tsql.py	77.27%	4 Missing and 1 partial ⚠️
src/databricks/labs/lakebridge/reconcile/utils.py	0.00%	5 Missing ⚠️
...abs/lakebridge/reconcile/connectors/data_source.py	75.00%	2 Missing ⚠️
...labs/lakebridge/reconcile/connectors/databricks.py	80.00%	1 Missing ⚠️
.../lakebridge/reconcile/connectors/source_adapter.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@                    Coverage Diff                     @@
##           refactor/creds-manager    #2159      +/-   ##
==========================================================
- Coverage                   63.68%   63.53%   -0.16%     
==========================================================
  Files                         100       98       -2     
  Lines                        8542     8561      +19     
  Branches                      886      895       +9     
==========================================================
- Hits                         5440     5439       -1     
- Misses                       2935     2950      +15     
- Partials                      167      172       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

✅ 51/51 passed, 4 flaky, 4m46s total

Flaky tests:

• 🤪 test_transpiles_informatica_to_sparksql (23.757s)
• 🤪 test_transpiles_informatica_to_sparksql_non_interactive[False] (20.609s)
• 🤪 test_transpile_teradata_sql_non_interactive[False] (21.488s)
• 🤪 test_transpile_teradata_sql (7.732s)

_{Running from acceptance #3224}