fix: downmerge main to dev

[Kanchan-Microsoft]

Purpose

This pull request updates dependencies across several requirements files to align with the latest versions and improve compatibility, especially for Azure SDKs and OpenTelemetry. The most important changes are grouped below by theme.

Dependency version updates:

• Upgraded azure-ai-projects from version 1.0.0b12 to 2.0.0b3 in content-gen/infra/vscode_web/requirements.txt to ensure compatibility with newer Azure AI Foundry features.
• Updated azure-ai-projects to version 2.0.0b3 in content-gen/src/backend/requirements.txt for consistency across environments.

Azure SDK and telemetry improvements:

• Added opentelemetry-semantic-conventions-ai==0.4.13 to content-gen/src/backend/requirements.txt to support OpenTelemetry tracing required by agent-framework.
• Added azure-search-documents>=11.4.0 to content-gen/src/backend/requirements.txt to provide search document capabilities for backend services.
• Removed azure-search-documents>=11.6.0 from content-gen/scripts/requirements-post-deploy.txt to centralize dependency management and avoid redundancy.

Does this introduce a breaking change?

• Yes
• No

Golden Path Validation

• I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

• I have validated the deployment process successfully and all services are running as expected with this change.

What to Check

Verify that the following are valid

• ...

Other Information