Agents 365 (A365) – Critical Gaps in Instance Lifecycle, Licensing Enforcement, and Runtime Support #197
Description
1. Agent Instance Visibility Issues
Challenges
After successfully creating an agent instance, the instance does not immediately appear in:
Microsoft 365 Copilot
Teams organizational hierarchy
Agent listings (only blueprints are visible)
2. Notifications and Feedback Mechanism
Challenges
Instance creation displays the message: “Your agent is being created. You will be notified when the setup process is complete.”
Notifications are not delivered via email as expected.
Notifications appear only inside the Teams activity feed, which is not documented.
3. Licensing Confusion
Challenges
Agent creation and deployment succeed even when required licenses are missing.
Errors such as “No license available” appear only at runtime or during interaction.
Licenses appear to be required per agent instance rather than per blueprint.
For pre-built solutions, instance creation fails, requiring recreation of the agent blueprint.
4. Observability and Debugging Limitations
Challenges
No centralized UI exists for viewing:
Agent runtime logs (Azure Bot logs were used as a workaround)
Instance-specific errors
Request/response traces
Developers must manually inspect Azure App Service logs.
5. Admin Approval and Instance Governance
Challenges
Admin approval is requested only during the first instance creation.
Subsequent instances can be created without clear admin notifications.
The approval lifecycle is not transparent in the Admin Center.
Permissions (e.g., MCP and the first five permissions during app registration) lack visibility and clarity post-approval.
6. Template Creation for Licenses
Challenges
Newly created license templates do not get saved or reflected immediately.
License templates cannot be added to already created and admin-approved blueprints.
7. Node.js Runtime Compatibility Issues (Node 20 vs Node 22+)
Challenge
Agents 365 (A365) agents fail at runtime when using Node.js 20, despite successful deployment and message receipt.
The issue is resolved only when upgrading to Node.js 22 or higher, indicating an undocumented runtime compatibility dependency.
Observed Symptoms
Agent receives messages from Microsoft 365 successfully.
/api/messages endpoint responds with HTTP 202 (Accepted).
Agent fails while attempting to send a reply.
Runtime logs show authentication failures related to Agentic token handling.
Issue occurs consistently on Node.js 20 and does not occur on Node.js 22+.
Key Error Observed
Only IConfidentialClientApplication is supported for Agentic.
Source: Microsoft.Agents.Authentication.Msal
8. Inaccessible GitHub Documentation Link During a365 setup all
Challenge
During the a365 setup all process, documentation references a GitHub guide that is no longer accessible.
The provided URL returns a 404 (Not Found) error, blocking developers from completing required setup steps.
Broken Reference URL:
https://github.com/microsoft/Agent365-devTools/blob/main/docs/guides/custom-client-app-registration…
9. Inconsistent Device Code Authentication Prompt
Challenge
Device Code authentication is sometimes required during A365 setup and execution, but the behavior is inconsistent and unpredictable.
In some cases, the Device Code prompt appears as expected; in others, it does not trigger at all, without any clear indication.
Additional Points to Note
After instance creation, the agent instance is not visible in Teams until a license is explicitly assigned to the instance.
These are some points our team has observed and checking with the engineering team sharing here as well