agent release - add traces (#226)

# Summary

We sometimes run into errors and having these traced makes it easier to
know which ones have failed for later analysis.

This pull request enhances the `queue_exception_handling` function in
`pipeline.py` by integrating OpenTelemetry tracing and adding detailed
metrics to improve observability of task queue error processing.

### Observability Enhancements:

* Added OpenTelemetry tracing to `queue_exception_handling` 
* Introduced metrics to track task queue processing, including:
  - Total number of tasks (`mck.agent.queue.tasks_total`).
  - Count of tasks with exceptions (`mck.agent.queue.exceptions_count`).
  - Success rate of tasks (`mck.agent.queue.success_rate`).
  - Types of exceptions encountered (`mck.agent.queue.exception_types`).
- Boolean flag indicating if exceptions were found
(`mck.agent.queue.has_exceptions`)
* Updated logging to provide more granular details about exceptions
encountered in the task queue

## Proof of Work

<!-- Enter your proof that it works here.-->

## Checklist
- [ ] Have you linked a jira ticket and/or is the ticket in the title?
- [ ] Have you checked whether your jira ticket required DOCSP changes?
- [ ] Have you checked for release_note changes?

## Reminder (Please remove this when merging)
- Please try to Approve or Reject Changes the PR, keep PRs in review as
short as possible
- Our Short Guide for PRs:
[Link](https://docs.google.com/document/d/1T93KUtdvONq43vfTfUt8l92uo4e4SEEvFbIEKOxGr44/edit?tab=t.0)
- Remember the following Communication Standards - use comment prefixes
for clarity:
  * **blocking**: Must be addressed before approval.
  * **follow-up**: Can be addressed in a later PR or ticket.
  * **q**: Clarifying question.
  * **nit**: Non-blocking suggestions.
  * **note**: Side-note, non-actionable. Example: Praise 
  * --> no prefix is considered a question

---------

Co-authored-by: Anand <[email protected]>
Co-authored-by: Anand Singh <[email protected]>

Author: nammn

pipeline.py

@@ -1352,12 +1352,31 @@ def build_agent_on_agent_bump(build_configuration: BuildConfiguration):
     queue_exception_handling(tasks_queue)
 
 
+@TRACER.start_as_current_span("queue_exception_handling")
 def queue_exception_handling(tasks_queue):
+    span = trace.get_current_span()
+
     exceptions_found = False
+    exception_count = 0
+    total_tasks = len(tasks_queue.queue)
+    exception_types = set()
+
+    span.set_attribute("mck.agent.queue.tasks_total", total_tasks)
+
     for task in tasks_queue.queue:
         if task.exception() is not None:
             exceptions_found = True
+            exception_count += 1
+            exception_types.add(type(task.exception()).__name__)
             logger.fatal(f"The following exception has been found when building: {task.exception()}")
+
+    span.set_attribute("mck.agent.queue.exceptions_count", exception_count)
+    span.set_attribute(
+        "mck.agent.queue.success_rate", ((total_tasks - exception_count) / total_tasks * 100) if total_tasks > 0 else 0
+    )
+    span.set_attribute("mck.agent.queue.exception_types", list(exception_types))
+    span.set_attribute("mck.agent.queue.has_exceptions", exceptions_found)
+
     if exceptions_found:
         raise Exception(
             f"Exception(s) found when processing Agent images. \nSee also previous logs for more info\nFailing the build"