[Enhancement] Optimizations for Bundle serialization

[brendankowitz]

Description

High-level goals:

• Allow Bundle Sub-request Resource to be passed through with async cotext
• Allows Bundle Handler to use Raw strings when building response
• This should reduce overall CPU, Memory and Latency while processing bundles.

Sequence of requests:

---
title: Bundle Processing
---

sequenceDiagram
    Request->>JsonFormatter: http
    JsonFormatter->>FHIRController: Deserialize
    FHIRController->>Mediatr: Handle
    Mediatr->>BundleHandler: Process
    
loop Each Request
    BundleHandler-->>+Request: Passes model through FHIRContext
    Request-->>JsonFormatter: Begin Sub-request
    JsonFormatter-->>FHIRController: Read model
    FHIRController-->>Mediatr: Handle operation
    Mediatr-->>DataStore: Serialize and persist
    DataStore--)Mediatr: Returns a "RawResource"
    Mediatr--)FHIRController: Raw Response
    FHIRController--)JsonFormatter: 
    JsonFormatter--)Request: Writes raw response
    Request--)-BundleHandler: Return Raw Entry response
end

    BundleHandler->>FHIRController: Build Bundle
    FHIRController-)JsonFormatter: Serialize Raw Bundle
    JsonFormatter-)Request: http

Loading

Details:

This pull request primarily involves changes to the Microsoft.Health.Fhir.Core project to improve serialization and deserialization of FHIR resources. The most significant changes include refactoring the RawResourceElementExtensions.cs and ResourceDeserializer.cs files to enhance serialization and deserialization respectively, adding a new SubRequest.cs class, and modifying the ResourceDeserializer.cs and RawResourceElement.cs files to improve resource deserialization.

Major changes:

• Serialization improvements:

  • src/Microsoft.Health.Fhir.Core/Extensions/RawResourceElementExtensions.cs: Refactored the SerializeToStreamAsUtf8Json method to use a memory stream for serialization, and added a new SerializeToBytes method for byte array serialization. [1] [2] [3]

• Deserialization improvements:

  • src/Microsoft.Health.Fhir.Core/Features/Persistence/ResourceDeserializer.cs: Added a RecyclableMemoryStreamManager for reusable memory streams, and changed the DeserializeRaw and DeserializeRawResourceElement methods to accept a RawResource object instead of a string. [1] [2] [3]

• New class addition:

  • src/Microsoft.Health.Fhir.Core/Features/Context/SubRequest.cs: Added a new SubRequest class.

• Resource deserialization changes:

  • src/Microsoft.Health.Fhir.Core/Models/RawResourceElement.cs: Added a new constructor to the RawResourceElement class that accepts a RawResource object and an instance type.
  • src/Microsoft.Health.Fhir.Core/Features/Persistence/ResourceTypeDetector.cs: Added a new ResourceTypeDetector class to find the resource type of a given FHIR JSON.

Minor changes:

• docs/rest/SearchExamples.http: Added a x-bundle-processing-logic: Parallel header to the HTTP request.
• src/Microsoft.Health.Fhir.Shared.Api.UnitTests/Features/Formatters/FhirJsonInputFormatterTests.cs: Updated the FhirJsonInputFormatter constructor calls to include a FhirRequestContextAccessor object. [1] [2]
• src/Microsoft.Health.Fhir.Shared.Api.UnitTests/Features/Resources/Bundle/BundleHandlerEdgeCaseTests.cs: Added Deserializers.ResourceDeserializer to the BundleHandler constructor call.
• src/Microsoft.Health.Fhir.Shared.Api.UnitTests/Features/Resources/Bundle/BundleHandlerTests.cs: Added Deserializers.ResourceDeserializer to the BundleHandler constructor call.
• src/Microsoft.Health.Fhir.Shared.Api.UnitTests/Features/Resources/Bundle/BundleSerializerTests.cs: Updated the BundleSerializerTests to handle OperationOutcome resources differently, and removed the BundleSerializer object. [1] [2] [3]

Related issues

Addresses AB#117384

Testing

Adds tests, existing e2e tests. Manual testing.

FHIR Team Checklist

• Update the title of the PR to be succinct and less than 65 characters
• Add a milestone to the PR for the sprint that it is merged (i.e. add S47)
• Tag the PR with the type of update: Bug, Build, Dependencies, Enhancement, New-Feature or Documentation
• Tag the PR with Open source, Azure API for FHIR (CosmosDB or common code) or Azure Healthcare APIs (SQL or common code) to specify where this change is intended to be released.
• Tag the PR with Schema Version backward compatible or Schema Version backward incompatible or Schema Version unchanged if this adds or updates Sql script which is/is not backward compatible with the code.
• CI is green before merge
• Review squash-merge requirements

Semver Change (docs)

Patch (enhancement)