Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Subscriptions engine #4628

Open
wants to merge 51 commits into
base: main
Choose a base branch
from

Conversation

brendankowitz
Copy link
Member

@brendankowitz brendankowitz commented Sep 19, 2024

Description

Describe the changes in this PR.

This PR includes changes from: #3814 (which could be merged first to reduce the complexity)

Related issues

Addresses [issue #].

Testing

Describe how this change was tested.

FHIR Team Checklist

  • Update the title of the PR to be succinct and less than 65 characters
  • Add a milestone to the PR for the sprint that it is merged (i.e. add S47)
  • Tag the PR with the type of update: Bug, Build, Dependencies, Enhancement, New-Feature or Documentation
  • Tag the PR with Open source, Azure API for FHIR (CosmosDB or common code) or Azure Healthcare APIs (SQL or common code) to specify where this change is intended to be released.
  • Tag the PR with Schema Version backward compatible or Schema Version backward incompatible or Schema Version unchanged if this adds or updates Sql script which is/is not backward compatible with the code.
  • CI is green before merge Build Status
  • Review squash-merge requirements

Semver Change (docs)

Patch|Skip|Feature|Breaking (reason)

brendankowitz and others added 18 commits August 21, 2024 17:05
* Improve fhirtimer

* Subscription infra

* Fixes wiring up of Transaction Watchdog => Orchestrator

* Adding code for writing to storage

* Allow resourceKey to deserialize

* Implement basic subscription filtering

* Implements Channel Interface

* Add example subscription

* DataLakeChannel.

* Changes in DataLakeChannel and the project config.

* Load from DB

* EventGrid WIP

* Improve fhirtimer

* Fixes for subscriptionwatchdog

* Aligns dotnet sdk version for build

* Adds subscription to docker build

* Adds SearchQueryInterpreter

* set up notification manager

* Cleanup of SearchQueryInterpreter

* integrate in memory search with filter search for subscriptions

* resolve notification manager build errors

* set up notification manager

* integrate in memory search with filter search for subscriptions

* resolve notification manager build errors

* remove build error in webhook channel

* rename webhook channel name

* set up notification manager

* integrate in memory search with filter search for subscriptions

* resolve notification manager build errors

* set up notification manager

* resolve notification manager build errors

* remove build error in webhook channel

* rename webhook channel name

* in memory search test set up

* subscription orchestrator job test checks for id

* subscription orchestator job only sends filtered resources instead of all resources

* new test for patient name filter

* add a failing test

* rename tests for subscription orchestrator job

* remove web hook code

* move test to r4 folder

* Improve fhirtimer

* Subscription infra

* Fixes wiring up of Transaction Watchdog => Orchestrator

* Adding code for writing to storage

* Allow resourceKey to deserialize

* Implement basic subscription filtering

* Implements Channel Interface

* Add example subscription

* DataLakeChannel.

* Changes in DataLakeChannel and the project config.

* Load from DB

* EventGrid WIP

* Improve fhirtimer

* Fixes for subscriptionwatchdog

* Aligns dotnet sdk version for build

* Adds subscription to docker build

* Adds SearchQueryInterpreter

* Cleanup of SearchQueryInterpreter

* clean up merge

* fix merge conflict params in orchestra job

---------

Co-authored-by: Brendan Kowitz <[email protected]>
Co-authored-by: Fernando Henrique Inocêncio Borba Ferreira <[email protected]>
* Improve fhirtimer

* Subscription infra

* Fixes wiring up of Transaction Watchdog => Orchestrator

* Adding code for writing to storage

* Allow resourceKey to deserialize

* Implement basic subscription filtering

* Implements Channel Interface

* Add example subscription

* DataLakeChannel.

* Changes in DataLakeChannel and the project config.

* Load from DB

* EventGrid WIP

* Improve fhirtimer

* Fixes for subscriptionwatchdog

* Aligns dotnet sdk version for build

* Adds subscription to docker build

* Adds SearchQueryInterpreter

* set up notification manager

* Cleanup of SearchQueryInterpreter

* integrate in memory search with filter search for subscriptions

* resolve notification manager build errors

* set up notification manager

* integrate in memory search with filter search for subscriptions

* resolve notification manager build errors

* remove build error in webhook channel

* rename webhook channel name

* set up notification manager

* integrate in memory search with filter search for subscriptions

* resolve notification manager build errors

* set up notification manager

* resolve notification manager build errors

* remove build error in webhook channel

* rename webhook channel name

* in memory search test set up

* subscription orchestrator job test checks for id

* subscription orchestator job only sends filtered resources instead of all resources

* new test for patient name filter

* add a failing test

* rename tests for subscription orchestrator job

* remove web hook code

* set up validation for subscriptions

* remove rest hook channel temporarily

* add generic notify method

* setup for creating bundles

* create subscription bundle for notification events

* set up heartbeat and handshake

* add notification bundle for rest hook channel and pass in additional fields for subscriptions

* handshake method for subscriptions

* set up heartbeat

* set up heartbeat background service and validation

* modify subscriptionUpdator to convert resource into element node to edit

* build contect for url resolver

* cleanup handshake and heatbeat implementation

* cleanup duplicated test files

* rename fhir context and tests

---------

Co-authored-by: Brendan Kowitz <[email protected]>
Co-authored-by: Fernando Henrique Inocêncio Borba Ferreira <[email protected]>
@brendankowitz brendankowitz force-pushed the feature/subscription-engine branch from 03cebf3 to dc0d6c9 Compare September 19, 2024 17:22
@brendankowitz brendankowitz marked this pull request as ready for review September 24, 2024 16:43
@brendankowitz brendankowitz requested a review from a team as a code owner September 24, 2024 16:43
…engine

# Conflicts:
#	src/Microsoft.Health.Fhir.Core/Configs/CoreFeatureConfiguration.cs
#	src/Microsoft.Health.Fhir.Shared.Web/appsettings.json
#	src/Microsoft.Health.Fhir.SqlServer/Features/Watchdogs/DefragWatchdog.cs
#	src/Microsoft.Health.Fhir.SqlServer/Features/Watchdogs/WatchdogsBackgroundService.cs
Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants