Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Foundation work for one-step debugger in Workflows #761

Merged
merged 45 commits into from
Nov 13, 2024
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
9661b48
WIP - first changes to add serializers and deserializers
PawelPeczek-Roboflow Oct 30, 2024
38fc7e5
Create first scratch of implementation for serializers and deserializers
PawelPeczek-Roboflow Oct 30, 2024
209866e
WIP - added handling for arbitrary dimensions in inputs
PawelPeczek-Roboflow Oct 30, 2024
297177f
Finish basic testing of the new feature
PawelPeczek-Roboflow Oct 31, 2024
d31a3d2
Fix batch vs non-batch oriented parameters
PawelPeczek-Roboflow Oct 31, 2024
ff227d5
Add additional tests
PawelPeczek-Roboflow Oct 31, 2024
3379902
Add remaining tests
PawelPeczek-Roboflow Oct 31, 2024
329b2cc
Apply refactor to add BatchOfDataSelector
PawelPeczek-Roboflow Oct 31, 2024
0a6fff4
WIP - add more tests
PawelPeczek-Roboflow Nov 1, 2024
e150394
Resolve conflicts with main
PawelPeczek-Roboflow Nov 1, 2024
2c5a13a
Add deserialization for more kinds to ensure ability to properly vali…
PawelPeczek-Roboflow Nov 4, 2024
598dff2
Merge branch 'main' into feature/add_support_for_all_kinds_inputs
PawelPeczek-Roboflow Nov 4, 2024
7bdcce8
Add tests for deserialization
PawelPeczek-Roboflow Nov 4, 2024
34773e3
Add tests for filtering of workflow results
PawelPeczek-Roboflow Nov 4, 2024
f58a6d1
Add documentation - part 1
PawelPeczek-Roboflow Nov 4, 2024
aba0aaf
Add docs - part 2
PawelPeczek-Roboflow Nov 4, 2024
51d5e6f
Adjust docs to changes
PawelPeczek-Roboflow Nov 4, 2024
272e7ef
Add extension to inference_sdk to handle nested batches of input para…
PawelPeczek-Roboflow Nov 5, 2024
52ddb9f
Add changes to align batches and scalars regarding their place in eco…
PawelPeczek-Roboflow Nov 5, 2024
b02502d
Start using scalar selector everywhere
PawelPeczek-Roboflow Nov 5, 2024
533dd4f
Start using BatchSelector for input images everywhere
PawelPeczek-Roboflow Nov 5, 2024
5b5ec6c
Update docs and add more tests
PawelPeczek-Roboflow Nov 5, 2024
08f2027
Fix block assembler
PawelPeczek-Roboflow Nov 5, 2024
9f31fe3
Merge branch 'main' into feature/add_support_for_all_kinds_inputs
PawelPeczek-Roboflow Nov 5, 2024
9d11498
Merge branch 'main' into feature/add_support_for_all_kinds_inputs
PawelPeczek-Roboflow Nov 7, 2024
e14a960
Refactor the PR to use Selector(...) type annotation for manifest sel…
PawelPeczek-Roboflow Nov 7, 2024
c2c08c1
WIP
PawelPeczek-Roboflow Nov 8, 2024
75ce9a4
WIP
PawelPeczek-Roboflow Nov 8, 2024
0d859a9
Add abstraction to mark mixed inputs
PawelPeczek-Roboflow Nov 8, 2024
9a35585
Fix docs
PawelPeczek-Roboflow Nov 8, 2024
a17c656
Adjust docs to changes
PawelPeczek-Roboflow Nov 11, 2024
5cf9bf9
Resolve conflicts with main
PawelPeczek-Roboflow Nov 11, 2024
2dff5de
Make linters happy
PawelPeczek-Roboflow Nov 11, 2024
89a0aef
Fix bug with Florence block and align blocks expected EE version
PawelPeczek-Roboflow Nov 11, 2024
6f3ca50
Fix typo in docs
PawelPeczek-Roboflow Nov 11, 2024
a42fb71
Fix issue with docs generation
PawelPeczek-Roboflow Nov 11, 2024
c9a3753
Merge branch 'main' into feature/add_support_for_all_kinds_inputs
hansent Nov 12, 2024
d3bed3a
Merge branch 'main' into feature/add_support_for_all_kinds_inputs
grzegorz-roboflow Nov 13, 2024
1c2e494
Update docs/workflows/execution_engine_changelog.md
hansent Nov 13, 2024
9b9ff3c
Update docs/workflows/execution_engine_changelog.md
hansent Nov 13, 2024
e897103
Update docs/workflows/workflows_compiler.md
hansent Nov 13, 2024
21cd0a6
Update docs/workflows/workflows_compiler.md
hansent Nov 13, 2024
5beeb04
Apply suggestions from PR CR
PawelPeczek-Roboflow Nov 13, 2024
6992db4
Merge branch 'main' into feature/add_support_for_all_kinds_inputs
PawelPeczek-Roboflow Nov 13, 2024
647fd28
Merge branch 'main' into feature/add_support_for_all_kinds_inputs
PawelPeczek-Roboflow Nov 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion docs/workflows/blocks.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,6 @@ hide:
<p class="card block-card" data-url="stability_ai_inpainting" data-name="Stability AI Inpainting" data-desc="Uses segmentation masks to inpaint objects into image" data-labels="MODEL, APACHE-2.0" data-author="dummy"></p>
<p class="card block-card" data-url="detections_stabilizer" data-name="Detections Stabilizer" data-desc="Apply smoothing algorithm to reduce noise and flickering across video frames" data-labels="TRANSFORMATION, APACHE-2.0" data-author="dummy"></p>
<p class="card block-card" data-url="stitch_images" data-name="Stitch Images" data-desc="Stitch two images by common parts." data-labels="TRANSFORMATION, APACHE-2.0" data-author="dummy"></p>
<p class="card block-card" data-url="stitch_ocr_detections" data-name="Stitch OCR Detections" data-desc="Combines OCR detection results into a coherent text string by organizing detections spatially." data-labels="TRANSFORMATION, APACHE-2.0" data-author="dummy"></p>
<p class="card block-card" data-url="template_matching" data-name="Template Matching" data-desc="Looks for instances of template in specific image" data-labels="CLASSICAL_COMPUTER_VISION, APACHE-2.0" data-author="dummy"></p>
<p class="card block-card" data-url="timeinzone" data-name="Time in zone" data-desc="Track duration of time spent by objects in zone" data-labels="ANALYTICS, APACHE-2.0" data-author="dummy"></p>
<p class="card block-card" data-url="triangle_visualization" data-name="Triangle Visualization" data-desc="Draws triangle markers on an image at specific coordinates based on provided detections." data-labels="VISUALIZATION, APACHE-2.0" data-author="dummy"></p>
Expand Down
59 changes: 59 additions & 0 deletions docs/workflows/blocks_bundling.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,65 @@ REGISTERED_INITIALIZERS = {
}
```

## Serializers and deserializers for *Kinds*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like a really powerful and extensible pattern. Wondering if it also opens the door for e.g. kind conversion/casting via serialize as one kind and deserialize as other

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes as an opportunity to hook up this extension
no if you ask if this is possible without any changes to the PR
and probably no - if the question is broader - the polymorphism in kinds type system - that would need to be approached from different angle


Support for custom serializers and deserializers was introduced in Execution Engine `v1.3.0`.
From that version onward it is possible to point custom functions that
Execution Engine should use to serialize and deserialize any *[kind](/workflows/kinds/)*.

Deserializers will determine how to decode inputs send through the wire
into internal data representation used by blocks. Serializers, on the other hand,
are useful when Workflow results are to be send through the wire.

Below you may find example on how to add serializer and deserializer
for arbitrary kind. The code should be placed in main `__init__.py` of
your plugin:

```python
from typing import Any

def serialize_kind(value: Any) -> Any:
# place here the code that will be used to
# transform internal Workflows data representation into
# the external one (that can be sent through the wire in JSON, using
# default JSON encoder for Python).
pass


def deserialize_kind(parameter_name: str, value: Any) -> Any:
# place here the code that will be used to decode
# data sent through the wire into the Execution Engine
# and transform it into proper internal Workflows data representation
# which is understood by the blocks.
pass


KINDS_SERIALIZERS = {
"name_of_the_kind": serialize_kind,
}
KINDS_DESERIALIZERS = {
"name_of_the_kind": deserialize_kind,
}
```

### Tips And Tricks

* Each serializer must be a function taking the value to serialize
and returning serialized value (accepted by default Python JSON encoder)

* Each deserializer must be a function accepting two parameters - name of
Workflow input to be deserialized and the value to be deserialized - the goal
of the function is to align input data with expected internal representation

* *Kinds* from `roboflow_core` plugin already have reasonable serializers and
deserializers

* If you do not like the way how data is serialized in `roboflow_core` plugin,
feel free to alter the serialization methods for *kinds*, simply registering
the function in your plugin and loading it to the Execution Engine - the
serializer/deserializer defined as the last one will be in use.


## Enabling plugin in your Workflows ecosystem

To load a plugin you must:
Expand Down
Loading