Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Velocity block (w/ Byte Tracker) #754

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Velocity block (w/ Byte Tracker) #754

wants to merge 7 commits into from

Conversation

reedajohns
Copy link
Collaborator

Description

Implemented VelocityBlockV1:

  • Developed a new analytics block to calculate and embed velocities and speeds of Byte Tracked objects across video frames.
  • Supports smoothing of velocity measurements using an exponential moving average with configurable smoothing_alpha.
  • Incorporated unit conversion from pixels to meters via the pixels_per_meter parameter (e.g., pixels_per_meter=1000 for 1000 pixels = 1 meter).

Key Features:

  • Velocity Calculation: Computes raw and smoothed velocities based on object movements between frames.
  • Speed Calculation: Determines the magnitude of velocity vectors to obtain speed.

List any dependencies that are required for this change.

Screenshot 2024-10-18 at 1 52 53 PM

This requires the input of Byte Track detections and appends the velocity / speed to the detections metadata for use downstream:

Example:
'velocity': {4: [-9.669, 14.509]}, 'speed': {4: 17.432}, 'smoothed_velocity': {4: [-6.033, 9.555]}, 'smoothed_speed': {4: 11.300}}

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

How has this change been tested, please provide a testcase or example of how you tested the change?

Unit Tests:

  • Test Scenarios:
    • Basic Calculations: Verified accurate velocity and speed computations across consecutive frames.
    • New Tracker Handling: Ensured the block correctly initializes velocities for newly appearing objects.
    • Error Conditions: Confirmed that invalid inputs (e.g., missing tracker_id, invalid smoothing_alpha, non-positive pixels_per_meter) raise appropriate ValueError exceptions.
    • Edge Cases: Tested scenarios such as zero time difference between frames and large object movements to validate robustness.

}
)
type: Literal["roboflow_core/velocity@v1"]
metadata: WorkflowVideoMetadataSelector
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

)
pixels_per_meter: Union[float, WorkflowParameterSelector(kind=[FLOAT_KIND])] = Field( # type: ignore
default=1.0,
description="Conversion from pixels to meters. Velocity will be converted to meters per second using this value.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add disclaimer, that this might not apply to all scenes and/or all movement directions (due to perspective and camera distortions; i.e. for some field of view 5px far away will result in different speed than 5px near the camera)

description="Predictions",
examples=["$steps.object_detection_model.predictions"],
)
smoothing_alpha: Union[float, WorkflowParameterSelector(kind=[FLOAT_KIND])] = Field( # type: ignore
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have StabilizeTrackedDetectionsBlock which might be already delivering some functionality that is being achieved here

Also - you can use FLOAT_ZERO_TO_ONE_KIND if this parameter is expected to only store values between (0, 1) - that way the value of parameter will be checked during compilation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants