Bug fixes

Fixes a problem in actions/__init__.py which imported the deprecated Verify class. Small updates to tests for the task interface.

Major overhaul

• A massive update and overhaul of documentation, which can be now found under examples.

This release includes several breaking changes:

• The following pre-defined interfaces have been removed: AddOutlines, LabelledSegmentationVerification, FixImageSegmentation, MulticlassVerification and LabelledSegmentationVerificationNoCheckbox.
  • The same functionalities are now implemented in fewer interfaces with additional components that may be invoked through the YAML configuration file.
• The Forward Action has been reworked.
  • The variable name containing the incoming data must now be defined under the key data, whereas the actions for processing incoming data are defined under the key on_result.
  • The top-level key messages must be included for any outputs that reject or accept assignments. These messages are added to the assignments so that the workers can know why their work was accepted or rejected.
• The VerifyPolygon Action now uses the top-level key data to define the variable that contains the polygons to be verified.
• The Verify Action has been dropped. The same functionality can be found in the Forward Action.

Prevent Pipelines from exiting after completion

Adds a no_exit flag (boolean) to the Pipeline object. If set to True, the program won't exit after completing the pipeline.

Add an Action for validating polygons and bounding box labels

This release adds a new Action named VerifyPolygon, which validates the bounding boxes created by crowdsourced workers.

The polygons are validated using Shapely, which is now required by the tool.

For an example of using this Action, see examples/verify_demo.py.

Remove support for manual CAPTCHAs

This release removes support for the manual configuration of CAPTCHAs on Toloka, which will be deprecated in the future. For a description of the new system, see this article: https://toloka.ai/blog/new-captcha/

Support for golden tasks

This release adds support for golden tasks.

A demo can be found under examples/gold_demo.py.

Essentially, golden tasks are defined in the YAML configuration under the key data/gold, which should contain key/value pairs for output columns and the columns that contain the golden answers in the TSV file.

To exemplify, if you have golden answers for the output named result, which are stored in a column with the header gold in the TSV file, the key/value pair result: gold.

New interface for verifying bounding boxes

Adds a new task class LabelledSegmentationVerificationNoCheckbox, which is essentially the same as LabelledSegmentationVerification but has no checkbox in the interface, which can be used to skip e.g. images that do not contain bounding boxes.

Minor improvements

Allow user-defined prompts for checkboxes in user interface.

Updates to task interfaces

This release makes the task interfaces more user-friendly by replacing the min_width parameter in the ImageAnnotationFieldV1 class with the parameter full_height.

Minor improvements

Adds an error message for a missing training key under the main pool configuration, which is needed for linking the training and main pools.