Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Store all drawn marks in the classification #5919

Conversation

eatyourgreens
Copy link
Contributor

After creating a new mark, add the new mark to the previous annotation value and store this as a new annotation. Should fix a bug raised in this comment: #5913 (comment)

You can try this out on How Did We Get Here, in the dev classifier. Draw on both frames of a subject, then check whether the drawn marks persist, in the classification, on the second step of the workflow.
https://local.zooniverse.org:8080/?project=communitiesandcrowds/how-did-we-get-here&env=production

Expected behaviour

When I create a new mark, classification.annotations should be updated to include all the marks that I've drawn for the current task.

Actual behaviour

When I draw a mark, the new drawing task annotation only contains mark IDs from the active frame. Mark IDs from other frames are silently dropped from the classification.

The marks themselves are still stored on their respective drawing tools, but the references to them are removed from the task annotation.

@coveralls
Copy link

coveralls commented Feb 15, 2024

Coverage Status

coverage: 81.214% (+0.008%) from 81.206%
when pulling dbe93b4 on eatyourgreens:fix-previous-drawn-marks
into 1be4462 on zooniverse:master.

@eatyourgreens
Copy link
Contributor Author

eatyourgreens commented Feb 15, 2024

@kieftrav @goplayoutside3 I don't have permission to request reviews or add labels to Zooniverse PRs, so I'm pinging you to take a look at this. There might be an urgent issue, affecting live transcription projects, where work is being lost.

I think it's worth double checking whether classifications are broken for live transcription projects, when you draw on more than one page of a subject. It might be that lines are only being saved for the current, active frame. I'm not 100% certain about that, because the code that serialises a classification (before POSTing it to the API) may actually loop through all drawing tools and marks, and serialise them to the saved classification.

From a quick re-read of the code, the saved classifier snapshot might only include mark IDs that were added to the transcription task annotation, so I'm concerned that volunteer work might have been lost. It depends whether annotation.toSnapshot() here includes all drawn marks, or only the marks that are referenced in annotation.value.

toSnapshot () {
let snapshot = getSnapshot(self)
let annotations = []
self.annotations.forEach(annotation => {
annotations = annotations.concat(annotation.toSnapshot())
})
snapshot = Object.assign({}, snapshot, { annotations })
return snapshot
},

@kieftrav
Copy link
Contributor

kieftrav commented Feb 15, 2024

@eatyourgreens @goplayoutside3 - this 1 line does fix the issue. I want to be clear that we are not losing any data upon classification submission.

@eatyourgreens
Copy link
Contributor Author

Excellent. If this is just a rendering bug, then that's less urgent.

@kieftrav
Copy link
Contributor

@eatyourgreens - 100% a rendering bug. I couldn't figure out where annotation.values['markID', 'markID'] was getting modified to only include the current frame's marks. That was the line that did it.

@eatyourgreens
Copy link
Contributor Author

eatyourgreens commented Feb 15, 2024

Drawing task annotations are slightly different from other task annotations. They use types.safeReference to hold references to marks.


Essentially, a drawing task annotation is an array of mark IDs. A side effect of this is that they're updated differently from other tasks, which hold a direct value rather than a reference.

I think this could be automated by having the drawing annotation model observe self.task.marks and update self.value whenever that changes. That would remove the manual update, which could be prone to breakage.

@eatyourgreens
Copy link
Contributor Author

self.task is a task key in the annotation model (the name is confusing, a legacy of older code.) It can be resolved to an actual task with something like this line.

const actualTask = resolveIdentifier(DrawingTask, getRoot(self), self.task)

Once you have the actual task, you can observe actualTask.marks for changes, as it's a MobX observable array. Then automate updating the stored annotation value whenever a mark is added or deleted.

@eatyourgreens eatyourgreens changed the title Store all drawn marks in the classification fix: Store all drawn marks in the classification Feb 15, 2024
@eatyourgreens
Copy link
Contributor Author

I've opened #5923 with those changes to the drawing annotation models. autorun might be more robust than manually updating annotations whenever a drawing tool changes, but it's a more involved change than the one-line fix here.

After creating a new mark, add the new mark to the previous annotation value and store this as a new annotation. Should fix a bug raised in this comment: zooniverse#5913 (comment)
@eatyourgreens
Copy link
Contributor Author

Closing this in favour of #5923.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants