Skip to content

Conversation

@Mantisus
Copy link
Collaborator

Description

  • Protect Request and Session from partial mutations on request handler failure relevant with AdaptivePlaywrightCrawler

Issues

@Mantisus Mantisus requested review from Pijukatel and janbuchar and removed request for janbuchar November 30, 2025 00:55
@Mantisus Mantisus self-assigned this Nov 30, 2025
@Mantisus Mantisus marked this pull request as draft November 30, 2025 01:33
@Mantisus Mantisus marked this pull request as ready for review December 3, 2025 01:22
@Pijukatel
Copy link
Collaborator

I am not sure about doing this due to some race conditions. My concern is reverting the changes made to the session by parallel handlers.

Scenario with 2 handlers running at the same time and both using the same session:

Both handlers create their copy of the session. (Identical at this point)
Both handlers modify the session.
One of the handlers fails and reverts both modifications.

Also, we should keep JS version aligned, so please add a note here if any changes should be made there as well:
apify/crawlee#2798

@Mantisus
Copy link
Collaborator Author

Mantisus commented Dec 3, 2025

I am not sure about doing this due to some race conditions. My concern is reverting the changes made to the session by parallel handlers.

Yes, I am also concerned about the race condition in the session for some cases. But I am not yet sure about the best approach to avoid it.

One of the handlers fails and reverts both modifications.

This should not revert both modifications.
The failed handler will continue to work with the original session.
The successful handler will apply the update from the copy to the original.

I will try to add a test that reproduces this behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Track changes made to request and session in RequestHandlerResult

2 participants