Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix timetable generation for StopTimeUpdate containing only arrival or departure #6392

Open
wants to merge 5 commits into
base: dev-2.x
Choose a base branch
from

Conversation

miklcct
Copy link
Contributor

@miklcct miklcct commented Jan 17, 2025

Summary

The GTFS-RT specification does not require both arrival and departure to be provided for a TripUpdate, only either of them is enough. This fixes NEGATIVE_DWELL_TIME or NEGATIVE_HOP_TIME resulting from these updates as they were not processed correctly.

Issue

Closes #6391

Unit tests

Added

Documentation

None needed

Changelog

Bumping the serialization version id

None needed

@miklcct miklcct force-pushed the arrival-only-tripupdate branch from 0541650 to 7a96e1a Compare January 17, 2025 11:58
@miklcct miklcct changed the title Fix NEGATIVE_DWELL_TIME for arrival only StopTimeUpdate Fix NEGATIVE_HOP_TIME for arrival only StopTimeUpdate Jan 17, 2025
@miklcct miklcct force-pushed the arrival-only-tripupdate branch from 7a96e1a to f60c1db Compare January 17, 2025 13:11
@miklcct miklcct force-pushed the arrival-only-tripupdate branch from f60c1db to 863c0cd Compare January 17, 2025 13:27
@miklcct miklcct changed the title Fix NEGATIVE_HOP_TIME for arrival only StopTimeUpdate Fix timetable generation for StopTimeUpdate containing only arrival or departure Jan 17, 2025
@miklcct miklcct marked this pull request as ready for review January 17, 2025 15:47
@miklcct miklcct requested a review from a team as a code owner January 17, 2025 15:47
@miklcct
Copy link
Contributor Author

miklcct commented Jan 17, 2025

I need discussion of the behavior in filling the missing times.

Co-authored-by: Thomas Gran <[email protected]>
Copy link

codecov bot commented Jan 28, 2025

Codecov Report

Attention: Patch coverage is 84.21053% with 3 lines in your changes missing coverage. Please review.

Project coverage is 69.84%. Comparing base (c8fbf02) to head (e29e4a1).

Files with missing lines Patch % Lines
...main/java/org/opentripplanner/model/Timetable.java 84.21% 1 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             dev-2.x    #6392   +/-   ##
==========================================
  Coverage      69.84%   69.84%           
- Complexity     18125    18132    +7     
==========================================
  Files           2069     2069           
  Lines          77268    77279   +11     
  Branches        7855     7861    +6     
==========================================
+ Hits           53965    53975   +10     
  Misses         20546    20546           
- Partials        2757     2758    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@t2gran t2gran added this to the 2.7 (next release) milestone Jan 29, 2025
.addStop(STOP_A1, "0:01:00", "0:01:01")
.addStop(STOP_B1, "0:01:10", "0:01:11")
.addStop(STOP_C1, "0:01:20", "0:01:21")
.addStop(STOP_A1, "0:00:00", "0:00:00")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't modify existing tests and create new ones instead that test this very feature.

@@ -312,7 +312,7 @@ public void fixIncoherentTimes() {
stopTimeUpdateBuilder.setStopSequence(1);
stopTimeUpdateBuilder.setScheduleRelationship(StopTimeUpdate.ScheduleRelationship.SCHEDULED);
var stopTimeEventBuilder = stopTimeUpdateBuilder.getArrivalBuilder();
stopTimeEventBuilder.setDelay(0);
stopTimeEventBuilder.setDelay(900);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this test fail if you don't change it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The time will be valid after my code change because the bus can make up time during its journey.

newTimes.updateDepartureDelay(i, delay);
}

// propagate arrival and departure times, taking care not to cause negative dwells / hops
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're mixing several features here, aren't you?

Can you please work in small increments? The first PR should only deal with the case where either arrival or departure is missing. In such a case you can use the other one in its place. This shouldn't need any new interpolation logic.

Once we have finished that PR we can look at more complex cases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I don't get it then and I think I need to see drawing then.

Using an arrival/departure fallback is, IMO, still a wise first step and something I would like to see first, even if it doesn't fix all of your problems.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing interpolation logic isn't what you describe. The existing logic is that if the departure time is missing, it is assumed that the bus will always stop for its original dwell time even if it is late (resulting in negative times down the trip), so what I have written and what you are describing are both changing the logic, which is an integral part of this PR.

@miklcct
Copy link
Contributor Author

miklcct commented Feb 4, 2025

To explain matter, the following is how OTP fails and my proposed change:

    GTFS static   GTFS-RT feed   Current behavior       Proposed behavior       Remarks
Stop Timepoint Sch. Arrival Sch. Departure Est. Arrival Est. Departure OTP arrival delay OTP departure delay OTP arrival delay OTP departure delay  
A TRUE 12:00:00 12:00:00 12:08:00   12:08:00 00:08:00 12:08:00 00:08:00 12:08:00 00:08:00 12:08:00 00:08:00  
B FALSE 12:02:00 12:02:00 12:10:00   12:10:00 00:08:00 12:10:00 00:08:00 12:10:00 00:08:00 12:10:00 00:08:00  
C TRUE 12:05:00 12:10:00 12:13:00   12:13:00 00:08:00 12:18:00 00:08:00 12:13:00 00:08:00 12:13:00 00:03:00 OTP currently assumes that a bus will not attempt to catch up delay by reducing dwell, causing negative run time to the next stop
D FALSE 12:13:00 12:13:00 12:16:00   Neg. run time       12:16:00 00:03:00 12:16:00 00:03:00  
E TRUE 12:15:00 12:15:00 12:18:00           12:18:00 00:03:00 12:18:00 00:03:00  
    GTFS static   GTFS-RT feed   Current behavior       Proposed behavior        
Stop Timepoint Sch. Arrival Sch. Departure Est. Arrival Est. Departure OTP arrival delay OTP departure delay OTP arrival delay OTP departure delay  
A TRUE 12:00:00 12:00:00   12:05:00 12:00:00 00:00:00 12:05:00 00:05:00 12:00:00 00:00:00 12:05:00 00:05:00  
B FALSE 12:02:00 12:02:00   12:06:30 12:07:00 00:05:00 Neg. dwell time   12:06:30 00:04:30 12:06:30 00:04:30 OTP currently assume that a bus always will run according to the speed in the timetable, even if the prediction thinks otherwise
C TRUE 12:05:00 12:10:00   12:10:00         12:09:30 00:04:30 12:10:00 00:00:00  
D FALSE 12:13:00 12:13:00   12:13:00         12:13:00 00:00:00 12:13:00 00:00:00  
E TRUE 12:15:00 12:15:00   12:15:00         12:15:00 00:00:00 12:15:00 00:00:00  
    GTFS static   GTFS-RT feed   Current behavior       Proposed behavior       Remarks
Stop Timepoint Sch. Arrival Sch. Departure Est. Arrival Est. Departure OTP arrival delay OTP departure delay OTP arrival delay OTP departure delay  
A TRUE 12:00:00 12:00:00 12:00:00   12:00:00 00:00:00 12:00:00 00:00:00 12:00:00 00:00:00 12:00:00 00:00:00  
B FALSE 12:02:00 12:02:00 12:01:30   12:01:30 -00:00:30 12:01:30 -00:00:30 12:01:30 -00:00:30 12:01:30 -00:00:30  
C TRUE 12:05:00 12:10:00 12:04:15   12:04:15 -00:00:45 12:09:15 -00:00:45 12:04:15 -00:00:45 12:10:00 00:00:00 OTP currently blindly propagate the same delay (earliness) disregarding timepoint reality
D FALSE 12:13:00 12:13:00 12:12:55   12:12:55 -00:00:05 12:12:55 -00:00:05 12:12:55 -00:00:05 12:12:55 -00:00:05  
E TRUE 12:15:00 12:15:00 12:14:50   12:14:50 -00:00:10 12:14:50 -00:00:10 12:14:50 -00:00:10 12:15:00 00:00:00  
    GTFS static   GTFS-RT feed   Current behavior       Proposed behavior       Remarks
Stop Timepoint Sch. Arrival Sch. Departure Est. Arrival Est. Departure OTP arrival delay OTP departure delay OTP arrival delay OTP departure delay  
A TRUE 12:00:00 12:00:00   12:00:00 12:00:00 00:00:00 12:00:00 00:00:00 12:00:00 00:00:00 12:00:00 00:00:00  
B FALSE 12:02:00 12:02:00   12:01:45 12:00:00 -00:02:00 Neg. dwell time   12:01:45 -00:00:15 12:01:45 -00:00:15 OTP currently assume that a bus always will run according to the speed in the timetable, even if the prediction thinks otherwise
C TRUE 12:05:00 12:10:00   12:10:00         12:04:45 -00:00:15 12:10:00 00:00:00  
D FALSE 12:13:00 12:13:00   12:12:45         12:12:45 -00:00:15 12:12:45 -00:00:15  
E TRUE 12:15:00 12:15:00   12:15:00         12:14:45 -00:00:15 12:15:00 00:00:00  

@leonardehrenfried
Copy link
Member

Thanks for those tables. I think I get it now.

I'm not enthusiastic about making a complicated method even more complicated but if you can write separate module tests for this logic I will soften my position.

One point remains: I'm aware that they are not doing anything that is explicitly disallowed by the spec, but they could make everyone's live easier if they just specified the exact arrival and departure time. Do we want to pay the cost of the complexity to allow them to keep doing what they are doing? Lets discuss this in the dev meeting.

@miklcct
Copy link
Contributor Author

miklcct commented Feb 6, 2025

As discussed in the meeting we should provide options for the deployment how to fill in the missing times.

  • The original behavior: assumes the delay always propagate without change
  • My proposed behavior: assumes a late bus will try to catch up by zeroing dwells until it gets back on time, and an early bus will wait at the next timepoint, otherwise propagate the delay
  • A third alternative: always assume that the dwell is zero if only one of arrival / departure is specified, and fall back to one of the above on no-data stops.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants