Schedules don't work well with time zones

[strickvl]

Open Source Contributors Welcomed!

Please comment below if you would like to work on this issue!

Contact Details [Optional]

support@zenml.io

What happened?

There is an issue with Schedules not aligning properly with different time zones. When a pipeline is scheduled with a certain cron expression, it does not run at the expected local time due to time zone differences. For instance, a pipeline scheduled with the cron expression 0 9 * * * actually ran at 11 AM instead of the expected 9 AM.

Reproduction steps

This issue can be reproduced by setting up a pipeline with a specific cron expression and observing the time it actually runs, which differs due to the time zone. Here's an example:

from zenml.config.schedule import Schedule
from zenml.pipelines import pipeline

@pipeline
def my_pipeline(...):
    ...

schedule = Schedule(cron_expression="0 9 * * *")
my_pipeline = my_pipeline.with_options(schedule=schedule)
my_pipeline.run()

Expected behavior

The scheduled pipelines should run according to the local time zone of the user or the server where ZenML is running, or there should be clear documentation and warning messages indicating the time zone in which the schedules are interpreted.

Potential Solutions

Implement a technical solution that adjusts the scheduling time based on the user's or server's local time zone.
Provide a warning or error message in the ZenML interface to indicate the time zone used for scheduling.
Add documentation specifying the behavior of the scheduler with regards to time zones, especially if it runs on UTC time by default.

Additional context

This issue is crucial for users who rely on precise scheduling for their pipelines, particularly in a global context where users may be in different time zones.

Code of Conduct

• I agree to follow this project's Code of Conduct

[ashutosh887]

@strickvl

I would like to work on this Issue!
Please assign...

[strickvl]

@ashutosh887 you're welcome to do so. Might be worth picking one of the two you mentioned and starting with that one first, but go for it! Let us know if you have any questions or to propose how you might tackle the issue (once you've dived into it a bit).

[smokeyScraper]

is anyone working on this issue?
I would like to work on it

[shivanjay-wagh]

if this is still open, i would like to work on this

[preyasshah9]

Hi @strickvl

If this issue is still open, I would like to work on it.
I do have a few clarification questions.

• When you schedule 0 9 * * *, why is it scheduled at 11 am? Is it because the server is configured for different time zone compared to the local time zone (of users machine) and that is causing this discrepancy? Would like to understand the issue clearly.
• There are a few deployments supported with zenml: AWS SageMaker, Databricks, Kubernetes. Depending upon the Pipeline, is the cron execution based on UTC, or local time zone? I'm not 💯 sure, but just wanted to understand if that also could result in this issue?
• One of the potential idea I have is to convert the local time to UTC using Pydantic Validator on cron_expression on the Schedule class.

[schustmi]

@preyasshah9 We've decided that this is actually best solved by explaining that cron schedules are usually treated as UTC timezone in the docstrings and our documentation. These should also point out that depending on which orchestrator is used in ZenML (e.g. SageMaker, Vertex), the cron schedule timezone might be configured server-side outside of ZenMLs control. This means any potential conversion we do on the cron expresssion is not guaranteed to work, and even worse might confuse users that don't expect them to be modified.

[preyasshah9]

@schustmi Thank you for the clarification! Happy to help out in the documentation!