Schedules

In cases where the pipeline should be run periodically, for example, to extract data daily, Schedules can be created.

A new Schedule can be created by first selecting the desired transformations to be run and then choosing a name for the Schedule, and then clicking the Create button in the Schedule panel.

Please note that only users with the EDITOR role or higher are authorized to create new schedules.

After a schedule is created, certain properties can be modified.

Schedule Name

The Schedule can be renamed by double-clicking on the Schedule name.

Pause/Resume

The button next to the Schedule name allows the schedules to be deactivated without deleting the entire Schedule.

Cron

The Cron value specifies the frequency of the run and is based on the standard cron notation. For more information, see the wiki article.

Next Run

Next Run displays the next time this schedule will run.

Transforms

The transforms panel inside the Schedule widget shows all target transforms that are associated with the current Schedule. Here, transforms can be added or removed.

Before a schedule executes, the list of target transforms is evaluated as a DAG and executed sequentially according to the dependency graph.

Failing Schedules

To prevent unnecessary resource waste, the Schedules will be disabled after 3 consecutive build failures, and the creator of the schedule will be notified via email.

This behavior can be adjusted with the MAX_SCHEDULE_FAIL_COUNT property in the .env file.