Behaviour and meaning of run_key

[the4thamigo-uk]

Hi, Im probably missing something here, but why does the run_key work differently for sensors than schedules. I would have thought that the run_key uniquely defines the run, so that if you launch a run with the same key multiple times, then it will be ignored, but it seems that the run happens multiple times with the same run_key... :/

[jamiedemaria]

You're right that run_key is intended to ensure that only one run with the same run_key is executed. However, for schedules, this is only scoped to a single tick. So for a particular tick of a schedule, you will only ever have one run with the same run_key executed, but at a later tick, the same run_key could be executed again

https://docs.dagster.io/_apidocs/schedules-sensors#dagster.RunRequest.run_key

[the4thamigo-uk]

Yeah that's what I read, but struggling to understand why you would even create two runs with the same key in a single tick anyway. I wonder what the design rationale is for this behaviour for schedules specifically...?

[erikvanzijst]

For those also running into this when switching a sensor to a schedule, you can put this in your schedule function to skip RunRequests for existing run_keys:

@schedule
def my_func(context: ScheduleEvaluationContext):
    run_key = ....
    if context.instance.get_runs(RunsFilter(tags={'dagster/run_key': run_key})):
        context.log.debug(f'Skipping previously processed key {run_key}')
    else:
        yield RunRequest(run_key=run_key, ...)