![]() I want the DAG to only run when its interval has come. So, why is the 5th run taking place for interval T02:30:00+00:00 to T02:30:00+00:00 even though the interval has passed? All times below are in UTC Dag Start_date 1st run(skip catchup=false) 2nd run(skip catchup=false) 3rd run(skip catchup=false) 4th run(skip catchup=false) What am I missing here? I can't wrap my head around why the scheduler is doing thisįollowing is my understanding after reading all the documentation, please do correct me if I'm wrong.ĭAG picked up by scheduler at T011:30:00+00:00 UTC, ideally it should run at T02:30:00+00:00 UTC according to the dag config. ![]() The weird thing is that when I deploy the container with the dag to the edge device the dag automatically starts its first run outside the scheduled interval, even though that interval has passed! By default it is set to utc, but you change it to use the system’s settings or an arbitrary IANA time zone, e.g. Cloud Composer is a fully managed workflow orchestration service, enabling you to create, schedule, monitor, and manage workflow pipelines that span across clouds and on-premises data centers. I want this dag to basically schedule its run at 19:30 PST or 2:30 UTC, whenever the edge device is on, otherwise don't. You can also set it to system or an IANA time zone (e.g. This dag runs on an edge device, that edge device is sometimes on and sometimes off. Default time zone If you just installed Airflow it will be set to utc, which is recommended. Note that this run will have executiondate of 08:00. Schedule_interval="30 19 * * *", # 7:30 PM every day The run kicks in at the end of interval so startdate datetime (2020, 12, 8, 8, 0,0) and interval of 0 8 will end at 08:00 and that is when the first run will kick in. Start_date=pendulum.datetime(2021, 4, 13, 19, 30, tz='US/Pacific'), # set start_date in US/Pacific (PST) timezone My timezone-aware dag has a start date of 13-04-2021 19:30 PST or 14-04-2021 2:30 UTC and has the following configuration: # define DAG and its parameters ![]() Airflow scheduler kinda left me scratching my head for the past few days as it backfills dag runs even after catchup=False.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |