DAG: crawlers

schedule: @daily

Task Instance: Wait

Task Instance Details

Dependencies Blocking Task From Getting Scheduled
Dependency Reason
Trigger Rule Task's trigger rule 'all_success' requires all upstream tasks to have succeeded, but found 42 non-success(es). upstream_tasks_state={'total': 42, 'successes': 0, 'skipped': 0, 'failed': 0, 'upstream_failed': 0, 'done': 0}, upstream_task_ids={'indeed_se-crawl', 'snagajobs-crawl', 'indeed_it-crawl', 'indeed_ua-crawl', 'optioncar-crawl', 'indeed_au-crawl', 'reeduk-crawl', 'indeed_eg-crawl', 'indeed_fr-crawl', 'indeed_uk-crawl', 'opcionempleo-crawl', 'indeed_es-crawl', 'glassdoor-crawl', 'indeed_pl-crawl', 'indeed_malaysia-crawl', 'dice-crawl', 'flexjobs-crawl', 'indeed_ar-crawl', 'dejobs-crawl', 'careerbuilder-crawl', 'ladders-crawl', 'indeed_canada-crawl', 'indeed_in-crawl', 'indeed_fi-crawl', 'indeed_ru-crawl', 'indeed_korea-crawl', 'indeed_china-crawl', 'polemployee-crawl', 'seek_au-crawl', 'indeed_ie-crawl', 'indeed_mx-crawl', 'indeed_nl-crawl', 'indeed_pt-crawl', 'indeed_pk-crawl', 'indeed_brasil-crawl', 'indeed-crawl', 'indeed_sa-crawl', 'indeed_ro-crawl', 'crypto-crawl', 'whoishiring-crawl', 'indeed_gr-crawl', 'indeed_tw-crawl'}
Dag Not Paused Task's DAG 'crawlers' is paused.
Task Instance Attributes
Attribute Value
dag_id crawlers
duration None
end_date None
execution_date 2025-05-28T02:01:54.110596+00:00
executor_config {}
generate_command <function TaskInstance.generate_command at 0x7fc36a440b70>
hostname
is_premature False
job_id None
key ('crawlers', 'Wait', <Pendulum [2025-05-28T02:01:54.110596+00:00]>, 1)
log <Logger airflow.task (INFO)>
log_filepath /usr/local/airflow/logs/crawlers/Wait/2025-05-28T02:01:54.110596+00:00.log
log_url http://localhost:8080/admin/airflow/log?dag_id=crawlers&task_id=Wait&execution_date=2025-05-28T02%3A01%3A54.110596%2B00%3A00
logger <Logger airflow.task (INFO)>
mark_success_url http://localhost:8080/admin/airflow/success?task_id=Wait&dag_id=crawlers&execution_date=2025-05-28T02%3A01%3A54.110596%2B00%3A00&upstream=false&downstream=false
max_tries 2
metadata MetaData(bind=None)
next_try_number 1
operator None
pid None
pool general
previous_ti <TaskInstance: crawlers.Wait 2025-03-29 00:00:00+00:00 [success]>
priority_weight 2
queue default
queued_dttm None
raw False
run_as_user None
start_date None
state None
task <Task(DummyOperator): Wait>
task_id Wait
test_mode False
try_number 1
unixname airflow
Task Attributes
Attribute Value
adhoc False
dag <DAG: crawlers>
dag_id crawlers
depends_on_past True
deps {<TIDep(Previous Dagrun State)>, <TIDep(Trigger Rule)>, <TIDep(Not In Retry Period)>}
downstream_list [<Task(DummyOperator): End>]
downstream_task_ids {'End'}
email ['airflow@airflow.com']
email_on_failure False
email_on_retry False
end_date None
execution_timeout None
executor_config {}
inlets []
lineage_data None
log <Logger airflow.task.operators (INFO)>
logger <Logger airflow.task.operators (INFO)>
max_retry_delay None
on_failure_callback None
on_retry_callback None
on_success_callback None
outlets []
owner airflow
params {}
pool general
priority_weight 1
priority_weight_total 2
queue default
resources {'cpus': {'_name': 'CPU', '_units_str': 'core(s)', '_qty': 1}, 'ram': {'_name': 'RAM', '_units_str': 'MB', '_qty': 512}, 'disk': {'_name': 'Disk', '_units_str': 'MB', '_qty': 512}, 'gpus': {'_name': 'GPU', '_units_str': 'gpu(s)', '_qty': 0}}
retries 2
retry_delay 0:30:00
retry_exponential_backoff False
run_as_user None
schedule_interval 0 0 * * *
sla None
start_date 2020-12-17T00:00:00+00:00
task_concurrency None
task_id Wait
task_type DummyOperator
template_ext []
template_fields ()
trigger_rule all_success
ui_color #e8f7e4
ui_fgcolor #000
upstream_list [<Task(DockerOperator): indeed_se-crawl>, <Task(DockerOperator): snagajobs-crawl>, <Task(DockerOperator): indeed_it-crawl>, <Task(DockerOperator): indeed_ua-crawl>, <Task(DockerOperator): optioncar-crawl>, <Task(DockerOperator): indeed_au-crawl>, <Task(DockerOperator): reeduk-crawl>, <Task(DockerOperator): indeed_eg-crawl>, <Task(DockerOperator): indeed_fr-crawl>, <Task(DockerOperator): indeed_uk-crawl>, <Task(DockerOperator): opcionempleo-crawl>, <Task(DockerOperator): indeed_es-crawl>, <Task(DockerOperator): glassdoor-crawl>, <Task(DockerOperator): indeed_pl-crawl>, <Task(DockerOperator): indeed_malaysia-crawl>, <Task(DockerOperator): dice-crawl>, <Task(DockerOperator): flexjobs-crawl>, <Task(DockerOperator): indeed_ar-crawl>, <Task(DockerOperator): dejobs-crawl>, <Task(DockerOperator): careerbuilder-crawl>, <Task(DockerOperator): ladders-crawl>, <Task(DockerOperator): indeed_canada-crawl>, <Task(DockerOperator): indeed_in-crawl>, <Task(DockerOperator): indeed_fi-crawl>, <Task(DockerOperator): indeed_ru-crawl>, <Task(DockerOperator): indeed_korea-crawl>, <Task(DockerOperator): indeed_china-crawl>, <Task(DockerOperator): polemployee-crawl>, <Task(DockerOperator): seek_au-crawl>, <Task(DockerOperator): indeed_ie-crawl>, <Task(DockerOperator): indeed_mx-crawl>, <Task(DockerOperator): indeed_nl-crawl>, <Task(DockerOperator): indeed_pt-crawl>, <Task(DockerOperator): indeed_pk-crawl>, <Task(DockerOperator): indeed_brasil-crawl>, <Task(DockerOperator): indeed-crawl>, <Task(DockerOperator): indeed_sa-crawl>, <Task(DockerOperator): indeed_ro-crawl>, <Task(DockerOperator): crypto-crawl>, <Task(DockerOperator): whoishiring-crawl>, <Task(DockerOperator): indeed_gr-crawl>, <Task(DockerOperator): indeed_tw-crawl>]
upstream_task_ids {'indeed_se-crawl', 'snagajobs-crawl', 'indeed_it-crawl', 'indeed_ua-crawl', 'optioncar-crawl', 'indeed_au-crawl', 'reeduk-crawl', 'indeed_eg-crawl', 'indeed_fr-crawl', 'indeed_uk-crawl', 'opcionempleo-crawl', 'indeed_es-crawl', 'glassdoor-crawl', 'indeed_pl-crawl', 'indeed_malaysia-crawl', 'dice-crawl', 'flexjobs-crawl', 'indeed_ar-crawl', 'dejobs-crawl', 'careerbuilder-crawl', 'ladders-crawl', 'indeed_canada-crawl', 'indeed_in-crawl', 'indeed_fi-crawl', 'indeed_ru-crawl', 'indeed_korea-crawl', 'indeed_china-crawl', 'polemployee-crawl', 'seek_au-crawl', 'indeed_ie-crawl', 'indeed_mx-crawl', 'indeed_nl-crawl', 'indeed_pt-crawl', 'indeed_pk-crawl', 'indeed_brasil-crawl', 'indeed-crawl', 'indeed_sa-crawl', 'indeed_ro-crawl', 'crypto-crawl', 'whoishiring-crawl', 'indeed_gr-crawl', 'indeed_tw-crawl'}
wait_for_downstream False
weight_rule downstream