Task retries 3 times despite succeed

[akbarnotopb]

Apache Airflow version

Airflow 2.9.3 (google composer 2)

What happened?

tl;dr
My task’s first attempt ends with Marking task as SUCCESS but never shows Task exited with return code 0. Then Airflow retries it two more times. Why? Which component is actually failing?

I have a task that ended up running 3 times.

---

Attempt 1

• The log ends with:
  Marking task as SUCCESS
  but no Task exited with return code 0 (which normally appears).
• After that, there are no additional logs, and the scheduler decides to retry.

Attempt 2

This attempt looks normal:

[2025-11-27T09:08:35.327+0000] {taskinstance.py:1206} INFO - Marking task as SUCCESS. dag_id=etl_extras, task_id=raw_kafka, run_id=manual__2025-11-27T08:58:13.686272+00:00, execution_date=20251127T085813, start_date=20251127T090752, end_date=20251127T090835
[2025-11-27T09:08:39.026+0000] {local_task_job_runner.py:243} INFO - Task exited with return code 0
[2025-11-27T09:08:39.746+0000] {taskinstance.py:3506} INFO - 0 downstream tasks scheduled from follow-on schedule check
[2025-11-27T09:08:39.856+0000] {local_task_job_runner.py:222} INFO - ::endgroup::

But despite the normal exit, the scheduler triggered a third attempt.

Attempt 3

It immediately logs:

Dependencies not met for <TaskInstance: etl_extras.raw_kafka manual__2025-11-27T08:58:13.686272+00:00 [success]>, dependency 'Task Instance State' FAILED: Task is in the 'success' state.

---

Questions

1. What exactly happens internally after Airflow logs “Marking task as SUCCESS”?
2. Under what conditions can Airflow mark a task as success but still treat the attempt as incomplete or failed?
3. How can I prevent this from happening again?

What you think should happen instead?

The dag should run succesfully

How to reproduce

I have no idea (yet) how to reproduce this problem outside of our existing environments.

Thanks!