Snowflake operators in deferred mode generate JWT tokens on every poll instead of reusing them

[manipatnam]

logs_test_dbt_multi_repo_v1_manual__2025-12-13T09_15_44+00_00_snow_-1_1.txt

Apache Airflow Provider(s)

snowflake

Versions of Apache Airflow Providers

apache-airflow-providers-snowflake==6.7.0

Apache Airflow version

3.1.4

Operating System

debian

Deployment

Astronomer

Deployment details

No response

What happened

When using SnowflakeSqlApiOperator in deferred mode (deferrable=True), the operator generates a new JWT token on every polling attempt instead of reusing the existing token. This causes:

1. Performance overhead: Unnecessary token generation on every poll (which can happen every few seconds)
2. Log pollution: INFO level logs are generated on every poll, flooding the task logs and making it difficult to track actual task progress

Current Behavior

In deferred mode, when the operator polls for query status, it generates a new JWT token for each poll attempt, even though the previous token is still valid. This results in:

• Excessive token generation operations
• Cluttered logs with repeated JWT generation messages

What you think should happen instead

The JWT token should be reused across polling attempts until it expires or needs renewal. Token generation should only occur when:

• No token exists yet
• The token has expired
• The token renewal time has been reached

How to reproduce

1. Create a SnowflakeSqlApiOperator with deferrable=True
2. Execute a long-running query that requires multiple polls
3. Observe the logs - you'll see JWT token generation logs on every poll
4. Check the task logs - they will be cluttered with JWT generation messages

Anything else

Impact

• Performance: Unnecessary cryptographic operations on every poll
• Log pollution: INFO level logs on every poll make it difficult to track actual task progress
• Resource waste: CPU cycles spent on token generation that could be avoided

Additional Context

The token caching mechanism exists but is not being utilized effectively in deferred mode.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[jroachgolf84]

Looks like this snippet of code... Doesn't actually seem like that hook is being used once it's created. But I could be missing something...

https://github.com/apache/airflow/blob/e1adf0b1eba8fb5e00046cf49f15da3de59ff705/providers/snowflake/src/airflow/providers/snowflake/triggers/snowflake_trigger.py#L70C1-L75C10

[jroachgolf84]

        """Wait for the query the snowflake query to complete."""
        SnowflakeSqlApiHook(
            self.snowflake_conn_id,
            self.token_life_time,
            self.token_renewal_delta,
        )

[jroachgolf84]

I've created a PR for this: #59470