Skip to content

Daft 0.5.7 having issues with autoscaling in Ray cluster #4691

Closed
@akshayah3

Description

@akshayah3

Describe the bug

Hi all. I am getting the below error when I am submitting a job to ray using the latest daft version (0.5.7). Ray version (2.47.1)

  File "/tmp/ray/session_2025-07-01_05-24-10_104053_1/runtime_resources/pip/69d4bc1d80e2ebaf3b261d5076621f8d79995eb8/virtualenv/lib/python3.11/site-packages/daft/runners/flotilla.py", line 207, in get_next_partition
    next_partition_ref = await self.curr_result_gens[plan_id].__anext__()
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'daft.runners.flotilla' has no attribute 'try_autoscale'

To Reproduce

Part of my code below. Basically trying to read data from s3 parquet and writing it to iceberg

from daft.daft import S3Config, IOConfig

s3_config_from_env = S3Config.from_env()
io_config = IOConfig(s3=s3_config_from_env)
df = daft.read_parquet(parquet_path, io_config=io_config)
df.write_iceberg(new_table, mode="overwrite")

Expected behavior

No response

Component(s)

Ray Runner

Additional context

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingp1Important to tackle soon, but preemptable by p0

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions