Skip to content

Tracking Issue for #![feature(async_iterator)] #79024

Open
Listed in
@yoshuawuyts

Description

@yoshuawuyts
Member

This is a tracking issue for the RFC "2996" (rust-lang/rfcs#2996).
The feature gate for the issue is #![feature(async_iterator)].

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Steps

  • Implement the RFC (cc @rust-lang/XXX -- can anyone write up mentoring
    instructions?)
    Adjust documentation (see instructions on rustc-dev-guide)

Unresolved Questions

  • Clarify the panic behavior of Stream and Iterator Add core::stream::Stream #79023 (comment)
    • Add a panic section to Iterator, clarifying panic behavior. The panic behavior between Stream and Iterator should be consistent.
  • Restore Stream::next. This was removed from the RFC because it prevents dynamic dispatch, and subsequently removed from the implementation. This should be resolved before stabilizing.
    • Investigate whether we can move the trait from fn poll_next to async fn next once we can use async in traits.
  • Investigate as part of keyword-generics whether we can merge Iterator and AsyncIterator into a single trait which is generic over "asyncness".
    Should we name this API AsyncIterator instead? Tracking Issue for #![feature(async_iterator)] #79024 (comment)

Implementation history

Activity

added
C-tracking-issueCategory: An issue tracking the progress of sth. like the implementation of an RFC
on Nov 13, 2020
added
T-libs-apiRelevant to the library API team, which will review and decide on the PR/issue.
on Nov 13, 2020
added
AsyncAwait-TriagedAsync-await issues that have been triaged during a working group meeting.
on Dec 3, 2020
added
Libs-TrackedLibs issues that are tracked on the team's project board.
on Dec 16, 2020
added a commit that references this issue on Jan 30, 2021

Rollup merge of rust-lang#79023 - yoshuawuyts:stream, r=KodrAus

ecd7cb1
kaimast

kaimast commented on Feb 8, 2021

@kaimast

Is there a plan to include a core::stream::StreamExt (similar to the one in the futures crate) as well?

yoshuawuyts

yoshuawuyts commented on Feb 9, 2021

@yoshuawuyts
MemberAuthor

Is there a plan to include a core::stream::StreamExt (similar to the one in the futures crate) as well?

The plan is to add methods directly onto Stream much like methods exist on Iterator, but we want to do so in a way that won't cause ambiguities with ecosystem-defined methods in order to not accidentally break existing codebases when upgrading to newer Rust versions.

joshtriplett

joshtriplett commented on Apr 9, 2021

@joshtriplett
Member

Some discussion on Zulip raised the question of naming. With full acknowledgement to the fact that the name Stream has a long history in the async ecosystem, multiple people observed that something like AsyncIterator or similar might be much more evocative for new users. Such a name would allow people to map their existing understanding of iterators. "I understand Iterator, and I understand async, and this is an async version of Iterator".

brainstorm

brainstorm commented on Apr 10, 2021

@brainstorm

As a new, inexperienced user, I find AsyncIterator way more foreign than Stream, to be honest... perhaps I'm not too involved in compiler discussions and I don't see how the jargon clicks together though :-S ... also, blogposts are already being written about Stream, so changing the name mid-flight will only breed confusion, I reckon?

yoshuawuyts

yoshuawuyts commented on Apr 15, 2021

@yoshuawuyts
MemberAuthor

Prior art on "iterator" and "async iterator" naming schemes in other languages:

bbros-dev

bbros-dev commented on Aug 17, 2021

@bbros-dev

Context: We're prototyping some simple CLI app functionality. We're following the mini-redis example code where we can.

We've bumped out head on streams, and the need for the crates async-streams, and parallel-streams.

In our experience some *Iterator terminology would have helped clarify what streams are.

Given the need for the crates we cited, especially the async-streams RFC, we wonder if there isn't a need for:

  • Iterator
  • ConcurrentIterator
  • ParallelIterator

Or is the intention that async-stream and parallel-stream crate efforts all able to converge into a stream that covers the concurrent and parallel use cases?

benkay86

benkay86 commented on Aug 17, 2021

@benkay86

Rayon provides a whole ecosystem of parallel iterators on top of a work-stealing threadpool, and is currently the de facto Rust standard for parallel iteration. But I don't foresee a parallel iterator trait getting into the Rust standard library anytime soon.

Streams are supposed to cover the use case of concurrent iterators only (per my understanding). Hopefully once streams are stabilized into the Rust standard library we can have some syntax to use them in concurrent for loops just like we currently use synchronous iterators in for loops. However, there are still some issues to work out like what to do if a a stream panics or is dropped. Settling on a name (Stream vs AsyncIterator vs ConcurrentIterator) will be the easy part! 😜

Unfortunately, it's difficult to combine parallel and concurrent iteration at the moment. This would require Rayon to support a way to move tasks from worker threads to an async executor thread, or for an async executor like Tokio to support parallel thread pools in a more sophisticated way than tokio::task::spawn_blocking(). Until that happens, most programmers try to get all their data into memory first on a concurrent executor-driven threadpool and then offload the synchronous computation to a parallel threadpool (e.g. managed by Rayon).

48 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-async-awaitArea: Async & AwaitAsyncAwait-TriagedAsync-await issues that have been triaged during a working group meeting.C-tracking-issueCategory: An issue tracking the progress of sth. like the implementation of an RFCLibs-TrackedLibs issues that are tracked on the team's project board.T-langRelevant to the language teamT-libs-apiRelevant to the library API team, which will review and decide on the PR/issue.WG-asyncWorking group: Async & await

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @joshtriplett@brainstorm@Amanieu@RalfJung@the8472

        Issue actions

          Tracking Issue for #![feature(async_iterator)] · Issue #79024 · rust-lang/rust