Tracking Issue for `core::future::join!`

[ibraheemdev]

Feature gate: #![feature(future_join)]

This is a tracking issue for core::future::join!, which polls multiple futures concurrently and returns their outputs.

async fn run() {
    let (a, b) = join!(async { 0 }, async { 1 }).await;
}

Public API

// core::future

pub macro join {
    ( $($fut:expr),* $(,)?) => {
        // ...
    }
}

Steps / History

• Implementation: Implement core::future::join! #91645
• Minor improvements: Minor improvements to future::join!'s implementation #91721
• Final comment period (FCP)
• Stabilization PR

Unresolved Questions

• Implicit .await? Tokio and futures currently do this.
• await_all! vs join!
• join/and method on Future: a.and(b).await
• Implement IntoFuture for N-arity tuples? let (a, b, c) = (a, b, c).await

[yoshuawuyts]

The @rust-lang/wg-async-foundations group was not notified of #91645 prior to merging. Tagging here to ensure the right people get the notification that this was merged into unstable.

[yoshuawuyts]

I don't think that #![feature(future_join)] should have been merged as-is, and instead needs to be part of a coherent design implementing async concurrency for Rust. The topic of async concurrency is a fundamental part of Rust's future, and nuanced enough that it should be going through the RFC process ¹.

Personally I've found the abstractions provided in futures-rs (e.g. join!, join_all!, select! {} et. al.) to be severely lacking. For that reason we never adopted these in async_std::future, and have instead been researching alternatives (1, 2, 3, 4, 5, 6) for the past three years.

Because of the size of the topic, I would like to see a project group started as part of the async foundations WG, tasked with describing a coherent model for async concurrency in Rust. The goal would be to produce an RFC + implementation of async concurrency adapters for the stdlib, which can then be implemented as part of the stdlib.

Which brings us back to what to do with #![feature(future_join)]. I see two plausible options here:

1. We change the feature gate to #[feature(future_concurrency)], and file a blocking concern mentioning this requires an RFC.
2. We file a PR removing the feature from the stdlib, close the tracking issue, and start fresh with a group under the async foundations WG.

I want to ensure we don't accidentally stabilize this feature without having considered the much wider topic of creating a coherent model and corresponding interfaces for async concurrency in Rust.

Footnotes

1. See Swift SE-0304: Structured Concurrency for an example of what scope I expect a Rust RFC on async concurrency to cover. ↩

[carllerche]

I agree w/ @yoshuawuyts and don't have much else to add. Tokio added join! mostly because it was an established pattern. If I were to do it again, I would not have included join!. After onboarding quite a few new devs to async rust, I think join! complicates the learning experience without adding much over spawning the sub-tasks. There are alternative strategies that could be explored over join!.

[eholk]

After onboarding quite a few new devs to async rust, I think join! complicates the learning experience without adding much over spawning the sub-tasks.

@carllerche - I was wondering if you could provide more details about your experience here? What did new devs find complicated about join!?

[cryptoquick]

Is there a better way to accomplish parallel await than with this feature? My use-case is basically the example here:
https://doc.rust-lang.org/std/future/macro.join.html

[yoshuawuyts]

@cryptoquick the stdlib doesn't provide any other APIs for concurrenly .awaiting futures right now, no. You can look at the futures, async-std, tokio, and futures-lite ecosystem crates for various variations on the join! functionality provided here.

Additionally I've been experimenting with alternative static async concurrency designs and writing about them in my blog, with the intent of arriving at a design we can put forward for stabilization. There's a crate published now too: futures-concurrency, which implements some of these designs. Note that right now this only covers static fixed-sized concurrency such as join!, and not dynamic variable-sized concurrency which is provided by abstractions such as FuturesUnordered and StreamMap.

As I mentioned in #91642 (comment) adding async concurrency APIs to the stdlib is a large enough topic that it should go through an RFC. Until we've done that, I think it's unlikely we'll be seeing any concurrency functionality stabilized in the stdlib.

I hope that answers your question!

[discreaminant2809]

Should it accept IntoFuture instead of Future?

[RalfJung]

Looks like the current implementation is possibly unsound: rust-lang/miri#3780.

[RalfJung]

Turns out join is fine, the issue is deeper -- self-referential async block lowering is still unsound.