-
Notifications
You must be signed in to change notification settings - Fork 13.6k
Description
In working on #44142 I've come across the need/desire a few times to have "always dirty" nodes that are always recomputed at the base of the dependency graph. One example of this is handling today's extern_mod_stmt_cnum
method.
Today there's a method CrateStore::extern_mod_stmt_cnum
, but as part of #41417 we want to move this to a query. That's relatively simple but what's actually happening here I think is a bit more subtle in terms of dependencies. This query will take the ID of an extern crate
statement and return the CrateNum
that it loaded. This happens very early in the compiler when we're loading crates and it's basically "however CrateStore
is implemented picks these numbers".
Right now, on my currently unmerged branch, the implementation looks like this:
extern_mod_stmt_cnum: |tcx, id| {
let id = tcx.hir.definitions().find_node_for_hir_id(id);
tcx.sess.cstore.extern_mod_stmt_cnum_untracked(id)
},
So in other words this is just defining a query that doesn't actually have any dependencies! The extern_mod_stmt_cnum_untracked
method just reaches into the internals of CStore
and plucks a CrateNum
seemingly out of thin air, returning it.
Ideally I think what we'll want here is a way of saying that queries like this need to be computed 100% of the time to determine if they're red or green. Typically they'll instantly turn green again which'll allow us to have lots of cache hits, but I'm worried about assumign they're always green because they have no inputs.
I believe this is a similar-ish issue to many of the maps in #44137 as well. For all the maps calculated in resolve then then hidden behind a query in TyCtxt
those nodes don't actually have any dependencies, they're just reading internal tables. We should always rerun the query though to ensure that it is properly tracked!
Activity
michaelwoerister commentedon Sep 4, 2017
Yes, I agree. I think what @nikomatsakis calls "constant data" in #44137 is really "input data" as far as the query system is concerned. Inputs have no dependencies, they always need to be read and checked for changes.
michaelwoerister commentedon Sep 4, 2017
Right now a
DepNode
can be marked as[anon]
. One way to handle this would be to add a[input]
modifier that informs the query system that it must treat the dependency differently.michaelwoerister commentedon Sep 4, 2017
cc @eddyb
nikomatsakis commentedon Sep 5, 2017
Yes so I'm in favor of this. I think roughly speaking the idea of an "input" query would just be that it's a query where we can never re-use the results. Seems like a simple thing for us to add and it will no doubt remain useful until such time as we push the query system all the way throughout the compiler (and maybe even then...).
(For example, it could eventually subsume the existing handling of HIR nodes -- i.e., we could just have a
hir(def-id)
query that returns the data, and let the natural hashing take care of it.)@michaelwoerister this does suggest that perhaps moving towards a lazy model of deciding when (e.g.) an input HIR node has changed might be better? i.e., closer to this scheme?
michaelwoerister commentedon Sep 5, 2017
I think so, yes.
alexcrichton commentedon Sep 13, 2017
Niko just had an excellent idea. Let's add an always red query called "Input", and then any other query that wants to always run will simply depend on this.
eddyb commentedon Sep 13, 2017
Just to be clear: having the compiler arguments and files read during parsing/macro expansion would serve the same purpose, if we can sneak all of that into queries? If so I'm fine with "Input".
michaelwoerister commentedon Sep 14, 2017
Yet, they are not always red :) An input can be green too, it's just that we cannot rely on their (non-existing) dependencies for determining if they are. It would rather have to be a "always re-compute" query.
michaelwoerister commentedon Dec 6, 2017
❤️ This has been implement in #45353 by @wesleywiser
Rollup merge of rust-lang#57290 - mark-i-m:remove-outdated-comment, r…
Remove outdated comment
Rollup merge of rust-lang#131965 - ChrisDenton:outdated-comment, r=ji…
Unrolled build for rust-lang#131965