`expand_expr` not being expanded.

[vivax3794]

rust-analyzer version: 1.88.0-nightly (27d6200 2025-05-06)
rustc version: rustc 1.88.0-nightly (27d6200a7 2025-05-06)

editor or extension: rustaceanvim on version 5.26.0

code snippet to reproduce:
A proc macro using the nightly expand_expr feature

#![feature(proc_macro_expand)]

extern crate proc_macro;

#[proc_macro]
pub fn expand(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
    let input = input.expand_expr().unwrap();
    panic!("expanded input: {input}");
}

when using this macro:

the_macro::expand!(concat!("Hello, ", "world!"));

cargo check (and its corresponding) diagnostic produces expanded input: "Hello, world!", but rust analyzer also produces its own error: proc-macro panicked: expanded input: concat ! ("Hello, " , "world!") [macro-error]

In addition, it seems the macro is not expanded at all in this case by RA, leading to items emitted by said macro to not be available in completions.

[vivax3794]

Seems like the current expand_expr implementation is a Noop

rust-analyzer/crates/proc-macro-srv/src/server_impl/token_id.rs

Lines 203 to 205 in debaef8

	fn expand_expr(&mut self, self_: &Self::TokenStream) -> Result<Self::TokenStream, ()> {
	Ok(self_.clone())
	}

rust-analyzer/crates/proc-macro-srv/src/server_impl/rust_analyzer_span.rs

Lines 218 to 225 in debaef8

	fn expand_expr(&mut self, self_: &Self::TokenStream) -> Result<Self::TokenStream, ()> {
	// FIXME: requires db, more importantly this requires name resolution so we would need to
	// eagerly expand this proc-macro, but we can't know that this proc-macro is eager until we
	// expand it ...
	// This calls for some kind of marker that a proc-macro wants to access this eager API,
	// otherwise we need to treat every proc-macro eagerly / or not support this.
	Ok(self_.clone())
	}

[vivax3794]

(in retrospect, this should maybe have been a feature request, but I hadn't looked into the code enough to know this was unimplemented and not just broken)

[ChayimFriedman2]

Indeed, we do not implement expand_expr(), and I think it will be very hard to implement for us, both from the proc macro bridge side (need to access the database) and because our current eager macro expansion infra is very hacky and not possible to plug into.

[Veykril]

Note that the current design of this API would require us to pessimize all proc-macros, assume that all of them are eager and discard the entire idea of expanding proc-macros lazily. So even if we do get to implementing the remaining proc-macro API (that needs to call the database), this one we will likely just not be able to implement.

[ChayimFriedman2]

@Veykril How so? I know our current eager infra does not allow this, but with an idealized eager infra we can still expand proc macros lazily and only eager expand their input when they request it.

[Veykril]

we do not know if a proc-macro is eager without executing it, that means we need to keep the name resolution context around (which we can't because that would make it a query input) when expanding it as (unlike with decl eager macros) there is no way for us to expand the input first. The API is on-demand expansion, not a "expand the entire input like is done for builtin eager macros". I already is a miracle that we can make the current set of builtin eager macros work as is tbh (its a horrifying sight to behold), given we already do it kind of differently than rustc does

[ChayimFriedman2]

Can't we just resolve it in the context of the outer macro call?

[Veykril]

My point is we don't have that context, we only have access to what the expand_proc_macro has access to. There is a reason why the current eager decl macros are statically marked as eager within our code (unlike rustc) and require completely different handling from non-eager macros before we even invoke any queries.

That is, from what I know of how we do macro expansion, to support this, we would need to side step the macro queries altogether to be able to support this as otherwise we do not have access to the containing name resolution context.

[ChayimFriedman2]

Isn't that context encoded in the def map? I.e. doesn't the def map has everything needed to resolve names?