improve smart flattening #379

Job	Run time
id	3m 40s
build (ubuntu-latest, linux, x86-64)	2m 20s
unavailable	1m 18s
build (macos-latest, darwin, aarch64)	1m 48s
audit (macos-latest)	1m 56s
test (darwin+aarch64)	2m 8s
bottle (gz, linux+x86-64)	2m 7s
bottle (xz, darwin+aarch64)	1m 59s
bottle (gz, darwin+aarch64)	1m 40s
test	2m 30s
bottle (xz, linux+x86-64)	1m 50s
audit (ubuntu-latest)	1m 46s
	25m 2s

Provide feedback