soundwire: fix programing unattached BUSCLOCK_SCALE issue #8196

Job	Run time
GCC build X86_64 with W=1	2m 57s
Sparse check	2m 55s
GCC build X86_64 with W=1 allmodconfig	1m 50s
GCC static -fanalyzer	4m 1s
GCC build arm64	22m 55s
GCC build i386	16m 56s
Clang build x86_64	4m 42s
GCC build arm64 allmodconfig	1m 39s
GCC build X86_64	20m 21s
	1h 18m 16s

Provide feedback