[DOP-29498] Allow overriding Spark session config #1504

Job	Run time
MSSQL tests / Run MSSQL tests	8m 27s
MySQL tests / Run MySQL tests	7m 57s
Hive tests / Run Hive tests	8m 7s
Clickhouse tests / Run Clickhouse tests	6m 49s
Oracle tests / Run Oracle tests	8m 21s
HDFS tests / Run HDFS tests	11m 26s
Samba tests / Run Samba tests	8m 4s
S3 tests / Run S3 tests	12m 9s
FTP tests / Run FTP tests	10m 10s
FTPS tests / Run FTPS tests	7m 33s
Scheduler tests / Run Scheduler tests	6m 22s
Unit tests / Run unit tests	5m 13s
WebDAV tests / Run WebDAV tests	7m 52s
SFTP tests / Run SFTP tests	8m 8s
Iceberg tests / Run Iceberg tests	8m 24s
Tests done	0s
	2h 5m 2s

Provide feedback