Split derived files generation errors in incremental mode

[codeman9]

The documentation for swift.split_derived_files_generation mentions whole module optimization, so this feature may only be applicable to that compilation mode:

# If enabled and whole module optimisation is being used, the `*.swiftdoc`,
# `*.swiftmodule` and `*-Swift.h` are generated with a separate action
# rather than as part of the compilation.
SWIFT_FEATURE_SPLIT_DERIVED_FILES_GENERATION = "swift.split_derived_files_generation"

When this feature is enabled and incremental mode is used, then the following error is seen:

error: merge-module command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: error opening input file '/var/folders/ws/gbr_5dz97rncvmckbmzl4h7r0000gn/T/TemporaryDirectory.xeR92j/I18N-1.swiftmodule' (No such file or directory)

Can this feature be used for incremental mode? If this feature is only relevant for whole module optimization, then would it be possible to output an error to that effect?

[brentleyjones]

I know swift does a form of split compilation in incremental mode. I also added code into the worker to make sure it copied files correctly. It might need this change: #699

In the same vein, in order to support split modules with incremental builds, we need to use this flag (though we could choose to use this version only with incremental + split modules, if we decide it's not worth the above regressions).

[codeman9]

I tried adding #699 to no change in effect. I'm noticing, also, that the .derived_output_file_map.json file is always {} even while using whole module optimization. It is written here: https://github.com/bazelbuild/rules_swift/blob/def553123dc56449ef7e1fa7ae8ed3440aec7a73/swift/internal/compiling.bzl#L2684-L2688 but is always empty. It used to contain the partial swiftmodule path, but that has since been removed:

rules_swift/swift/internal/compiling.bzl

Lines 2466 to 2469 in 23c46bb

	if split_derived_file_generation:
	derived_files_output_map[src.path] = {"swiftmodule": partial_module.path}
	else:

I'm not entirely sure what sorts of things should go in this file.

[brentleyjones]

You'll minimally need #699. It's not enough on its own to unlock incremental split mode. I would look to see how Xcode 14 is doing this, as now does incremental split mode. From it's commands you'll be able to piece together what rules_swift should do.