Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Users may request output groups' files be inlined in
TargetComplete
…
…/`AspectComplete` events in BEP using new flag. Some services that consume large BEP streams need to be able to eagerly identify critical output files without keeping the entire BEP stream in-memory. Traditionally, these were reported in the `TargetComplete.important_outputs` field, which is deprecated because it can grow without bound without any mechanism to break up the set of files included in that field. The replacement for `TargetComplete.important_outputs` is the recursive `NamedSetOfFiles` representation which can be broken up arbitrarily and supports efficient serialization directly from `NestedSet` objects inside the build tool. Unfortunately consumers must keep the entire BEP stream in-memory in order to efficiently identify critical files when they are linked from a `TargetComplete.output_groups.file_sets` field. As a compromise, this change introduces a new field: `OutputGroup.inline_files`. By using a new flag, `--experimental_build_event_output_group_mode`, users can request that specific output groups report their files inline instead of using the recursive `NamedSetOfFiles`. This still suffers from the same failure mode as `TargetComplete.important_outputs` - a single huge output group can produce an extremely large single event. Unlike `TargetComplete.important_outputs`, users may restructure their build rules using additional output groups to shrink the size of their events. RELNOTES: A new experimental flag, `--experimental_build_event_output_group_mode`, allows users to change how a given output group's files are reported in BEP. The current behavior is `FILESET` which populates `OutputGroup.file_sets`. Users may now specify `INLINE` to instead report files directly in the `TargetComplete`/`AspectComplete` event under `OutputGroup.inline_files`. Users may also specify `BOTH` to populate `OutputGroup.file_sets` and `OutputGroup.inline_files`. (cherry picked from commit ff1f674)
- Loading branch information