perf: Implement query chunking for charts (#1233)

# Summary

Closes HDX-2310
Closes HDX-2616

This PR implements chunking of chart queries to improve performance of charts on large data sets and long time ranges. Recent data is loaded first, then older data is loaded one-chunk-at-a-time until the full chart date range has been queried.

https://github.com/user-attachments/assets/83333041-9e41-438a-9763-d6f6c32a0576

## Performance Impacts

### Expectations

This change is intended to improve performance in a few ways:

1. Queries over long time ranges are now much less likely to time out, since the range is chunked into several smaller queries
2. Average memory usage should decrease, since the total result size and number of rows being read are smaller
3. _Perceived_ latency of queries over long date ranges is likely to decrease, because users will start seeing charts render (more recent) data as soon as the first chunk is queried, instead of after the entire date range has been queried. **However**, _total_ latency to display results for the entire date range is likely to increase, due to additional round-trip network latency being added for each additional chunk.

### Measured Results

Overall, the results match the expectations outlined above.

- Total latency changed between ~-4% and ~25%
- Average memory usage decreased by between 18% and 80%

<details>
<summary>Scenarios and data</summary>

In each of the following tests:

1. Queries were run 5 times before starting to measure, to ensure data is filesystem cached.
2. Queries were then run 3 times. The results shown are the median result from the 3 runs.

#### Scenario: Log Search Histogram in Staging V2, 2 Day Range, No Filter

|   |  Total Latency | Memory Usage (Avg) | Memory Usage (Max)  |  Chunk Count |
|---|---|---|---|---|
|  Original |  5.36 |  409.23 MiB |  409.23 MiB | 1  |
|  Chunked |  5.14 | 83.06 MiB  | 232.69 MiB  |  4 |

#### Scenario: Log Search Histogram in Staging V2, 14 Day Range, No Filter

|   |  Total Latency | Memory Usage (Avg) | Memory Usage (Max)  |  Chunk Count |
|---|---|---|---|---|
|  Original |  26.56 |  383.63 MiB |  383.63 MiB | 1  |
|  Chunked |  33.08 | 130.00 MiB  | 241.21 MiB  |  16 |

#### Scenario: Chart Explorer Line Chart with p90 and p99 trace durations, Staging V2 Traces, Filtering for "GET" spans, 7 Day range

|   |  Total Latency | Memory Usage (Avg) | Memory Usage (Max)  |  Chunk Count |
|---|---|---|---|---|
|  Original |  2.79 |  346.12 MiB |  346.12 MiB | 1  |
|  Chunked |  3.26 | 283.00 MiB  | 401.38 MiB  |  9 |

</details>

## Implementation Notes

<details>
<summary>When is chunking used?</summary>
Chunking is used when all of the following are true:

1. `granularity` and `timestampValueExpression` are defined in the config. This ensures that the query is already being bucketed. Without bucketing, chunking would break aggregation queries, since groups can span multiple chunks.
4. `dateRange` is defined in the config. Without a date range, we'd need an unbounded set of chunks or the start and end chunks would have to be unbounded at their start and end, respectively.
5. The config is not a metrics query. Metrics queries have complex logic which we want to avoid breaking with the initial delivery of this feature.
6. The consumer of `useQueriedChartConfig` does not pass the `disableQueryChunking: true` option. This option is provided to disable chunking when necessary.
</details>

<details>
<summary>How are time windows chosen?</summary>

1. First, generate the windows as they are generated for the existing search chunking feature (eg. 6 hours back, 6 hours back, 12 hours back, 24 hours back...)
4. Then, the start and end of each window is aligned to the start of a time bucket that depends on the "granularity" of the chart.
7. The first and last windows are shortened or extended so that the combined date range of all of the windows matches the start and end of the original config.
</details>

<details>
<summary>Which order are the chunks queried in?</summary>

Chunks are queried sequentially, most-recent first, due to the expectation that more recent data is typically more important to the user. Unlike with `useOffsetPaginatedSearch`, we are not paginating the data beyond the chunks, and all data is typically displayed together, so there is no need to support "ascending" order.
</details>

<details>
<summary>Does this improve client-side caching behavior?</summary>

One theoretical way in which query chunking could improve performance to enable client-side caching of individual chunks, which could then be re-used if the same query is run over a longer time range.

Unfortunately, using streamedQuery, react-query stores the entire time range as one item in the cache, so it does not re-use individual chunks or "pages" from another query.

We could accomplish this improvement by using useQueries instead of streamedQuery or useInfiniteQuery. In that case, we'd treat each chunk as its own query. This would require a number of changes:

1. Our query key would have to include the chunk's window duration
2. We'd need some hacky way of making the useQueries requests fire in sequence. This can be done using `enabled` but requires some additional state to figure out whether the previous query is done.
5. We'd need to emulate the return value of a useQuery using the useQueries result, or update consumers.
</details>

Author: pulpdrew

.changeset/soft-donkeys-fetch.md

@@ -0,0 +1,5 @@
+---
+"@hyperdx/app": patch
+---
+
+feat: Implement query chunking for charts

packages/app/src/components/DBTimeChart.tsx

@@ -65,8 +65,9 @@ function DBTimeChartComponent({
   const { data, isLoading, isError, error, isPlaceholderData, isSuccess } =
     useQueriedChartConfig(queriedConfig, {
       placeholderData: (prev: any) => prev,
-      queryKey: [queryKeyPrefix, queriedConfig],
+      queryKey: [queryKeyPrefix, queriedConfig, 'chunked'],
       enabled,
+      enableQueryChunking: true,
     });
 
   useEffect(() => {
@@ -75,7 +76,8 @@ function DBTimeChartComponent({
     }
   }, [isError, isErrorExpanded, errorExpansion]);
 
-  const isLoadingOrPlaceholder = isLoading || isPlaceholderData;
+  const isLoadingOrPlaceholder =
+    isLoading || !data?.isComplete || isPlaceholderData;
   const { data: source } = useSource({ id: sourceId });
 
   const { graphResults, timestampColumn, groupKeys, lineNames, lineColors } =

packages/app/src/components/PatternTable.tsx

@@ -29,10 +29,11 @@ export default function PatternTable({
 
   const [selectedPattern, setSelectedPattern] = useState<Pattern | null>(null);
 
-  const { totalCount, isLoading: isTotalCountLoading } = useSearchTotalCount(
-    totalCountConfig,
-    totalCountQueryKeyPrefix,
-  );
+  const {
+    totalCount,
+    isLoading: isTotalCountLoading,
+    isTotalCountComplete,
+  } = useSearchTotalCount(totalCountConfig, totalCountQueryKeyPrefix);
 
   const {
     data: groupedResults,
@@ -46,7 +47,8 @@ export default function PatternTable({
     totalCount,
   });
 
-  const isLoading = isTotalCountLoading || isGroupedPatternsLoading;
+  const isLoading =
+    isTotalCountLoading || !isTotalCountComplete || isGroupedPatternsLoading;
 
   const sortedGroupedResults = useMemo(() => {
     return Object.values(groupedResults).sort(

packages/app/src/components/SearchTotalCountChart.tsx

@@ -10,7 +10,7 @@ export function useSearchTotalCount(
   config: ChartConfigWithDateRange,
   queryKeyPrefix: string,
 ) {
-  // copied from DBTimeChart
+  // queriedConfig, queryKey, and enableQueryChunking match DBTimeChart so that react query can de-dupe these queries.
   const { granularity } = useTimeChartSettings(config);
   const queriedConfig = {
     ...config,
@@ -22,12 +22,15 @@ export function useSearchTotalCount(
     isLoading,
     isError,
   } = useQueriedChartConfig(queriedConfig, {
-    queryKey: [queryKeyPrefix, queriedConfig],
+    queryKey: [queryKeyPrefix, queriedConfig, 'chunked'],
     staleTime: 1000 * 60 * 5,
     refetchOnWindowFocus: false,
     placeholderData: keepPreviousData, // no need to flash loading state when in live tail
+    enableQueryChunking: true,
   });
 
+  const isTotalCountComplete = !!totalCountData?.isComplete;
+
   const totalCount = useMemo(() => {
     return totalCountData?.data?.reduce(
       (p: number, v: any) => p + Number.parseInt(v['count()']),
@@ -39,6 +42,7 @@ export function useSearchTotalCount(
     totalCount,
     isLoading,
     isError,
+    isTotalCountComplete,
   };
 }