Skip to content

Conversation

@romseygeek
Copy link
Contributor

No description provided.

@romseygeek romseygeek self-assigned this Oct 23, 2025
@romseygeek romseygeek marked this pull request as draft October 23, 2025 13:26

this.dimension = TimeSeriesParams.dimensionParam(m -> toType(m).fieldType().isDimension()).addValidator(v -> {
if (v && (indexed.getValue() == false || hasDocValues.getValue() == false)) {
if (v && (hasDocValues.getValue() == false)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change will need to be applied to all field mappers that can be dimensions:

BooleanFieldMapper
FlattenedFieldMapper
GeoPointFieldMapper
IpFieldMapper
NumberFieldMapper
ScaledFloatFieldMapper
UnsignedLongFieldMapper

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and it will need gating behind a FeatureFlag and IndexVersion too. This is just for experimenting, really.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we at least also add IpFieldMapper? I think that will be important for the experimentation.

@felixbarny
Copy link
Member

felixbarny commented Oct 23, 2025

I ran this against the metricsgen track. The highlights:

  • Store size reduced by 18.51%
  • Ingest throughput increased by 20.55% (should be more pronounced when ingesting via the OTLP endpoint)
  • Cumulative merge time reduced by 34.70%
  • Cumulative refresh time reduced by 44.77%
  • Impact on query latency 0-20%. As expected, the biggest impact is seen on queries that filter for a specific set of hosts, for example.
Click to expand full rally comparison
|                                                                                                                    Metric |                                       Task |         Baseline |       Contender |              Diff |   Unit |    Diff % |
|--------------------------------------------------------------------------------------------------------------------------:|-------------------------------------------:|-----------------:|----------------:|------------------:|-------:|----------:|
|                                                                                Cumulative indexing time of primary shards |                                            |    348.269       |   310.975       |     -37.2948      |    min |   -10.71% |
|                                                                         Min cumulative indexing time across primary shard |                                            |    113.961       |   101.815       |     -12.1456      |    min |   -10.66% |
|                                                                      Median cumulative indexing time across primary shard |                                            |    116.486       |   103.979       |     -12.5067      |    min |   -10.74% |
|                                                                         Max cumulative indexing time across primary shard |                                            |    117.823       |   105.181       |     -12.6426      |    min |   -10.73% |
|                                                                       Cumulative indexing throttle time of primary shards |                                            |      0           |     0           |       0           |    min |     0.00% |
|                                                                Min cumulative indexing throttle time across primary shard |                                            |      0           |     0           |       0           |    min |     0.00% |
|                                                             Median cumulative indexing throttle time across primary shard |                                            |      0           |     0           |       0           |    min |     0.00% |
|                                                                Max cumulative indexing throttle time across primary shard |                                            |      0           |     0           |       0           |    min |     0.00% |
|                                                                                   Cumulative merge time of primary shards |                                            |     89.4657      |    58.4233      |     -31.0424      |    min |   -34.70% |
|                                                                                  Cumulative merge count of primary shards |                                            |     96           |    61           |     -35           |        |   -36.46% |
|                                                                            Min cumulative merge time across primary shard |                                            |     27.2882      |    19.0999      |      -8.18832     |    min |   -30.01% |
|                                                                         Median cumulative merge time across primary shard |                                            |     27.7449      |    19.1056      |      -8.63932     |    min |   -31.14% |
|                                                                            Max cumulative merge time across primary shard |                                            |     34.4327      |    20.2179      |     -14.2148      |    min |   -41.28% |
|                                                                          Cumulative merge throttle time of primary shards |                                            |      9.95505     |     4.1309      |      -5.82415     |    min |   -58.50% |
|                                                                   Min cumulative merge throttle time across primary shard |                                            |      2.54392     |     1.29497     |      -1.24895     |    min |   -49.10% |
|                                                                Median cumulative merge throttle time across primary shard |                                            |      2.61875     |     1.41272     |      -1.20603     |    min |   -46.05% |
|                                                                   Max cumulative merge throttle time across primary shard |                                            |      4.79238     |     1.42322     |      -3.36917     |    min |   -70.30% |
|                                                                                 Cumulative refresh time of primary shards |                                            |      2.254       |     1.2449      |      -1.0091      |    min |   -44.77% |
|                                                                                Cumulative refresh count of primary shards |                                            |    273           |   230           |     -43           |        |   -15.75% |
|                                                                          Min cumulative refresh time across primary shard |                                            |      0.7044      |     0.388       |      -0.3164      |    min |   -44.92% |
|                                                                       Median cumulative refresh time across primary shard |                                            |      0.7633      |     0.412383    |      -0.35092     |    min |   -45.97% |
|                                                                          Max cumulative refresh time across primary shard |                                            |      0.7863      |     0.444517    |      -0.34178     |    min |   -43.47% |
|                                                                                   Cumulative flush time of primary shards |                                            |      4.84823     |     0.780917    |      -4.06732     |    min |   -83.89% |
|                                                                                  Cumulative flush count of primary shards |                                            |     88           |    23           |     -65           |        |   -73.86% |
|                                                                            Min cumulative flush time across primary shard |                                            |      1.55692     |     0.13315     |      -1.42377     |    min |   -91.45% |
|                                                                         Median cumulative flush time across primary shard |                                            |      1.63477     |     0.2794      |      -1.35537     |    min |   -82.91% |
|                                                                            Max cumulative flush time across primary shard |                                            |      1.65655     |     0.368367    |      -1.28818     |    min |   -77.76% |
|                                                                                                   Total Young Gen GC time |                                            |    132.35        |   126.411       |      -5.939       |      s |    -4.49% |
|                                                                                                  Total Young Gen GC count |                                            |   4930           |  4340           |    -590           |        |   -11.97% |
|                                                                                                     Total Old Gen GC time |                                            |      0           |     0           |       0           |      s |     0.00% |
|                                                                                                    Total Old Gen GC count |                                            |      0           |     0           |       0           |        |     0.00% |
|                                                                                                              Dataset size |                                            |      3.58509     |     2.92149     |      -0.6636      |     GB |   -18.51% |
|                                                                                                                Store size |                                            |      3.58509     |     2.92149     |      -0.6636      |     GB |   -18.51% |
|                                                                                                             Translog size |                                            |      1.53668e-07 |     1.53668e-07 |       0           |     GB |     0.00% |
|                                                                                                    Heap used for segments |                                            |      0           |     0           |       0           |     MB |     0.00% |
|                                                                                                  Heap used for doc values |                                            |      0           |     0           |       0           |     MB |     0.00% |
|                                                                                                       Heap used for terms |                                            |      0           |     0           |       0           |     MB |     0.00% |
|                                                                                                       Heap used for norms |                                            |      0           |     0           |       0           |     MB |     0.00% |
|                                                                                                      Heap used for points |                                            |      0           |     0           |       0           |     MB |     0.00% |
|                                                                                               Heap used for stored fields |                                            |      0           |     0           |       0           |     MB |     0.00% |
|                                                                                                             Segment count |                                            |      3           |     3           |       0           |        |     0.00% |
|                                                                                               Total Ingest Pipeline count |                                            |      0           |     0           |       0           |        |     0.00% |
|                                                                                                Total Ingest Pipeline time |                                            |      0           |     0           |       0           |     ms |     0.00% |
|                                                                                              Total Ingest Pipeline failed |                                            |      0           |     0           |       0           |        |     0.00% |
|                           .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.status inverted index |                                            | 282053           |     0           | -282053           |  bytes |  -100.00% |
|                               .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.status doc values |                                            |     72.1211      |    72.376       |       0.25488     |     kB |    +0.35% |
|                                    .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.status total |                                            |    347.563       |    72.376       |    -275.188       |     kB |   -79.18% |
| .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.executable.name inverted index |                                            | 439193           |     0           | -439193           |  bytes |  -100.00% |
|     .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.executable.name doc values |                                            |     67.6582      |    67.7432      |       0.08496     |     kB |    +0.13% |
|          .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.executable.name total |                                            |    496.558       |    67.7432      |    -428.814       |     kB |   -86.36% |
|         .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.command inverted index |                                            | 439199           |     0           | -439199           |  bytes |  -100.00% |
|             .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.command doc values |                                            |     67.6611      |    67.7461      |       0.08496     |     kB |    +0.13% |
|                  .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.command total |                                            |    496.566       |    67.7461      |    -428.82        |     kB |   -86.36% |
| .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.executable.path inverted index |                                            | 439199           |     0           | -439199           |  bytes |  -100.00% |
|     .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.executable.path doc values |                                            |     67.6611      |    67.7461      |       0.08496     |     kB |    +0.13% |
|          .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.executable.path total |                                            |    496.566       |    67.7461      |    -428.82        |     kB |   -86.36% |
|    .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.command_line inverted index |                                            | 439325           |     0           | -439325           |  bytes |  -100.00% |
|        .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.command_line doc values |                                            |     67.7842      |    67.8691      |       0.08496     |     kB |    +0.13% |
|             .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.command_line total |                                            |    496.812       |    67.8691      |    -428.943       |     kB |   -86.34% |
|                .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.cpu.logical.count doc values |                                            |    501.776       |   501.776       |       0           |     kB |     0.00% |
|                     .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.cpu.logical.count total |                                            |    501.776       |   501.776       |       0           |     kB |     0.00% |
|          .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.disk.pending_operations doc values |                                            |    503.887       |   503.887       |       0           |     kB |     0.00% |
|               .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.disk.pending_operations total |                                            |    503.887       |   503.887       |       0           |     kB |     0.00% |
|                  .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.memory.virtual doc values |                                            |    507.407       |   507.407       |       0           |     kB |     0.00% |
|                       .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.memory.virtual total |                                            |    507.407       |   507.407       |       0           |     kB |     0.00% |
|                    .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.memory.usage doc values |                                            |    507.972       |   507.972       |       0           |     kB |     0.00% |
|                         .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.memory.usage total |                                            |    507.972       |   507.972       |       0           |     kB |     0.00% |
|                         .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.threads doc values |                                            |    509.581       |   509.581       |       0           |     kB |     0.00% |
|                              .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.threads total |                                            |    509.581       |   509.581       |       0           |     kB |     0.00% |
|           .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.open_file_descriptors doc values |                                            |    511.227       |   511.227       |       0           |     kB |     0.00% |
|                .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.open_file_descriptors total |                                            |    511.227       |   511.227       |       0           |     kB |     0.00% |
|                         .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.protocol inverted index |                                            | 479126           |     0           | -479126           |  bytes |  -100.00% |
|                             .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.protocol doc values |                                            |     61.3496      |    61.4346      |       0.08496     |     kB |    +0.14% |
|                                  .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.protocol total |                                            |    529.246       |    61.4346      |    -467.812       |     kB |   -88.39% |
|                                           .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 _tsid doc values |                                            |    639.023       |   639.023       |       0           |     kB |     0.00% |
|                                                .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 _tsid total |                                            |    639.023       |   639.023       |       0           |     kB |     0.00% |
|                             .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.mode inverted index |                                            | 599141           |     0           | -599141           |  bytes |  -100.00% |
|                                 .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.mode doc values |                                            |     65.8936      |    65.9785      |       0.08496     |     kB |    +0.13% |
|                                      .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.mode total |                                            |    650.992       |    65.9785      |    -585.014       |     kB |   -89.86% |
|                             .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.type inverted index |                                            | 599147           |     0           | -599147           |  bytes |  -100.00% |
|                                 .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.type doc values |                                            |     65.8994      |    65.9844      |       0.08496     |     kB |    +0.13% |
|                                      .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.type total |                                            |    651.004       |    65.9844      |    -585.02        |     kB |   -89.86% |
|                .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.processes.created doc values |                                            |    702.492       |   702.492       |       0           |     kB |     0.00% |
|                     .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.processes.created total |                                            |    702.492       |   702.492       |       0           |     kB |     0.00% |
|          .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.parent_pid doc values |                                            |    203.568       |   203.568       |       0           |     kB |     0.00% |
|              .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.parent_pid points |                                            |    667.662       |   667.662       |       0           |     kB |     0.00% |
|               .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.parent_pid total |                                            |    871.23        |   871.23        |       0           |     kB |     0.00% |
|                 .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.pid doc values |                                            |    339.569       |   339.569       |       0           |     kB |     0.00% |
|                     .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.pid points |                                            |    667.662       |   667.662       |       0           |     kB |     0.00% |
|                      .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.process.pid total |                                            |   1007.23        |  1007.23        |       0           |     kB |     0.00% |
|                          .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.disk.io doc values |                                            |      1.30337     |     1.30337     |       0           |     MB |     0.00% |
|                               .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.disk.io total |                                            |      1.30337     |     1.30337     |       0           |     MB |     0.00% |
|                  .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.disk.operations doc values |                                            |      1.30896     |     1.30896     |       0           |     MB |     0.00% |
|                       .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.disk.operations total |                                            |      1.30896     |     1.30896     |       0           |     MB |     0.00% |
|                         .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.disk.io doc values |                                            |      1.32152     |     1.32152     |       0           |     MB |     0.00% |
|                              .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.disk.io total |                                            |      1.32152     |     1.32152     |       0           |     MB |     0.00% |
|                 .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.disk.operations doc values |                                            |      1.32185     |     1.32185     |       0           |     MB |     0.00% |
|                      .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.disk.operations total |                                            |      1.32185     |     1.32185     |       0           |     MB |     0.00% |
|                  .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.processes.count doc values |                                            |      1.32661     |     1.32661     |       0           |     MB |     0.00% |
|                       .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.processes.count total |                                            |      1.32661     |     1.32661     |       0           |     MB |     0.00% |
|                       .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.mountpoint inverted index |                                            |      1.23209e+06 |     0           |      -1.23209e+06 |  bytes |  -100.00% |
|                           .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.mountpoint doc values |                                            |    328.198       |   328.708       |       0.50977     |     kB |    +0.16% |
|                                .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.mountpoint total |                                            |   1531.42        |   328.708       |   -1202.71        |     kB |   -78.54% |
|                     .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.memory.usage doc values |                                            |      2.65692     |     2.65692     |       0           |     MB |     0.00% |
|                          .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.memory.usage total |                                            |      2.65692     |     2.65692     |       0           |     MB |     0.00% |
|          .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.filesystem.inodes.usage doc values |                                            |      2.66647     |     2.66647     |       0           |     MB |     0.00% |
|               .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.filesystem.inodes.usage total |                                            |      2.66647     |     2.66647     |       0           |     MB |     0.00% |
|                             .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 data_stream.dataset doc values |                                            |      3.08995     |     3.08995     |       0           |     MB |     0.00% |
|                                  .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 data_stream.dataset total |                                            |      3.08995     |     3.08995     |       0           |     MB |     0.00% |
|                           .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 data_stream.namespace doc values |                                            |      3.08995     |     3.08995     |       0           |     MB |     0.00% |
|                                .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 data_stream.namespace total |                                            |      3.08995     |     3.08995     |       0           |     MB |     0.00% |
|                                .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 data_stream.type doc values |                                            |      3.08995     |     3.08995     |       0           |     MB |     0.00% |
|                                     .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 data_stream.type total |                                            |      3.08995     |     3.08995     |       0           |     MB |     0.00% |
|                        .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.direction inverted index |                                            |      3.20908e+06 |     0           |      -3.20908e+06 |  bytes |  -100.00% |
|                            .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.direction doc values |                                            |    719.438       |   722.654       |       3.2168      |     kB |    +0.45% |
|                                 .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.direction total |                                            |   3853.3         |   722.654       |   -3130.65        |     kB |   -81.25% |
|                 .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.filesystem.usage doc values |                                            |      3.96893     |     3.96893     |       0           |     MB |     0.00% |
|                      .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.filesystem.usage total |                                            |      3.96893     |     3.96893     |       0           |     MB |     0.00% |
|                              .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.cpu inverted index |                                            |      3.8685e+06  |     0           |      -3.8685e+06  |  bytes |  -100.00% |
|                                  .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.cpu doc values |                                            |    696.259       |   697.19        |       0.93164     |     kB |    +0.13% |
|                                       .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.cpu total |                                            |   4474.09        |   697.19        |   -3776.9         |     kB |   -84.42% |
|                       .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.network.io doc values |                                            |      5.07874     |     5.07874     |       0           |     MB |     0.00% |
|                            .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.network.io total |                                            |      5.07874     |     5.07874     |       0           |     MB |     0.00% |
|                  .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.network.dropped doc values |                                            |      5.09654     |     5.09654     |       0           |     MB |     0.00% |
|                       .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.network.dropped total |                                            |      5.09654     |     5.09654     |       0           |     MB |     0.00% |
|                  .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.network.packets doc values |                                            |      5.09686     |     5.09686     |       0           |     MB |     0.00% |
|                       .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.network.packets total |                                            |      5.09686     |     5.09686     |       0           |     MB |     0.00% |
|                   .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.network.errors doc values |                                            |      5.09724     |     5.09724     |       0           |     MB |     0.00% |
|                        .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.network.errors total |                                            |      5.09724     |     5.09724     |       0           |     MB |     0.00% |
|                           .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.device inverted index |                                            |      4.47597e+06 |     0           |      -4.47597e+06 |  bytes |  -100.00% |
|                               .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.device doc values |                                            |    935.647       |   937.171       |       1.52344     |     kB |    +0.16% |
|                                    .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.device total |                                            |   5306.71        |   937.171       |   -4369.54        |     kB |   -82.34% |
|              .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.network.connections doc values |                                            |      5.26719     |     5.26719     |       0           |     MB |     0.00% |
|                   .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.network.connections total |                                            |      5.26719     |     5.26719     |       0           |     MB |     0.00% |
|                                .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 _ts_routing_hash doc values |                                            |      5.35067     |     5.35067     |       0           |     MB |     0.00% |
|                                     .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 _ts_routing_hash total |                                            |      5.35067     |     5.35067     |       0           |     MB |     0.00% |
|      .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.cache.l2.size doc values |                                            |      1.54495     |     1.54495     |       0           |     MB |     0.00% |
|          .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.cache.l2.size points |                                            |      7.57459     |     7.57459     |       0           |     MB |     0.00% |
|           .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.cache.l2.size total |                                            |      9.11955     |     9.11955     |       0           |     MB |     0.00% |
|         .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.family inverted index |                                            |      1.00262e+07 |     0           |      -1.00262e+07 |  bytes |  -100.00% |
|             .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.family doc values |                                            |      3           |    90           |      87           |  bytes | +2900.00% |
|                  .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.family total |                                            |      1.00262e+07 |    90           |      -1.00262e+07 |  bytes |  -100.00% |
|     .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.model.name inverted index |                                            |      1.00262e+07 |     0           |      -1.00262e+07 |  bytes |  -100.00% |
|         .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.model.name doc values |                                            |      3           |    90           |      87           |  bytes | +2900.00% |
|              .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.model.name total |                                            |      1.00262e+07 |    90           |      -1.00262e+07 |  bytes |  -100.00% |
|       .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.stepping inverted index |                                            |      1.00262e+07 |     0           |      -1.00262e+07 |  bytes |  -100.00% |
|           .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.stepping doc values |                                            |      6           |    93           |      87           |  bytes | +1450.00% |
|                .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.stepping total |                                            |      1.00262e+07 |    93           |      -1.00262e+07 |  bytes |  -100.00% |
|               .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.arch inverted index |                                            |      1.00263e+07 |     0           |      -1.00263e+07 |  bytes |  -100.00% |
|                   .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.arch doc values |                                            |     18           |   105           |      87           |  bytes |  +483.33% |
|                        .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.arch total |                                            |      1.00263e+07 |   105           |      -1.00262e+07 |  bytes |  -100.00% |
|       .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.model.id inverted index |                                            |      1.00263e+07 |     0           |      -1.00263e+07 |  bytes |  -100.00% |
|           .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.model.id doc values |                                            |     18           |   105           |      87           |  bytes |  +483.33% |
|                .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.model.id total |                                            |      1.00263e+07 |   105           |      -1.00262e+07 |  bytes |  -100.00% |
|      .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.vendor.id inverted index |                                            |      1.00263e+07 |     0           |      -1.00263e+07 |  bytes |  -100.00% |
|          .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.vendor.id doc values |                                            |     18           |   105           |      87           |  bytes |  +483.33% |
|               .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.cpu.vendor.id total |                                            |      1.00263e+07 |   105           |      -1.00262e+07 |  bytes |  -100.00% |
|                 .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.os.type inverted index |                                            |      1.00263e+07 |     0           |      -1.00263e+07 |  bytes |  -100.00% |
|                     .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.os.type doc values |                                            |     18           |   105           |      87           |  bytes |  +483.33% |
|                          .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.os.type total |                                            |      1.00263e+07 |   105           |      -1.00262e+07 |  bytes |  -100.00% |
|                         .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.schema_url inverted index |                                            |      9.56188     |     9.56187     |      -1e-05       |     MB |    -0.00% |
|                             .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.schema_url doc values |                                            |    117           |   117           |       0           |  bytes |     0.00% |
|                                  .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.schema_url total |                                            |      9.56199     |     9.56198     |      -1e-05       |     MB |    -0.00% |
|          .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.os.description inverted index |                                            |      1.00265e+07 |     0           |      -1.00265e+07 |  bytes |  -100.00% |
|              .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.os.description doc values |                                            |    273           |   360           |      87           |  bytes |   +31.87% |
|                   .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.os.description total |                                            |      1.00268e+07 |   360           |      -1.00264e+07 |  bytes |  -100.00% |
|                            .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.state inverted index |                                            |      8.53043e+06 |     0           |      -8.53043e+06 |  bytes |  -100.00% |
|                                .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.state doc values |                                            |      1.82151     |     1.85684     |       0.03534     |     MB |    +1.94% |
|                                     .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 attributes.state total |                                            |      9.95676     |     1.85684     |      -8.09992     |     MB |   -81.35% |
|              .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.cpu.load_average.5m doc values |                                            |     11.0148      |    11.0148      |       0           |     MB |     0.00% |
|                   .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.cpu.load_average.5m total |                                            |     11.0148      |    11.0148      |       0           |     MB |     0.00% |
|                     .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.disk.io_time doc values |                                            |     11.0173      |    11.0173      |       0           |     MB |     0.00% |
|                          .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.disk.io_time total |                                            |     11.0173      |    11.0173      |       0           |     MB |     0.00% |
|             .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.cpu.load_average.15m doc values |                                            |     11.0179      |    11.0179      |       0           |     MB |     0.00% |
|                  .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.cpu.load_average.15m total |                                            |     11.0179      |    11.0179      |       0           |     MB |     0.00% |
|              .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.cpu.load_average.1m doc values |                                            |     11.0197      |    11.0197      |       0           |     MB |     0.00% |
|                   .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.cpu.load_average.1m total |                                            |     11.0197      |    11.0197      |       0           |     MB |     0.00% |
|              .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.memory.utilization doc values |                                            |     11.2137      |    11.2137      |       0           |     MB |     0.00% |
|                   .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.memory.utilization total |                                            |     11.2137      |    11.2137      |       0           |     MB |     0.00% |
|                               .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 scope.version inverted index |                                            |      9.94257     |     9.94257     |       0           |     MB |     0.00% |
|                                   .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 scope.version doc values |                                            |      1.81071     |     1.81071     |       0           |     MB |     0.00% |
|                                        .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 scope.version total |                                            |     11.7533      |    11.7533      |       0           |     MB |     0.00% |
|                          .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 _metric_names_hash inverted index |                                            |      1.3139e+07  |     0           |      -1.3139e+07  |  bytes |  -100.00% |
|                              .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 _metric_names_hash doc values |                                            |    751.071       |   753.441       |       2.37012     |     kB |    +0.32% |
|                                   .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 _metric_names_hash total |                                            |  13582.1         |   753.441       |  -12828.7         |     kB |   -94.45% |
|                                  .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 scope.name inverted index |                                            |      1.25237e+07 |     0           |      -1.25237e+07 |  bytes |  -100.00% |
|                                      .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 scope.name doc values |                                            |      1.8737      |     1.87494     |       0.00124     |     MB |    +0.07% |
|                                           .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 scope.name total |                                            |     13.8172      |     1.87494     |     -11.9423      |     MB |   -86.43% |
|                                        .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 unit inverted index |                                            |      1.31542e+07 |     0           |      -1.31542e+07 |  bytes |  -100.00% |
|                                            .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 unit doc values |                                            |      1.96721     |     1.96944     |       0.00223     |     MB |    +0.11% |
|                                                 .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 unit total |                                            |     14.512       |     1.96944     |     -12.5426      |     MB |   -86.43% |
|                                 .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 start_timestamp doc values |                                            |     10.8147      |    10.8147      |       0           |     MB |     0.00% |
|                                     .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 start_timestamp points |                                            |      7.57459     |     7.57459     |       0           |     MB |     0.00% |
|                                      .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 start_timestamp total |                                            |     18.3893      |    18.3893      |       0           |     MB |     0.00% |
|               .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.name inverted index |                                            |      1.56712e+07 |     0           |      -1.56712e+07 |  bytes |  -100.00% |
|                   .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.name doc values |                                            |      4.44692     |     4.84916     |       0.40224     |     MB |    +9.05% |
|                        .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.name total |                                            |     19.3921      |     4.84916     |     -14.5429      |     MB |   -74.99% |
|              .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.disk.operation_time doc values |                                            |     21.9816      |    21.9816      |       0           |     MB |     0.00% |
|                   .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.disk.operation_time total |                                            |     21.9816      |    21.9816      |       0           |     MB |     0.00% |
|                        .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.cpu.time doc values |                                            |     32.9526      |    32.9526      |       0           |     MB |     0.00% |
|                             .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.process.cpu.time total |                                            |     32.9526      |    32.9526      |       0           |     MB |     0.00% |
|               .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.memory.utilization doc values |                                            |     65.8417      |    65.8417      |       0           |     MB |     0.00% |
|                    .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.memory.utilization total |                                            |     65.8417      |    65.8417      |       0           |     MB |     0.00% |
|                                      .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 @timestamp doc values |                                            |     95.405       |    95.405       |       0           |     MB |     0.00% |
|                                           .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 @timestamp total |                                            |     95.405       |    95.405       |       0           |     MB |     0.00% |
|                .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.mac inverted index |                                            |      9.44503e+07 |     0           |      -9.44503e+07 |  bytes |  -100.00% |
|                    .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.mac doc values |                                            |    128.782       |   130.374       |       1.5928      |     MB |    +1.24% |
|                         .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.mac total |                                            |    218.857       |   130.374       |     -88.4821      |     MB |   -40.43% |
|                         .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.cpu.time doc values |                                            |    438.675       |   438.675       |       0           |     MB |     0.00% |
|                              .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 metrics.system.cpu.time total |                                            |    438.675       |   438.675       |       0           |     MB |     0.00% |
|                                         .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 _seq_no doc values |                                            |    551.604       |   547.437       |      -4.16651     |     MB |    -0.76% |
|                                              .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 _seq_no total |                                            |    551.604       |   547.437       |      -4.16651     |     MB |    -0.76% |
|                     .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.ip doc values |                                            |    496.802       |   498.394       |       1.5928      |     MB |    +0.32% |
|                         .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.ip points |                                            |      4.25299e+08 |     0           |      -4.25299e+08 |  bytes |  -100.00% |
|                          .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 resource.attributes.host.ip total |                                            |    902.399       |   498.394       |    -404.004       |     MB |   -44.77% |
|                                         .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 _id inverted index |                                            |    919.122       |   919.122       |       0           |     MB |     0.00% |
|                                                  .ds-metrics-hostmetricsreceiver.otel-default-2025.10.23-000001 _id total |                                            |    919.122       |   919.122       |       0           |     MB |     0.00% |
|                                                                                                            Min Throughput |                                      index |  64594.9         | 78064.5         |   13469.6         | docs/s |   +20.85% |
|                                                                                                           Mean Throughput |                                      index |  65638           | 79126.8         |   13488.8         | docs/s |   +20.55% |
|                                                                                                         Median Throughput |                                      index |  65569.4         | 79124.3         |   13555           | docs/s |   +20.67% |
|                                                                                                            Max Throughput |                                      index |  69519.2         | 80238.7         |   10719.6         | docs/s |   +15.42% |
|                                                                                                   50th percentile latency |                                      index |    449.449       |   406.852       |     -42.5973      |     ms |    -9.48% |
|                                                                                                   90th percentile latency |                                      index |    688.44        |   568.422       |    -120.018       |     ms |   -17.43% |
|                                                                                                   99th percentile latency |                                      index |   4905.6         |  3003.81        |   -1901.79        |     ms |   -38.77% |
|                                                                                                 99.9th percentile latency |                                      index |   7389.81        |  5578.74        |   -1811.07        |     ms |   -24.51% |
|                                                                                                99.99th percentile latency |                                      index |  11379.5         |  7376.61        |   -4002.9         |     ms |   -35.18% |
|                                                                                                  100th percentile latency |                                      index |  14209.3         |  9824.95        |   -4384.38        |     ms |   -30.86% |
|                                                                                              50th percentile service time |                                      index |    449.591       |   406.365       |     -43.2253      |     ms |    -9.61% |
|                                                                                              90th percentile service time |                                      index |    701.029       |   569.301       |    -131.728       |     ms |   -18.79% |
|                                                                                              99th percentile service time |                                      index |   4913.92        |  2972.46        |   -1941.46        |     ms |   -39.51% |
|                                                                                            99.9th percentile service time |                                      index |   7398.43        |  5600.46        |   -1797.97        |     ms |   -24.30% |
|                                                                                           99.99th percentile service time |                                      index |  11379.5         |  7376.61        |   -4002.9         |     ms |   -35.18% |
|                                                                                             100th percentile service time |                                      index |  14209.3         |  9824.95        |   -4384.38        |     ms |   -30.86% |
|                                                                                                                error rate |                                      index |      0           |     0           |       0           |      % |     0.00% |
|                                                                                                            Min Throughput |                avg_avgot_memory_by_host_1h |      8.65748     |    12.1321      |       3.47466     |  ops/s |   +40.13% |
|                                                                                                           Mean Throughput |                avg_avgot_memory_by_host_1h |     13.7943      |    14.6632      |       0.86891     |  ops/s |    +6.30% |
|                                                                                                         Median Throughput |                avg_avgot_memory_by_host_1h |     14.3267      |    14.943       |       0.61628     |  ops/s |    +4.30% |
|                                                                                                            Max Throughput |                avg_avgot_memory_by_host_1h |     15.5971      |    15.5115      |      -0.0856      |  ops/s |    -0.55% |
|                                                                                                   50th percentile latency |                avg_avgot_memory_by_host_1h |     57.1113      |    61.1565      |       4.04524     |     ms |    +7.08% |
|                                                                                                   90th percentile latency |                avg_avgot_memory_by_host_1h |     62.5122      |    64.7026      |       2.19047     |     ms |    +3.50% |
|                                                                                                   99th percentile latency |                avg_avgot_memory_by_host_1h |     84.3541      |    68.8739      |     -15.4802      |     ms |   -18.35% |
|                                                                                                 99.9th percentile latency |                avg_avgot_memory_by_host_1h |     93.2046      |    75.4073      |     -17.7974      |     ms |   -19.09% |
|                                                                                                  100th percentile latency |                avg_avgot_memory_by_host_1h |    143.753       |    81.3521      |     -62.4006      |     ms |   -43.41% |
|                                                                                              50th percentile service time |                avg_avgot_memory_by_host_1h |     57.1156      |    61.1589      |       4.04333     |     ms |    +7.08% |
|                                                                                              90th percentile service time |                avg_avgot_memory_by_host_1h |     62.5226      |    64.7113      |       2.18876     |     ms |    +3.50% |
|                                                                                              99th percentile service time |                avg_avgot_memory_by_host_1h |     84.3728      |    68.8838      |     -15.489       |     ms |   -18.36% |
|                                                                                            99.9th percentile service time |                avg_avgot_memory_by_host_1h |     93.2046      |    75.4073      |     -17.7974      |     ms |   -19.09% |
|                                                                                             100th percentile service time |                avg_avgot_memory_by_host_1h |    143.753       |    81.3521      |     -62.4006      |     ms |   -43.41% |
|                                                                                                                error rate |                avg_avgot_memory_by_host_1h |      0           |     0           |       0           |      % |     0.00% |
|                                                                                                            Min Throughput |                    avg_rate_cpu_by_host_1h |      1.607       |     1.47396     |      -0.13304     |  ops/s |    -8.28% |
|                                                                                                           Mean Throughput |                    avg_rate_cpu_by_host_1h |      1.74319     |     1.57166     |      -0.17153     |  ops/s |    -9.84% |
|                                                                                                         Median Throughput |                    avg_rate_cpu_by_host_1h |      1.7599      |     1.58146     |      -0.17844     |  ops/s |   -10.14% |
|                                                                                                            Max Throughput |                    avg_rate_cpu_by_host_1h |      1.78143     |     1.60105     |      -0.18038     |  ops/s |   -10.13% |
|                                                                                                   50th percentile latency |                    avg_rate_cpu_by_host_1h |    545.742       |   613.115       |      67.3731      |     ms |   +12.35% |
|                                                                                                   90th percentile latency |                    avg_rate_cpu_by_host_1h |    585.516       |   648.46        |      62.9438      |     ms |   +10.75% |
|                                                                                                   99th percentile latency |                    avg_rate_cpu_by_host_1h |    612.033       |   676.487       |      64.4538      |     ms |   +10.53% |
|                                                                                                  100th percentile latency |                    avg_rate_cpu_by_host_1h |    626.191       |   701.034       |      74.8429      |     ms |   +11.95% |
|                                                                                              50th percentile service time |                    avg_rate_cpu_by_host_1h |    545.742       |   613.115       |      67.3731      |     ms |   +12.35% |
|                                                                                              90th percentile service time |                    avg_rate_cpu_by_host_1h |    585.516       |   648.46        |      62.9438      |     ms |   +10.75% |
|                                                                                              99th percentile service time |                    avg_rate_cpu_by_host_1h |    612.033       |   676.487       |      64.4538      |     ms |   +10.53% |
|                                                                                             100th percentile service time |                    avg_rate_cpu_by_host_1h |    626.191       |   701.034       |      74.8429      |     ms |   +11.95% |
|                                                                                                                error rate |                    avg_rate_cpu_by_host_1h |      0           |     0           |       0           |      % |     0.00% |
|                                                                                                            Min Throughput |                avg_avgot_memory_by_host_1m |      4.96097     |     4.36143     |      -0.59955     |  ops/s |   -12.09% |
|                                                                                                           Mean Throughput |                avg_avgot_memory_by_host_1m |      5.16703     |     4.44755     |      -0.71948     |  ops/s |   -13.92% |
|                                                                                                         Median Throughput |                avg_avgot_memory_by_host_1m |      5.19132     |     4.44999     |      -0.74133     |  ops/s |   -14.28% |
|                                                                                                            Max Throughput |                avg_avgot_memory_by_host_1m |      5.22961     |     4.49635     |      -0.73326     |  ops/s |   -14.02% |
|                                                                                                   50th percentile latency |                avg_avgot_memory_by_host_1m |    188.528       |   218.535       |      30.0074      |     ms |   +15.92% |
|                                                                                                   90th percentile latency |                avg_avgot_memory_by_host_1m |    192.42        |   232.661       |      40.2407      |     ms |   +20.91% |
|                                                                                                   99th percentile latency |                avg_avgot_memory_by_host_1m |    198.891       |   244.382       |      45.4901      |     ms |   +22.87% |
|                                                                                                  100th percentile latency |                avg_avgot_memory_by_host_1m |    241.827       |   263.924       |      22.0964      |     ms |    +9.14% |
|                                                                                              50th percentile service time |                avg_avgot_memory_by_host_1m |    188.528       |   218.535       |      30.0074      |     ms |   +15.92% |
|                                                                                              90th percentile service time |                avg_avgot_memory_by_host_1m |    192.42        |   232.661       |      40.2407      |     ms |   +20.91% |
|                                                                                              99th percentile service time |                avg_avgot_memory_by_host_1m |    198.891       |   244.382       |      45.4901      |     ms |   +22.87% |
|                                                                                             100th percentile service time |                avg_avgot_memory_by_host_1m |    241.827       |   263.924       |      22.0964      |     ms |    +9.14% |
|                                                                                                                error rate |                avg_avgot_memory_by_host_1m |      0           |     0           |       0           |      % |     0.00% |
|                                                                                                            Min Throughput |                    avg_rate_cpu_by_host_1m |      0.277752    |     0.274326    |      -0.00343     |  ops/s |    -1.23% |
|                                                                                                           Mean Throughput |                    avg_rate_cpu_by_host_1m |      0.278822    |     0.277946    |      -0.00088     |  ops/s |    -0.31% |
|                                                                                                         Median Throughput |                    avg_rate_cpu_by_host_1m |      0.278879    |     0.278313    |      -0.00057     |  ops/s |    -0.20% |
|                                                                                                            Max Throughput |                    avg_rate_cpu_by_host_1m |      0.279223    |     0.278819    |      -0.0004      |  ops/s |    -0.14% |
|                                                                                                   50th percentile latency |                    avg_rate_cpu_by_host_1m |   3573.39        |  3575.19        |       1.79614     |     ms |    +0.05% |
|                                                                                                   90th percentile latency |                    avg_rate_cpu_by_host_1m |   3605.86        |  3610.93        |       5.06956     |     ms |    +0.14% |
|                                                                                                  100th percentile latency |                    avg_rate_cpu_by_host_1m |   3636.36        |  3626.95        |      -9.40991     |     ms |    -0.26% |
|                                                                                              50th percentile service time |                    avg_rate_cpu_by_host_1m |   3573.39        |  3575.19        |       1.79614     |     ms |    +0.05% |
|                                                                                              90th percentile service time |                    avg_rate_cpu_by_host_1m |   3605.86        |  3610.93        |       5.06956     |     ms |    +0.14% |
|                                                                                             100th percentile service time |                    avg_rate_cpu_by_host_1m |   3636.36        |  3626.95        |      -9.40991     |     ms |    -0.26% |
|                                                                                                                error rate |                    avg_rate_cpu_by_host_1m |      0           |     0           |       0           |      % |     0.00% |
|                                                                                                            Min Throughput | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |    106.186       |    92.2683      |     -13.9181      |  ops/s |   -13.11% |
|                                                                                                           Mean Throughput | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |    112.766       |    95.3146      |     -17.451       |  ops/s |   -15.48% |
|                                                                                                         Median Throughput | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |    113.384       |    95.4661      |     -17.9181      |  ops/s |   -15.80% |
|                                                                                                            Max Throughput | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |    115.212       |    97.9082      |     -17.3033      |  ops/s |   -15.02% |
|                                                                                                   50th percentile latency | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |      7.96005     |     9.53068     |       1.57062     |     ms |   +19.73% |
|                                                                                                   90th percentile latency | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |      8.24853     |    10.4564      |       2.20784     |     ms |   +26.77% |
|                                                                                                   99th percentile latency | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |     11.383       |    12.7205      |       1.33757     |     ms |   +11.75% |
|                                                                                                 99.9th percentile latency | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |     28.6722      |    31.6895      |       3.01727     |     ms |   +10.52% |
|                                                                                                99.99th percentile latency | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |     34.1065      |    42.985       |       8.8785      |     ms |   +26.03% |
|                                                                                                  100th percentile latency | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |     40.6031      |    96.9706      |      56.3675      |     ms |  +138.83% |
|                                                                                              50th percentile service time | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |      7.96106     |     9.53084     |       1.56977     |     ms |   +19.72% |
|                                                                                              90th percentile service time | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |      8.24841     |    10.4557      |       2.20732     |     ms |   +26.76% |
|                                                                                              99th percentile service time | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |     11.0354      |    12.2693      |       1.23385     |     ms |   +11.18% |
|                                                                                            99.9th percentile service time | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |     28.3041      |    31.4437      |       3.13965     |     ms |   +11.09% |
|                                                                                           99.99th percentile service time | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |     34.1065      |    42.985       |       8.8785      |     ms |   +26.03% |
|                                                                                             100th percentile service time | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |     40.6031      |    96.9706      |      56.3675      |     ms |  +138.83% |
|                                                                                                                error rate | avg_avgot_cpu_load_1m_filtered_by_hosts_5m |      0           |     0           |       0           |      % |     0.00% |
|                                                                                                            Min Throughput |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |     19.9166      |    16.9486      |      -2.96795     |  ops/s |   -14.90% |
|                                                                                                           Mean Throughput |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |     21.7823      |    17.4016      |      -4.38069     |  ops/s |   -20.11% |
|                                                                                                         Median Throughput |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |     21.9835      |    17.4569      |      -4.52662     |  ops/s |   -20.59% |
|                                                                                                            Max Throughput |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |     22.349       |    17.5596      |      -4.78938     |  ops/s |   -21.43% |
|                                                                                                   50th percentile latency |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |     43.191       |    55.671       |      12.48        |     ms |   +28.89% |
|                                                                                                   90th percentile latency |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |     45.4172      |    58.275       |      12.8578      |     ms |   +28.31% |
|                                                                                                   99th percentile latency |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |     48.3212      |    61.92        |      13.5988      |     ms |   +28.14% |
|                                                                                                 99.9th percentile latency |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |     50.0389      |    82.2649      |      32.226       |     ms |   +64.40% |
|                                                                                                  100th percentile latency |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |     50.8054      |    91.6899      |      40.8845      |     ms |   +80.47% |
|                                                                                              50th percentile service time |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |     43.192       |    55.6689      |      12.477       |     ms |   +28.89% |
|                                                                                              90th percentile service time |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |     45.4039      |    58.2701      |      12.8662      |     ms |   +28.34% |
|                                                                                              99th percentile service time |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |     48.3399      |    61.9306      |      13.5907      |     ms |   +28.11% |
|                                                                                            99.9th percentile service time |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |     50.0389      |    82.2649      |      32.226       |     ms |   +64.40% |
|                                                                                             100th percentile service time |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |     50.8054      |    91.6899      |      40.8845      |     ms |   +80.47% |
|                                                                                                                error rate |   avg_avgot_cpu_load_1m_prefix_by_hosts_5m |      0           |     0           |       0           |      % |     0.00% |
|                                                                                                            Min Throughput |      sum_rate_sys_cpu_time_large_clause_5m |      0.180514    |     0.188035    |       0.00752     |  ops/s |    +4.17% |
|                                                                                                           Mean Throughput |      sum_rate_sys_cpu_time_large_clause_5m |      0.187528    |     0.205424    |       0.0179      |  ops/s |    +9.54% |
|                                                                                                         Median Throughput |      sum_rate_sys_cpu_time_large_clause_5m |      0.18854     |     0.207657    |       0.01912     |  ops/s |   +10.14% |
|                                                                                                            Max Throughput |      sum_rate_sys_cpu_time_large_clause_5m |      0.189867    |     0.211108    |       0.02124     |  ops/s |   +11.19% |
|                                                                                                   50th percentile latency |      sum_rate_sys_cpu_time_large_clause_5m |   5156.84        |  4586.03        |    -570.813       |     ms |   -11.07% |
|                                                                                                   90th percentile latency |      sum_rate_sys_cpu_time_large_clause_5m |   5204.76        |  4651.77        |    -552.989       |     ms |   -10.62% |
|                                                                                                  100th percentile latency |      sum_rate_sys_cpu_time_large_clause_5m |   5234.7         |  4690           |    -544.703       |     ms |   -10.41% |
|                                                                                              50th percentile service time |      sum_rate_sys_cpu_time_large_clause_5m |   5156.84        |  4586.03        |    -570.813       |     ms |   -11.07% |
|                                                                                              90th percentile service time |      sum_rate_sys_cpu_time_large_clause_5m |   5204.76        |  4651.77        |    -552.989       |     ms |   -10.62% |
|                                                                                             100th percentile service time |      sum_rate_sys_cpu_time_large_clause_5m |   5234.7         |  4690           |    -544.703       |     ms |   -10.41% |
|                                                                                                                error rate |      sum_rate_sys_cpu_time_large_clause_5m |      0           |     0           |       0           |      % |     0.00% |
|                                                                                                            Min Throughput |              sum_lot_filesystem_usage_top5 |     15.3948      |    12.6893      |      -2.70554     |  ops/s |   -17.57% |
|                                                                                                           Mean Throughput |              sum_lot_filesystem_usage_top5 |     16.2367      |    13.5184      |      -2.71827     |  ops/s |   -16.74% |
|                                                                                                         Median Throughput |              sum_lot_filesystem_usage_top5 |     16.3375      |    13.6122      |      -2.7253      |  ops/s |   -16.68% |
|                                                                                                            Max Throughput |              sum_lot_filesystem_usage_top5 |     16.4756      |    13.758       |      -2.71764     |  ops/s |   -16.49% |
|                                                                                                   50th percentile latency |              sum_lot_filesystem_usage_top5 |     57.3326      |    69.447       |      12.1144      |     ms |   +21.13% |
|                                                                                                   90th percentile latency |              sum_lot_filesystem_usage_top5 |     69.3777      |    79.9172      |      10.5395      |     ms |   +15.19% |
|                                                                                                   99th percentile latency |              sum_lot_filesystem_usage_top5 |     72.8638      |    83.7886      |      10.9248      |     ms |   +14.99% |
|                                                                                                 99.9th percentile latency |              sum_lot_filesystem_usage_top5 |     75.7118      |    85.5959      |       9.88407     |     ms |   +13.05% |
|                                                                                                  100th percentile latency |              sum_lot_filesystem_usage_top5 |     77.402       |    86.171       |       8.76896     |     ms |   +11.33% |
|                                                                                              50th percentile service time |              sum_lot_filesystem_usage_top5 |     57.346       |    69.5866      |      12.2406      |     ms |   +21.35% |
|                                                                                              90th percentile service time |              sum_lot_filesystem_usage_top5 |     69.3806      |    79.9215      |      10.5409      |     ms |   +15.19% |
|                                                                                              99th percentile service time |              sum_lot_filesystem_usage_top5 |     72.8654      |    83.7663      |      10.9009      |     ms |   +14.96% |
|                                                                                            99.9th percentile service time |              sum_lot_filesystem_usage_top5 |     75.7118      |    85.5959      |       9.88407     |     ms |   +13.05% |
|                                                                                             100th percentile service time |              sum_lot_filesystem_usage_top5 |     77.402       |    86.171       |       8.76896     |     ms |   +11.33% |
|                                                                                                                error rate |              sum_lot_filesystem_usage_top5 |      0           |     0           |       0           |      % |     0.00% |

@felixbarny
Copy link
Member

I've extracted flame graphs for two queries that have regressed the most:

avg_avgot_cpu_load_1m_filtered_by_hosts_5m-main.html
avg_avgot_cpu_load_1m_filtered_by_hosts_5m-skippers.html

avg_avgot_cpu_load_1m_prefix_by_hosts_5m-main.html
avg_avgot_cpu_load_1m_prefix_by_hosts_5m-skippers.html

What both have in common that they're using a two phase iterator in the skippers version. The prefix_by_hosts query is also doing a lot of UTF-8 to UTF-16 conversions that it didn't do before.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants