Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Shard query splitting: Split queries by stream shards #715

Merged
merged 53 commits into from
Sep 11, 2024
Merged

Shard query splitting: Split queries by stream shards #715

merged 53 commits into from
Sep 11, 2024

Commits on Sep 11, 2024

  1. feat(CombineResponses): create module

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    14d614a View commit details
    Browse the repository at this point in the history

  2. test(CombineResponses): add unit tests and adjust implementation

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    02ab877 View commit details
    Browse the repository at this point in the history

  3. chore: install uuid

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    7ebdfd0 View commit details
    Browse the repository at this point in the history

  4. feat(shardQuerySplitting): create module

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    001c0d2 View commit details
    Browse the repository at this point in the history

  5. chore: interpolate queries

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    8c40a09 View commit details
    Browse the repository at this point in the history

  6. feat(shardQuerySplitting): query from high to low shard

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    b93b873 View commit details
    Browse the repository at this point in the history

  7. fix(combineResponses): improve nanos handling

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    73f141c View commit details
    Browse the repository at this point in the history

  8. feat(shardQuerySplitting): group shards in requests

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    258b7f8 View commit details
    Browse the repository at this point in the history

  9. feat(mergeResponses): properly calculate dataframe length

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    ce3402e View commit details
    Browse the repository at this point in the history

  10. feat(shardQuerySplitting): retry failed requests

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    a305dfc View commit details
    Browse the repository at this point in the history

  11. feat(mergeResponses): compare timestamps using nanoseconds if present

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    aa29e4e View commit details
    Browse the repository at this point in the history

  12. test(combineResponses): add extra test case

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    dd0d805 View commit details
    Browse the repository at this point in the history

  13. feat(mergeResponses): fix frame length calculation

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    3e07749 View commit details
    Browse the repository at this point in the history

  14. feat(shardQuerySplitting): sort shards desc

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    ef009ab View commit details
    Browse the repository at this point in the history

  15. fix(shardQuerySplitting): increase retries

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    e97ae94 View commit details
    Browse the repository at this point in the history

  16. feat(shardQuerySplitting): adjust shards based on timerange

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    3fc44b2 View commit details
    Browse the repository at this point in the history

  17. chore(ci): simplify

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    1bd685d View commit details
    Browse the repository at this point in the history

  18. feat(shardQuerySplitting): dynamically calculate max requests

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    4c7d95d View commit details
    Browse the repository at this point in the history

  19. fix(shardQuerySplitting): do not rely on max shard as shards length

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    a689d90 View commit details
    Browse the repository at this point in the history

  20. feat(shardQuerySplitting): pass service name to values call

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    6069305 View commit details
    Browse the repository at this point in the history

  21. feat(shardQuerySplitting): do not retry if maximum series reached

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    7e030d1 View commit details
    Browse the repository at this point in the history

  22. feat(combineResponses): consider id when comparing log timestamps

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    8c64a37 View commit details
    Browse the repository at this point in the history

  23. chore: print shards and not max shard

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    86e6af3 View commit details
    Browse the repository at this point in the history

  24. chore: remove max shard

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    113fe61 View commit details
    Browse the repository at this point in the history

  25. fix(combineResponses): fix nanos recombination

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    abdc129 View commit details
    Browse the repository at this point in the history

  26. chore: words

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    145797c View commit details
    Browse the repository at this point in the history

  27. feat(querying): exclude log queries from sharding

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    47b5501 View commit details
    Browse the repository at this point in the history

  28. fix(getShardRequests): debug group generation

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    0066811 View commit details
    Browse the repository at this point in the history

  29. feat(shardQuerySplitting): integrate feature flag

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    deb533d View commit details
    Browse the repository at this point in the history

  30. feat(shardQuerySplitting): fix falling back to regular query

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    f1bc31b View commit details
    Browse the repository at this point in the history

  31. chore: rename functions

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    d786308 View commit details
    Browse the repository at this point in the history

  32. feat(jest): add observable matchers

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    9c3977b View commit details
    Browse the repository at this point in the history

  33. test(shardQuerySplitting): add base test setup

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    23a8170 View commit details
    Browse the repository at this point in the history

  34. test(shardQuerySplitting): add more tests

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    8b4d8eb View commit details
    Browse the repository at this point in the history

  35. Revert "chore(ci): simplify" 

    This reverts commit c6c2cd0.
    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    896b64f View commit details
    Browse the repository at this point in the history

  36. chore: spelling

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    563767a View commit details
    Browse the repository at this point in the history

  37. chore(shardQuerySplitting): add docs

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    94c3a6e View commit details
    Browse the repository at this point in the history

  38. chore: simplify

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    1d32c98 View commit details
    Browse the repository at this point in the history

  39. chore: fix test title

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    1f51768 View commit details
    Browse the repository at this point in the history

  40. fix(logql): improve selector for single shard

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    37bc0e8 View commit details
    Browse the repository at this point in the history

  41. feat(shardQuerySpitting): try new sharding approach

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    65cb27e View commit details
    Browse the repository at this point in the history

  42. feat(shardQuerySpitting): improve shard volume spreading

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    f457123 View commit details
    Browse the repository at this point in the history

  43. chore: spelling

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    de87626 View commit details
    Browse the repository at this point in the history

  44. feat(shardQuerySplitting): start with the highest numbers

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    0705076 View commit details
    Browse the repository at this point in the history

  45. fix(combineResponses): small optimization

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    284af9f View commit details
    Browse the repository at this point in the history

  46. feat(shardQuerySplitting): implement incremental backoff

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    b07278c View commit details
    Browse the repository at this point in the history

  47. chore: words

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    dd4ba97 View commit details
    Browse the repository at this point in the history

  48. fix(shardQuerySplitting): skip avg from sharding

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    ef38488 View commit details
    Browse the repository at this point in the history

  49. fix(shardQuerySplitting): start with loading state Loading and switch… 

    … to Streaming
    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    bbe917c View commit details
    Browse the repository at this point in the history

  50. chore: update e2e

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    d968b23 View commit details
    Browse the repository at this point in the history

  51. chore: update e2e tests

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    bcbf11c View commit details
    Browse the repository at this point in the history

  52. chore: disable exploreLogsShardSplitting in ci

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    333f7d5 View commit details
    Browse the repository at this point in the history

  53. chore: revert test update

    @matyax
    matyax committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    9a91c2d View commit details
    Browse the repository at this point in the history