Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patterns: Use runtime datasource for queries #672

Merged
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
9e7aa16
chore: refactor to allow multiple queries
gtk-grafana Aug 6, 2024
d55c9ac
chore: wip - major functionality implemented
gtk-grafana Aug 6, 2024
80879ca
Merge remote-tracking branch 'origin/main' into gtk-grafana/issues/67…
gtk-grafana Aug 7, 2024
bc010ee
chore: refactor patterns
gtk-grafana Aug 7, 2024
b786d67
chore: clean up
gtk-grafana Aug 7, 2024
468361b
chore: clean up
gtk-grafana Aug 7, 2024
af3673f
Merge remote-tracking branch 'origin/main' into gtk-grafana/issues/67…
gtk-grafana Aug 7, 2024
6bda2fd
test: fix test
gtk-grafana Aug 7, 2024
a34dc57
fix: refactor and fix buggy re-rendering in logs panels
gtk-grafana Aug 8, 2024
d3f1486
fix: fix data node not being set from scene cache on route/activation
gtk-grafana Aug 8, 2024
1c2e57b
fix: removing multiple queries per query runner, adding multiple quer…
gtk-grafana Aug 9, 2024
c3a6182
chore: remove console.log
gtk-grafana Aug 9, 2024
365a4ef
chore: docs
gtk-grafana Aug 9, 2024
982d76c
chore: why are the videos gone in failed playwright reports?
gtk-grafana Aug 9, 2024
1ef913a
chore: clean up
gtk-grafana Aug 9, 2024
7bcf816
chore: clean up
gtk-grafana Aug 9, 2024
1f4d517
fix: fix table loading state
gtk-grafana Aug 9, 2024
bf8c10a
chore: clean up
gtk-grafana Aug 9, 2024
9a9a3e0
chore: clean up
gtk-grafana Aug 9, 2024
6a8241a
chore: clean up def
gtk-grafana Aug 9, 2024
e8bbf8c
chore: clean up datasource
gtk-grafana Aug 9, 2024
d2907f0
chore: clean up
gtk-grafana Aug 9, 2024
bc33f3b
Merge remote-tracking branch 'origin/main' into gtk-grafana/issues/67…
gtk-grafana Aug 9, 2024
8e5314c
chore: add loading state
gtk-grafana Aug 9, 2024
a851425
chore: add the dang video
gtk-grafana Aug 9, 2024
41f90e8
chore: remove e2e video in ci
gtk-grafana Aug 12, 2024
d49c8c2
Merge branch 'main' into gtk-grafana/issues/670/patterns-runtime-reso…
gtk-grafana Aug 12, 2024
c4d63aa
fix: dont run when query is already running, or we have cached state
gtk-grafana Aug 12, 2024
9733c28
chore: clean up extra rendering
gtk-grafana Aug 13, 2024
9a5f2ca
chore: remove console.warn
gtk-grafana Aug 13, 2024
4447791
chore: refactor, clean up, remove getPatternsFrames
gtk-grafana Aug 13, 2024
3d56793
chore: rename LogsActionBarScene to ActionBarScene
gtk-grafana Aug 13, 2024
b6cfe4d
chore: refactor counter ternary
gtk-grafana Aug 13, 2024
e9cd5d7
chore: remove nested if
gtk-grafana Aug 13, 2024
080bf3d
chore: clean up
gtk-grafana Aug 13, 2024
5e5101c
chore: partition queries by resource
gtk-grafana Aug 13, 2024
3101bf8
chore: add assertion that queries do not have multiple targets
gtk-grafana Aug 13, 2024
db0b7a7
chore: refactor
gtk-grafana Aug 13, 2024
96f1daf
chore: no need to iterate
gtk-grafana Aug 13, 2024
24a6060
chore: move patterns files
gtk-grafana Aug 13, 2024
4508fc2
chore: update types, prevent runtime error
gtk-grafana Aug 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions project-words.txt
Original file line number Diff line number Diff line change
Expand Up @@ -469,4 +469,5 @@ subqueries
dbscan
svgs
Dont
REFID
unroute
4 changes: 3 additions & 1 deletion src/Components/IndexScene/IndexScene.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,9 @@ function getContentScene(drillDownLabel?: string) {
return new ServiceSelectionScene({});
}

return new ServiceScene({ drillDownLabel });
return new ServiceScene({
drillDownLabel,
});
}

function getVariableSet(initialDatasourceUid: string, initialFilters?: AdHocVariableFilter[]) {
Expand Down
110 changes: 110 additions & 0 deletions src/Components/ServiceScene/ActionBarScene.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
import { SceneComponentProps, sceneGraph, SceneObjectBase, SceneObjectState } from '@grafana/scenes';
import { Box, Stack, Tab, TabsBar, useStyles2 } from '@grafana/ui';
import { getExplorationFor } from '../../services/scenes';
import { getDrilldownSlug, getDrilldownValueSlug, PageSlugs, ValueSlugs } from '../../services/routing';
import { GoToExploreButton } from './GoToExploreButton';
import { reportAppInteraction, USER_EVENTS_ACTIONS, USER_EVENTS_PAGES } from '../../services/analytics';
import { ALL_VARIABLE_VALUE, getLabelsVariable } from '../../services/variables';
import { SERVICE_NAME } from '../ServiceSelectionScene/ServiceSelectionScene';
import { navigateToDrilldownPage, navigateToIndex } from '../../services/navigate';
import React from 'react';
import { ServiceScene, ServiceSceneState } from './ServiceScene';
import { GrafanaTheme2 } from '@grafana/data';
import { css } from '@emotion/css';
import { BreakdownViewDefinition, breakdownViewsDefinitions } from './BreakdownViews';

export interface ActionBarSceneState extends SceneObjectState {}

export class ActionBarScene extends SceneObjectBase<ActionBarSceneState> {
public static Component = ({ model }: SceneComponentProps<ActionBarScene>) => {
const styles = useStyles2(getStyles);
const exploration = getExplorationFor(model);
let currentBreakdownViewSlug = getDrilldownSlug();
let allowNavToParent = false;

if (!Object.values(PageSlugs).includes(currentBreakdownViewSlug)) {
const drilldownValueSlug = getDrilldownValueSlug();
allowNavToParent = true;
if (drilldownValueSlug === ValueSlugs.field) {
currentBreakdownViewSlug = PageSlugs.fields;
}
if (drilldownValueSlug === ValueSlugs.label) {
currentBreakdownViewSlug = PageSlugs.labels;
}
}

const serviceScene = sceneGraph.getAncestor(model, ServiceScene);
const { loading, $data, ...state } = serviceScene.useState();

return (
<Box paddingY={0}>
<div className={styles.actions}>
<Stack gap={1}>
<GoToExploreButton exploration={exploration} />
</Stack>
</div>

<TabsBar>
{breakdownViewsDefinitions.map((tab, index) => {
return (
<Tab
data-testid={tab.testId}
key={index}
label={tab.displayName}
active={currentBreakdownViewSlug === tab.value}
counter={loading ? undefined : getCounter(tab, { ...state, $data })}
icon={loading ? 'spinner' : undefined}
onChangeTab={() => {
if ((tab.value && tab.value !== currentBreakdownViewSlug) || allowNavToParent) {
reportAppInteraction(
USER_EVENTS_PAGES.service_details,
USER_EVENTS_ACTIONS.service_details.action_view_changed,
{
newActionView: tab.value,
previousActionView: currentBreakdownViewSlug,
}
);

const serviceScene = sceneGraph.getAncestor(model, ServiceScene);
const variable = getLabelsVariable(serviceScene);
const service = variable.state.filters.find((f) => f.key === SERVICE_NAME);

if (service?.value) {
navigateToDrilldownPage(tab.value, serviceScene);
} else {
navigateToIndex();
}
}
}}
/>
);
})}
</TabsBar>
</Box>
);
};
}
const getCounter = (tab: BreakdownViewDefinition, state: ServiceSceneState) => {
switch (tab.value) {
case 'fields':
return state.fieldsCount ?? (state.fields?.filter((l) => l !== ALL_VARIABLE_VALUE) ?? []).length;
case 'patterns':
return state.patternsCount;
case 'labels':
return (state.labels?.filter((l) => l.label !== ALL_VARIABLE_VALUE) ?? []).length;
default:
return undefined;
}
};

function getStyles(theme: GrafanaTheme2) {
return {
actions: css({
[theme.breakpoints.up(theme.breakpoints.values.md)]: {
position: 'absolute',
right: 0,
zIndex: 2,
},
}),
};
}
120 changes: 120 additions & 0 deletions src/Components/ServiceScene/BreakdownViews.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
import { PageSlugs, ValueSlugs } from '../../services/routing';
import { LogsListScene } from './LogsListScene';
import { testIds } from '../../services/testIds';
import { buildLabelValuesBreakdownActionScene, LabelBreakdownScene } from './Breakdowns/LabelBreakdownScene';
import { FieldsBreakdownScene } from './Breakdowns/FieldsBreakdownScene';
import { PatternsBreakdownScene } from './Breakdowns/Patterns/PatternsBreakdownScene';
import { SceneFlexItem, SceneFlexLayout, SceneObject } from '@grafana/scenes';
import { LogsVolumePanel } from './LogsVolumePanel';

interface ValueBreakdownViewDefinition {
displayName: string;
value: ValueSlugs;
testId: string;
getScene: (value: string) => SceneObject;
}

export interface BreakdownViewDefinition {
displayName: string;
value: PageSlugs;
testId: string;
getScene: (changeFields: (f: string[]) => void) => SceneObject;
}

export const breakdownViewsDefinitions: BreakdownViewDefinition[] = [
{
displayName: 'Logs',
value: PageSlugs.logs,
getScene: () => buildLogsListScene(),
testId: testIds.exploreServiceDetails.tabLogs,
},
{
displayName: 'Labels',
value: PageSlugs.labels,
getScene: () => buildLabelBreakdownActionScene(),
testId: testIds.exploreServiceDetails.tabLabels,
},
{
displayName: 'Fields',
value: PageSlugs.fields,
getScene: (f) => buildFieldsBreakdownActionScene(f),
testId: testIds.exploreServiceDetails.tabFields,
},
{
displayName: 'Patterns',
value: PageSlugs.patterns,
getScene: () => buildPatternsScene(),
testId: testIds.exploreServiceDetails.tabPatterns,
},
];
export const valueBreakdownViews: ValueBreakdownViewDefinition[] = [
{
displayName: 'Label',
value: ValueSlugs.label,
getScene: (value: string) => buildLabelValuesBreakdownActionScene(value),
testId: testIds.exploreServiceDetails.tabLabels,
},
{
displayName: 'Field',
value: ValueSlugs.field,
getScene: (value: string) => buildFieldValuesBreakdownActionScene(value),
testId: testIds.exploreServiceDetails.tabFields,
},
];

function buildPatternsScene() {
return new SceneFlexLayout({
children: [
new SceneFlexItem({
body: new PatternsBreakdownScene({}),
}),
],
});
}

function buildFieldsBreakdownActionScene(changeFieldNumber: (n: string[]) => void) {
return new SceneFlexLayout({
children: [
new SceneFlexItem({
body: new FieldsBreakdownScene({ changeFields: changeFieldNumber }),
}),
],
});
}

function buildFieldValuesBreakdownActionScene(value: string) {
return new SceneFlexLayout({
children: [
new SceneFlexItem({
body: new FieldsBreakdownScene({ value }),
}),
],
});
}

function buildLogsListScene() {
return new SceneFlexLayout({
direction: 'column',
children: [
new SceneFlexItem({
minHeight: 200,
body: new LogsVolumePanel({}),
}),
new SceneFlexItem({
minHeight: '470px',
height: 'calc(100vh - 500px)',
body: new LogsListScene({}),
}),
],
});
}

function buildLabelBreakdownActionScene() {
return new SceneFlexLayout({
children: [
new SceneFlexItem({
body: new LabelBreakdownScene({}),
}),
],
});
}
24 changes: 2 additions & 22 deletions src/Components/ServiceScene/Breakdowns/FieldsBreakdownScene.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,7 @@ export class FieldsBreakdownScene extends SceneObjectBase<FieldsBreakdownSceneSt
legendFormat: `{{${optionValue}}}`,
refId: optionValue,
});
const queryRunner = getQueryRunner(query);
const queryRunner = getQueryRunner([query]);
let body = PanelBuilders.timeseries().setTitle(optionValue).setData(queryRunner);

if (!isAvgField(optionValue)) {
Expand Down Expand Up @@ -348,7 +348,7 @@ export class FieldsBreakdownScene extends SceneObjectBase<FieldsBreakdownSceneSt
const getFilter = () => this.state.search.state.filter ?? '';

return new LayoutSwitcher({
$data: getQueryRunner(query),
$data: getQueryRunner([query]),
options: [
{ value: 'single', label: 'Single' },
{ value: 'grid', label: 'Grid' },
Expand Down Expand Up @@ -522,26 +522,6 @@ function getExpr(field: string) {

const GRID_TEMPLATE_COLUMNS = 'repeat(auto-fit, minmax(400px, 1fr))';

export function buildFieldsBreakdownActionScene(changeFieldNumber: (n: string[]) => void) {
return new SceneFlexLayout({
children: [
new SceneFlexItem({
body: new FieldsBreakdownScene({ changeFields: changeFieldNumber }),
}),
],
});
}

export function buildFieldValuesBreakdownActionScene(value: string) {
return new SceneFlexLayout({
children: [
new SceneFlexItem({
body: new FieldsBreakdownScene({ value }),
}),
],
});
}

interface SelectLabelActionState extends SceneObjectState {
labelName: string;
}
Expand Down
18 changes: 4 additions & 14 deletions src/Components/ServiceScene/Breakdowns/LabelBreakdownScene.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -218,9 +218,9 @@ export class LabelBreakdownScene extends SceneObjectBase<LabelBreakdownSceneStat
body: PanelBuilders.timeseries()
.setTitle(optionValue)
.setData(
getQueryRunner(
buildDataQuery(getTimeSeriesExpr(this, optionValue), { legendFormat: `{{${optionValue}}}` })
)
getQueryRunner([
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need multiple queries anymore, and I would highly suggest we don't mix queries that call different API endpoints, but if we wanted to include multiple queries that call the same API endpoint this should work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first glance, I like this ability of running more than one query. What was the original purpose of this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially I tried to smoosh all the queries into $data, which was a bad idea as queries would end at different times, but if we have multiple data queries, that should work fine as they are all returned from the backend at the same time

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes sense for "combined panels", i.e. the ones at the service selection scene, where we do one logs and one metrics query for each service 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, that's probably worth opening in another issue. Might be good for the lazy-loaded panels as well to at least batch things up a bit.

buildDataQuery(getTimeSeriesExpr(this, optionValue), { legendFormat: `{{${optionValue}}}` }),
])
)
.setHeaderActions(new SelectLabelAction({ labelName: optionValue }))
.setCustomFieldConfig('stacking', { mode: StackingMode.Normal })
Expand Down Expand Up @@ -276,7 +276,7 @@ export class LabelBreakdownScene extends SceneObjectBase<LabelBreakdownSceneStat
const getFilter = () => this.state.search.state.filter ?? '';

return new LayoutSwitcher({
$data: getQueryRunner(query),
$data: getQueryRunner([query]),
options: [
{ value: 'single', label: 'Single' },
{ value: 'grid', label: 'Grid' },
Expand Down Expand Up @@ -424,16 +424,6 @@ function getStyles(theme: GrafanaTheme2) {

const GRID_TEMPLATE_COLUMNS = 'repeat(auto-fit, minmax(400px, 1fr))';

export function buildLabelBreakdownActionScene() {
return new SceneFlexLayout({
children: [
new SceneFlexItem({
body: new LabelBreakdownScene({}),
}),
],
});
}

export function buildLabelValuesBreakdownActionScene(value: string) {
return new SceneFlexLayout({
children: [
Expand Down
Loading
Loading