Skip to content

chore(test): adjust order of vulnerabilities after a backend change #1205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mrrajan merged 1 commit into from
Aug 13, 2026
Merged

chore(test): adjust order of vulnerabilities after a backend change #1205

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion e2e/tests/ui/pages/home/attention.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ import { login } from "../../helpers/Auth";
import { HomePage } from "./HomePage";

const EXPECTED_TOP_3 = [
{ identifier: "CVE-2026-18236", severity: /critical/i },

@sourcery-ai sourcery-ai Bot Aug 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Consider making the test resilient to ordering changes when vulnerabilities share the same severity.

The test currently depends on the exact order of the two critical vulnerabilities. Since the backend has already changed this once, it would be more robust to assert that both critical CVEs appear in the "top" vulnerabilities section (and are marked as critical) without relying on their relative order—unless the product explicitly guarantees a specific sort criterion. If ordering is guaranteed, please add a brief comment in the test documenting the expected sort key so future backend changes don’t unintentionally break CI.

Suggested implementation:

import { HomePage } from "./HomePage";

/**
 * Expected vulnerabilities in the "Top vulnerabilities" section.
 *
 * Do not rely on the relative ordering of vulnerabilities that share
 * the same severity (e.g. multiple critical CVEs): backend sorting may
 * change as long as the section still contains the same set of items.
 */
const EXPECTED_TOP_3 = [
  { identifier: "CVE-2026-63223", severity: /critical/i },
  { identifier: "CVE-2026-18236", severity: /critical/i },
  { identifier: "CVE-2026-18358", severity: /high/i },
] as const;

/**
 * Subset of EXPECTED_TOP_3 that should be marked as critical.
 * Tests should use this to assert presence and severity of critical CVEs
 * without depending on their order in the list.
 */
const EXPECTED_CRITICAL_TOP_VULNERABILITIES = EXPECTED_TOP_3.filter(
  ({ severity }) => severity.test("critical"),
);

To fully implement the suggestion and make the test resilient to ordering changes:

  1. Update the assertion that checks the "Top vulnerabilities" list so that it does not assert exact ordering for the critical CVEs.
    • Instead of something like expect(renderedTopVulns).toEqual(EXPECTED_TOP_3), use a combination of:
      • expect(renderedTopVulns).toEqual(expect.arrayContaining(EXPECTED_TOP_3)); if you only care about membership, or
      • expect(renderedTopVulns.map(v => v.identifier)).toEqual(expect.arrayContaining(EXPECTED_CRITICAL_TOP_VULNERABILITIES.map(v => v.identifier)));
        plus explicit severity checks per item.
  2. Ensure the test verifies that:
    • All identifiers from EXPECTED_CRITICAL_TOP_VULNERABILITIES appear in the "top vulnerabilities" section.
    • Each of those entries is marked as critical (matching the /critical/i severity).
    • The test does not assume any specific relative order between the critical CVEs.
  3. If the product does guarantee a specific sort key (e.g. severity desc, exploitability desc, identifier asc), add a short inline comment near the assertion documenting that sort criterion so backend changes do not unintentionally break CI.

{ identifier: "CVE-2026-63223", severity: /critical/i },
{ identifier: "CVE-2026-18236", severity: /critical/i },
{ identifier: "CVE-2026-18358", severity: /high/i },
] as const;

Expand Down