Skip to content

[Docs] Add Slurm cluster deployment to Qwen3.5 cookbook#31572

Draft
nvpohanh wants to merge 1 commit into
sgl-project:mainfrom
nvpohanh:docs/qwen3.5-slurm-deployment
Draft

[Docs] Add Slurm cluster deployment to Qwen3.5 cookbook#31572
nvpohanh wants to merge 1 commit into
sgl-project:mainfrom
nvpohanh:docs/qwen3.5-slurm-deployment

Conversation

@nvpohanh

@nvpohanh nvpohanh commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

[by Codex]

Motivation

Add copy/pasteable Slurm-cluster deployment instructions to the Qwen3.5 cookbook, including curated GB300 SGLang prefill/decode-disaggregated and agentic serving recipes launched with the NVIDIA/srt-slurm tool.

Modifications

  • add Deployment in A Slurm Cluster after the existing Qwen3.5 basic deployment generator
  • document NVIDIA/srt-slurm installation, GB300 setup, model/container aliases, dry-run and submit commands, and the optional InferenceX AgentX harness
  • add an interactive picker for three FP8 1K/1K, three FP8 8K/1K, and three NVFP4 agentic operating points
  • link the complete ten fixed-length and nine agentic recipe matrix from NVIDIA/srt-slurm-recipes#2

This draft currently has two temporary dependencies:

  1. sgl-project/sglang#30996 introduces the shared NVIDIA/srt-slurm picker. This PR carries a byte-identical copy so it can be previewed independently; after [Docs] Add Slurm cluster deployment to DeepSeek-V4 cookbook #30996 merges, this branch will be rebased and the duplicate will disappear from the diff.
  2. Add Qwen3.5 GB300 SGLang recipes NVIDIA/srt-slurm-recipes#2 is still under review, so recipe links and clone instructions temporarily point to nvpohanh/srt-slurm-recipes:add-qwen35-gb300-agentic-sglang-recipes. They will be switched to the upstream repository after that PR merges.

Accuracy Tests

Not applicable; documentation-only change.

Speed Tests and Profiling

Not applicable; documentation-only change. The linked recipes retain their commit-pinned InferenceX benchmark configurations.

Validation

Checklist


CI States

Latest PR Test (Base): ✅ Run #29566189211
Latest PR Test (Extra): ❌ Run #29566189097

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Jul 17, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds documentation and an interactive React component (SrtSlurmDeployment) for deploying Qwen3.5 on a Slurm cluster using the NVIDIA/srt-slurm tool. Feedback on the PR highlights a critical issue in the new React component where useState and useEffect are used without being imported, and are invoked after a conditional return, violating the Rules of Hooks.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +1 to +157
// Interactive NVIDIA/srt-slurm recipe picker for cookbook deployment sections.
// Keep model-specific data in a plain config module and pass it through MDX.
//
// Config fields:
// model/hardware fixed display chips
// workloads/strategies selectable dimensions
// recipes {workload, strategy, path, prefillNodes,
// decodeNodes, servingNodes, infrastructureNodes,
// description, repositoryUrl, branch, status,
// benchmark}[]
// repositoryUrl/branch source link for the selected YAML
// recipesPath local path used in generated srtctl commands

export const SrtSlurmDeployment = ({ config }) => {
if (
!config ||
!config.model ||
!config.hardware ||
!Array.isArray(config.workloads) ||
!Array.isArray(config.strategies) ||
!Array.isArray(config.recipes) ||
config.recipes.length === 0
) {
return (
<div style={{padding: 12, color: "#b91c1c"}}>
NVIDIA/srt-slurm deployment: missing recipe configuration
</div>
);
}

const makeStyles = (isDark) => ({
container: {
maxWidth: "900px", margin: "0 auto", display: "flex",
flexDirection: "column", gap: "3px",
},
card: {
padding: "5px 10px",
border: `1px solid ${isDark ? "#374151" : "#e5e7eb"}`,
borderLeft: `3px solid ${isDark ? "#E85D4D" : "#D45D44"}`,
borderRadius: "4px", display: "flex", alignItems: "center", gap: "10px",
background: isDark ? "#1f2937" : "#fff",
},
title: {
fontSize: "12px", fontWeight: 600, minWidth: "108px", flexShrink: 0,
color: isDark ? "#e5e7eb" : "inherit",
},
itemsGrid: {
display: "grid", gridTemplateColumns: "repeat(auto-fit, minmax(110px, 1fr))",
gap: "4px", flex: 1,
},
choice: {
padding: "3px 8px",
border: `1px solid ${isDark ? "#9ca3af" : "#d1d5db"}`,
borderRadius: "3px", cursor: "pointer", minHeight: "30px",
display: "inline-flex", flexDirection: "column", alignItems: "center",
justifyContent: "center", fontWeight: 500, fontSize: "12px",
transition: "all 0.2s", userSelect: "none", textAlign: "center",
background: isDark ? "#374151" : "#fff",
color: isDark ? "#e5e7eb" : "inherit",
},
selected: { background: "#D45D44", color: "white", borderColor: "#D45D44" },
disabled: { cursor: "not-allowed", opacity: 0.4 },
subtitle: { fontSize: "9px", marginTop: "1px", lineHeight: 1.1, opacity: 0.75 },
topology: {
flex: 1, display: "flex", flexWrap: "wrap", alignItems: "center",
justifyContent: "space-between", gap: "4px 12px", fontSize: "12px",
color: isDark ? "#e5e7eb" : "#374151",
},
topologyMeta: { fontSize: "10px", color: isDark ? "#9ca3af" : "#6b7280" },
commandWrap: {
position: "relative", flex: 1, overflow: "hidden", borderRadius: "6px",
border: `1px solid ${isDark ? "#374151" : "#e5e7eb"}`,
background: isDark ? "#111827" : "#f5f5f5",
},
commandHeader: {
display: "flex", flexWrap: "wrap", justifyContent: "space-between",
alignItems: "center", gap: "6px 10px", padding: "6px 10px",
borderBottom: `1px solid ${isDark ? "#374151" : "#e5e7eb"}`,
background: isDark ? "#1f2937" : "#fafafa",
},
headerLeft: { display: "inline-flex", flexWrap: "wrap", alignItems: "center", gap: "8px" },
badge: (isPending) => ({
display: "inline-flex", alignItems: "center", gap: "6px",
padding: "2px 8px", borderRadius: "10px", fontSize: "11px", fontWeight: 600,
background: isPending
? (isDark ? "#78350f" : "#fef3c7")
: (isDark ? "#064e3b" : "#d1fae5"),
color: isPending
? (isDark ? "#fde68a" : "#92400e")
: (isDark ? "#a7f3d0" : "#065f46"),
}),
badgeDot: (isPending) => ({
width: "8px", height: "8px", borderRadius: "50%",
background: isPending ? "#f59e0b" : "#10b981",
}),
runModeWrap: {
display: "inline-flex", overflow: "hidden", borderRadius: "10px",
border: `1px solid ${isDark ? "#4b5563" : "#d1d5db"}`,
fontSize: "11px", fontWeight: 600, userSelect: "none",
},
runModeChip: (active, last) => ({
padding: "2px 10px", cursor: "pointer",
borderRight: last ? "none" : `1px solid ${isDark ? "#4b5563" : "#d1d5db"}`,
background: active ? (isDark ? "#374151" : "#fff") : "transparent",
color: active
? (isDark ? "#e5e7eb" : "#111827")
: (isDark ? "#9ca3af" : "#6b7280"),
}),
iconRow: { display: "inline-flex", flexWrap: "wrap", gap: "6px" },
iconButton: {
padding: "4px 10px", borderRadius: "4px", cursor: "pointer",
border: `1px solid ${isDark ? "#4b5563" : "#d1d5db"}`,
background: isDark ? "#1f2937" : "#fff",
color: isDark ? "#e5e7eb" : "#374151",
fontSize: "11px", fontWeight: 500, lineHeight: 1.4,
display: "inline-flex", alignItems: "center", gap: "4px", textDecoration: "none",
},
commandPre: {
padding: "12px 16px", margin: 0, overflowX: "auto",
fontFamily: "'Menlo', 'Monaco', 'Courier New', monospace",
fontSize: "12px", lineHeight: 1.5, whiteSpace: "pre-wrap",
color: isDark ? "#e5e7eb" : "#374151",
},
});

const first = config.recipes[0];
const [selection, setSelection] = useState({
workload: first.workload,
strategy: first.strategy,
});
const [runMode, setRunMode] = useState("dry-run");
const [copied, setCopied] = useState(false);
const [isDark, setIsDark] = useState(false);

useEffect(() => {
const check = () => {
const html = document.documentElement;
setIsDark(
html.classList.contains("dark") ||
html.getAttribute("data-theme") === "dark" ||
html.style.colorScheme === "dark"
);
};
check();
const observer = new MutationObserver(check);
observer.observe(document.documentElement, {
attributes: true,
attributeFilter: ["class", "data-theme", "style"],
});
return () => observer.disconnect();
}, []);

const recipe = config.recipes.find(
(item) => item.workload === selection.workload && item.strategy === selection.strategy
) || first;
const styles = makeStyles(isDark);
const recipesPath = (config.recipesPath || "../srt-slurm-recipes").replace(/\/$/, "");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The component uses React hooks (useState and useEffect) but does not import them, which will cause a runtime ReferenceError. Additionally, the hooks are defined after a conditional return statement (if (!config || ...)), which violates the Rules of Hooks (hooks must be called unconditionally at the top level of the component).

This suggestion imports the hooks and restructures the component so that state initialization and hooks are executed unconditionally at the top level, using safe fallbacks before the configuration validation check.

import { useState, useEffect } from "react";

// Interactive NVIDIA/srt-slurm recipe picker for cookbook deployment sections.
// Keep model-specific data in a plain config module and pass it through MDX.
//
// Config fields:
//   model/hardware          fixed display chips
//   workloads/strategies   selectable dimensions
//   recipes                {workload, strategy, path, prefillNodes,
//                           decodeNodes, servingNodes, infrastructureNodes,
//                           description, repositoryUrl, branch, status,
//                           benchmark}[]
//   repositoryUrl/branch   source link for the selected YAML
//   recipesPath            local path used in generated srtctl commands

export const SrtSlurmDeployment = ({ config }) => {
  const recipes = Array.isArray(config?.recipes) ? config.recipes : [];
  const firstRecipe = recipes[0] || {};

  const [selection, setSelection] = useState({
    workload: firstRecipe.workload || "",
    strategy: firstRecipe.strategy || "",
  });
  const [runMode, setRunMode] = useState("dry-run");
  const [copied, setCopied] = useState(false);
  const [isDark, setIsDark] = useState(false);

  useEffect(() => {
    const check = () => {
      const html = document.documentElement;
      setIsDark(
        html.classList.contains("dark") ||
        html.getAttribute("data-theme") === "dark" ||
        html.style.colorScheme === "dark"
      );
    };
    check();
    const observer = new MutationObserver(check);
    observer.observe(document.documentElement, {
      attributes: true,
      attributeFilter: ["class", "data-theme", "style"],
    });
    return () => observer.disconnect();
  }, []);

  if (
    !config ||
    !config.model ||
    !config.hardware ||
    !Array.isArray(config.workloads) ||
    !Array.isArray(config.strategies) ||
    recipes.length === 0
  ) {
    return (
      <div style={{padding: 12, color: "#b91c1c"}}>
        NVIDIA/srt-slurm deployment: missing recipe configuration
      </div>
    );
  }

  const makeStyles = (isDark) => ({
    container: {
      maxWidth: "900px", margin: "0 auto", display: "flex",
      flexDirection: "column", gap: "3px",
    },
    card: {
      padding: "5px 10px",
      border: `1px solid ${isDark ? "#374151" : "#e5e7eb"}`,
      borderLeft: `3px solid ${isDark ? "#E85D4D" : "#D45D44"}`,
      borderRadius: "4px", display: "flex", alignItems: "center", gap: "10px",
      background: isDark ? "#1f2937" : "#fff",
    },
    title: {
      fontSize: "12px", fontWeight: 600, minWidth: "108px", flexShrink: 0,
      color: isDark ? "#e5e7eb" : "inherit",
    },
    itemsGrid: {
      display: "grid", gridTemplateColumns: "repeat(auto-fit, minmax(110px, 1fr))",
      gap: "4px", flex: 1,
    },
    choice: {
      padding: "3px 8px",
      border: `1px solid ${isDark ? "#9ca3af" : "#d1d5db"}`,
      borderRadius: "3px", cursor: "pointer", minHeight: "30px",
      display: "inline-flex", flexDirection: "column", alignItems: "center",
      justifyContent: "center", fontWeight: 500, fontSize: "12px",
      transition: "all 0.2s", userSelect: "none", textAlign: "center",
      background: isDark ? "#374151" : "#fff",
      color: isDark ? "#e5e7eb" : "inherit",
    },
    selected: { background: "#D45D44", color: "white", borderColor: "#D45D44" },
    disabled: { cursor: "not-allowed", opacity: 0.4 },
    subtitle: { fontSize: "9px", marginTop: "1px", lineHeight: 1.1, opacity: 0.75 },
    topology: {
      flex: 1, display: "flex", flexWrap: "wrap", alignItems: "center",
      justifyContent: "space-between", gap: "4px 12px", fontSize: "12px",
      color: isDark ? "#e5e7eb" : "#374151",
    },
    topologyMeta: { fontSize: "10px", color: isDark ? "#9ca3af" : "#6b7280" },
    commandWrap: {
      position: "relative", flex: 1, overflow: "hidden", borderRadius: "6px",
      border: `1px solid ${isDark ? "#374151" : "#e5e7eb"}`,
      background: isDark ? "#111827" : "#f5f5f5",
    },
    commandHeader: {
      display: "flex", flexWrap: "wrap", justifyContent: "space-between",
      alignItems: "center", gap: "6px 10px", padding: "6px 10px",
      borderBottom: `1px solid ${isDark ? "#374151" : "#e5e7eb"}`,
      background: isDark ? "#1f2937" : "#fafafa",
    },
    headerLeft: { display: "inline-flex", flexWrap: "wrap", alignItems: "center", gap: "8px" },
    badge: (isPending) => ({
      display: "inline-flex", alignItems: "center", gap: "6px",
      padding: "2px 8px", borderRadius: "10px", fontSize: "11px", fontWeight: 600,
      background: isPending
        ? (isDark ? "#78350f" : "#fef3c7")
        : (isDark ? "#064e3b" : "#d1fae5"),
      color: isPending
        ? (isDark ? "#fde68a" : "#92400e")
        : (isDark ? "#a7f3d0" : "#065f46"),
    }),
    badgeDot: (isPending) => ({
      width: "8px", height: "8px", borderRadius: "50%",
      background: isPending ? "#f59e0b" : "#10b981",
    }),
    runModeWrap: {
      display: "inline-flex", overflow: "hidden", borderRadius: "10px",
      border: `1px solid ${isDark ? "#4b5563" : "#d1d5db"}`,
      fontSize: "11px", fontWeight: 600, userSelect: "none",
    },
    runModeChip: (active, last) => ({
      padding: "2px 10px", cursor: "pointer",
      borderRight: last ? "none" : `1px solid ${isDark ? "#4b5563" : "#d1d5db"}`,
      background: active ? (isDark ? "#374151" : "#fff") : "transparent",
      color: active
        ? (isDark ? "#e5e7eb" : "#111827")
        : (isDark ? "#9ca3af" : "#6b7280"),
    }),
    iconRow: { display: "inline-flex", flexWrap: "wrap", gap: "6px" },
    iconButton: {
      padding: "4px 10px", borderRadius: "4px", cursor: "pointer",
      border: `1px solid ${isDark ? "#4b5563" : "#d1d5db"}`,
      background: isDark ? "#1f2937" : "#fff",
      color: isDark ? "#e5e7eb" : "#374151",
      fontSize: "11px", fontWeight: 500, lineHeight: 1.4,
      display: "inline-flex", alignItems: "center", gap: "4px", textDecoration: "none",
    },
    commandPre: {
      padding: "12px 16px", margin: 0, overflowX: "auto",
      fontFamily: "'Menlo', 'Monaco', 'Courier New', monospace",
      fontSize: "12px", lineHeight: 1.5, whiteSpace: "pre-wrap",
      color: isDark ? "#e5e7eb" : "#374151",
    },
  });

  const recipe = recipes.find(
    (item) => item.workload === selection.workload && item.strategy === selection.strategy
  ) || firstRecipe;
  const styles = makeStyles(isDark);
  const recipesPath = (config.recipesPath || "../srt-slurm-recipes").replace(/\/$/, "");

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant