[Playground] Verified cell: mi355x / pro / fp4 / high-throughput / single

[seungrokj]

Cookbook model

deepseek-ai/deepseek-v4

Combination

mi355x / pro / fp4 / high-throughput / single

Proposed cell snippet

{
      match: { hw: "mi355x", variant: "pro", quant: "fp4", strategy: "high-throughput", nodes: "single" },
      verified: true,
      env: [
        "SGLANG_USE_ROCM700A=0",
        "SGLANG_DP_USE_GATHERV=1",
        "SGLANG_HACK_FLASHMLA_BACKEND=unified_kv_triton",
        "AITER_BF16_FP8_MOE_BOUND=0",
      ],
      flags: [
        "--trust-remote-code",
        "--model-path {{MODEL_NAME}}",
        "--tp 8",
        "--dp 8",
        "--enable-dp-attention",
        "--enable-prefill-delayer",
        "--prefill-delayer-max-delay-ms 5000",
        "--attention-backend dsv4",
        "--page-size 256",
        "--mem-fraction-static 0.90",
        "--swa-full-tokens-ratio 0.1",
        "--disable-shared-experts-fusion",
        "--kv-cache-dtype fp8_e4m3",
        "--chunked-prefill-size 65536",
        "--speculative-algorithm EAGLE",
        "--speculative-num-steps 3",
        "--speculative-eagle-topk 1",
        "--speculative-num-draft-tokens 4",
        "--enable-hierarchical-cache",
        "--hicache-ratio 4",
        "--hicache-write-policy write_through",
        "--hicache-io-backend direct",
        "--hicache-mem-layout page_first_direct",
        "--host {{HOST_IP}}",
        "--port {{PORT}}",
      ],
    },

Existing cell at this match (for diff)

{
      match: { hw: "mi355x", variant: "pro", quant: "fp4", strategy: "high-throughput", nodes: "single" },
      verified: true,
      env: [
        "SGLANG_USE_ROCM700A=0",
        "SGLANG_DP_USE_GATHERV=1",
        "SGLANG_HACK_FLASHMLA_BACKEND=unified_kv_triton",
        "AITER_BF16_FP8_MOE_BOUND=0",
      ],
      flags: [
        "--trust-remote-code",
        "--model-path {{MODEL_NAME}}",
        "--tp 8",
        "--dp 8",
        "--enable-dp-attention",
        "--enable-prefill-delayer",
        "--prefill-delayer-max-delay-ms 5000",
        "--attention-backend dsv4",
        "--page-size 256",
        "--mem-fraction-static 0.90",
        "--swa-full-tokens-ratio 0.1",
        "--disable-shared-experts-fusion",
        "--kv-cache-dtype fp8_e4m3",
        "--chunked-prefill-size 65536",
        "--speculative-algorithm EAGLE",
        "--speculative-num-steps 3",
        "--speculative-eagle-topk 1",
        "--speculative-num-draft-tokens 4",
        "--host {{HOST_IP}}",
        "--port {{PORT}}",
      ],
    },

SGLang version

sglang-rocm:v0.5.14-rocm720-mi35x-20260710

Benchmark result (optional but encouraged)

No response

Notes / caveats

--enable-hierarchical-cache
--hicache-ratio 4
--hicache-write-policy write_through
--hicache-io-backend direct
--hicache-mem-layout page_first_direct

Attestation

• I ran this exact command on the listed hardware.
• The server reached READY and answered a cURL request successfully.
• Output looked correct on at least one prompt.

[seungrokj]

@HaiShaw this is hicache setting for mi355x,
This setting is verified by SA agentX perf benchmark in this PR SemiAnalysisAI/InferenceX#2146

this default setting is breaking
--hicache-io-backend kernel (default) &--hicache-mem-layout page_first
so have to use:
--hicache-io-backend direct & --hicache-mem-layout page_first_direct