Netboot/PXE LAN Service for osdev

[dckc]

Fiddling with USB thumb drives is a pain. Can we just boot from the network?

I'd like a persistent, low-administration Preboot Execution Environment (PXE) service on the local area network (LAN) to serve two types of boot images:

1. Generic Installers: Easy access to standard OS/utility installers (via netboot.xyz).
2. Custom Images: A platform for rapid, iterative testing and deployment of custom-built kernels and bootloaders (OSDev projects) on physical target hardware (e.g., ThinkPad T430).

My original work is in another repo:

2024-08-30 22:25 8c65f7e feat: DHCP server running with next-server
2024-02-02 20:58 4f58237 c4014a0

netboot-changes.zip

[dckc]

Replace Router? Isolated Server?

1. Networking Infrastructure (The PXE Prerequisite)

The primary obstacle is ensuring the client machine (T430) receives a DHCP offer correctly pointing it to the boot server.

Strategy	Status	Rationale for Status
Strategy A (Original Attempt): Full DHCP on Workstation	Failed (Conflict - 8c65f7e)	Running a full DHCP server (isc-dhcp-server) on the primary subnet immediately conflicted with the Google Wi-Fi router's DHCP service.
Strategy B: Native Router Configuration	Infeasible (Dead End)	Google Wi-Fi does not expose the custom DHCP options (Option 66/67) required for PXE booting.
Strategy C: Isolated DHCP Server	CONFIGURED (Current Path)	Deploy a dedicated server (Mini-PC/Pi) to run DHCP/TFTP on an isolated segment, providing the required PXE options without disrupting the main network. Detailed implementation is captured in the Isolated DHCP Guide.
Strategy D: Replace Main Router	Viable (High Effort)	Replaces the entire network backbone with a prosumer solution that provides native, full DHCP control (Option 66/67). This is considered the "cleanest" solution but requires the highest initial effort, cost, and full network disruption.

misc notes

2. Boot Service Content (The Content)

Subgoal	Requirement	Component	Status	OSDev Relevance
Generic Menu	Serve a variety of popular OS installers and utilities.	netboot.xyz Docker Container	✅ Deployed	The Docker Compose configuration is complete and the service is ready to run.
OSDev Custom Image	Serve local, custom-compiled kernels/bootloaders for testing.	TFTP/HTTP root directory mapping.	⏳ In Progress	This is the crucial step for iterative OSDev testing; the current infrastructure needs final volume mapping.

3. Critical Remaining Steps

1. OSDev Integration: Finalize the docker-compose.yml volumes to map a local directory (e.g., ./osdev-builds) to the server's TFTP/HTTP root path. This will make custom-compiled kernels and bootloaders accessible for the T430.
2. Physical Testing: Conduct the first end-to-end test by booting the ThinkPad T430 on the isolated network segment and verifying it receives the PXE offer, loads the initial bootloader, and successfully displays the netboot.xyz menu.

[dckc]

Replace Google Wi-Fi devices with OpenWRT device?

This document details the requirements and trade-offs for Strategy D: Replace Main Router, which involves upgrading the entire network backbone to a Prosumer-grade device (like the GL.iNet GL-MT6000, or Flint 2) to natively support PXE/DHCP Option 66/67.

Key Comparison: Google Wi-Fi (Current) vs. GL.iNet GL-MT6000 (Proposed)

Replacing the router solves the underlying DHCP issue (Strategy B failure) while significantly upgrading network performance (as you noted, the Flint 2 is much faster).

Feature / Metric	Current Router (Google Wi-Fi / Nest Wi-Fi)	Proposed Router (GL.iNet GL-MT6000 / Flint 2)
DHCP Control (PXE)	Minimal/None. Critical PXE options (66/67 for TFTP Server IP and Boot File) are unavailable in the consumer-grade firmware.	Full Control (Native). Runs OpenWrt/Custom Firmware. Allows granular DHCP settings including Options 66/67, essential for native PXE booting.
Wired Interface Speed	1 Gigabit (1G). Typical consumer hardware is limited to 1G WAN/LAN ports.	2.5 Gigabit (2.5G). Features 1x 2.5G WAN and 4x 2.5G LAN ports.
Wired Performance	Max theoretical throughput of 1 Gbps. (Slower for local file transfers/OSDev image pushing).	Max theoretical throughput of 2.5 Gbps. (250% increase in wired speed, beneficial for large file transfers).
Processor / RAM	Typically lower-spec Quad-Core (e.g., 1.4 GHz) / 1GB RAM.	High-performance Quad-Core ARM CPU (2.0 GHz) / 1GB DDR4 RAM.
Operating System	Proprietary / Closed Source.	OpenWrt (Open Source). Allows installation of packages like TFTP servers, DNS/DHCP control, and VPN clients directly on the router.
Complexity / Effort	Low administrative effort, plug-and-play.	High initial setup effort and configuration complexity. Requires learning OpenWrt UI/concepts.
Primary Goal	Ease of setup, Mesh Networking.	Performance, Security, and Advanced Control.

💡 Rationale for Strategy D

The primary driver for considering the GL-MT6000 is its native support for OpenWrt, which grants full administrative control over the DHCP server.

1. Native PXE: The router can be configured directly to serve the necessary PXE boot instructions (TFTP server IP and boot filename), eliminating the need for complex firewall rules or an isolated DHCP server on a dedicated mini-PC (Strategy C).
2. Performance Boost: The 2.5G Ethernet ports significantly improve wired data transfer speeds, which is a massive quality-of-life upgrade for moving large OSDev build artifacts or accessing network storage.
3. Future-Proofing: Provides a robust platform for future OSDev/networking needs, such as setting up VLANs for isolation or running lightweight services directly on the router.

⚠️ Drawbacks and Effort

The main obstacle is the High Effort involved: replacing the main router requires configuring and migrating all existing network settings (Port Forwarding, Static IPs, Wi-Fi SSIDs, etc.) to the new device, resulting in full network disruption during the transition.

[dckc]

Isolated DHCP Server / segment on a Raspberry Pi class device?

🎯 Overview and Technical Implementation

The Isolated DHCP Server strategy is the chosen path to overcome the limitations of consumer mesh Wi-Fi systems (like Google Wi-Fi) which do not allow the configuration of custom DHCP options 66 (next-server) and 67 (filename). This guide serves as the definitive technical implementation plan for this strategy, consolidating all configuration and workflow details.

This approach involves setting up a dedicated server (like a Raspberry Pi or Mini-PC) to run the DHCP and TFTP/HTTP services on a separate, isolated network segment.

The "Isolation" Principle

By running the PXE service on its own subnet, we ensure:

1. No Conflict: The main home router continues to manage the primary network, preventing the conflict that occurred in Strategy A (Full DHCP on Workstation).
2. Full Control: The isolated DHCP server has full control over the necessary PXE options (66 and 67), ensuring the target hardware (e.g., ThinkPad T430) receives the correct boot instructions.

🛠️ Configuration Details (ISC DHCP Server)

The configuration below uses isc-dhcp-server to manage the isolated subnet 192.168.86.0/24. This setup is crucial as it directs the PXE client to the correct server IP and the appropriate bootloader file based on its architecture.

Critical Parameters

Parameter	Configuration Value	Purpose
subnet / range	192.168.86.0 / 192.168.86.10 - .200	Defines the address range for the isolated network.
next-server (Option 66)	192.168.86.62	The IP of the PXE/TFTP Host. All clients are told to look here for the bootloader files.
filename (Option 67)	Conditional based on client architecture (see below).	Specifies the initial file the client should download from the next-server.

dhcpd.conf Configuration Snippet

The following excerpt from dhcpd.conf implements the conditional boot loading, which is a best practice for modern PXE environments supporting both legacy and UEFI systems.

# tftpd stuff from netboot.xyz
option arch code 93 = unsigned integer 16;

subnet 192.168.86.0 netmask 255.255.255.0 {
range 192.168.86.10 192.168.86.200; # IP range for PXE clients
next-server 192.168.86.62; # The static IP of the PXE/TFTP host
option subnet-mask 255.255.255.0;
option routers 192.168.86.1; # The gateway for the isolated subnet
option broadcast-address 192.168.86.255;
option domain-name-servers 1.1.1.1;

# Conditional booting logic based on DHCP Option 93 (client architecture)
if exists user-class and ( option user-class = "iPXE" ) {
# Clients that report as iPXE get the netboot.xyz menu via HTTP
filename "[http://boot.netboot.xyz/menu.ipxe\](http://boot.netboot.xyz/menu.ipxe)";
} elsif option arch = encode-int ( 16, 16 ) {
# UEFI clients (arch 16, 64-bit) get the UEFI bootloader via HTTP (Best Practice)
filename "[http://boot.netboot.xyz/ipxe/netboot.xyz.efi\](http://boot.netboot.xyz/ipxe/netboot.xyz.efi)";
option vendor-class-identifier "HTTPClient";
} elsif option arch = 00:07 {
# Other UEFI/x64 clients get the standard UEFI bootloader
filename "netboot.xyz.efi";
} else {
# Legacy BIOS clients get the legacy bootloader
filename "netboot.xyz.kpxe";
}
}

🚀 OSDev and Docker Integration

Once the DHCP server is running on the isolated segment, the netboot.xyz Docker container will serve the actual boot files.

The next critical step is ensuring the local custom OSDev images are accessible:

• Action: Finalize the docker-compose.yml volumes to map your local OSDev build directory (e.g., ./osdev-builds) into the Docker container's TFTP/HTTP root path.
• Result: When the ThinkPad T430 requests a file specified by the iPXE menu (served by netboot.xyz), the file will be pulled directly from the local volume, enabling rapid, iterative testing of custom kernels and bootloaders.

[dckc]

netboot.xyz proxmox script

A dhcp in proxy mode provides the next server and boot file information, but does not provide ip addresses. The use case is that, on most consumer routers, it is not possible to configure next server and boot file information, and thus a proxy dhcp server fulfills only this role, while the main dhcp server only hands out ip addresses.

-- Jul 21, 2023 · edited by jerrywoo96