Network Kernel debugging via luv netconsole
Linux* UEFI Validation (LUV) has grown since we last discussed it. In this article, we will talk about one of the coolest features of LUV, debugging.
Debugging is especially important if you are trying to resolve an issue, if you can’t move further with your testing, or if your system doesn’t support any sort of testing. Debugging can be done via several methods.
The first debugging method is via the serial console. The serial console is a connection over the RS-232 or serial port connection that allows an user to access a computer/system console. The serial console is accessed using applications such as PuTTY*, PICOCOM*, and others. To use the serial console, you must have a second computer to access the first computer. If you face an issue such as a kernel panic that is displayed on the screen, then a serial console can be used to record that message. This is one of the simplest forms of debugging and is very reliable, especially when you don’t have access to the hardware debugger. However, the downside of the serial console is that it will miss very early boot problems when the serial ports are not yet enabled.
If the computer you want to debug is located elsewhere and you want to use the serial console to record the messages reliably, you can use the second debugging method, Telnet. Telnet is an application-layer protocol that works over TCP/IP to access a remote computer. You must install Telnet applications on the local computer and on the remote computer.
The third debugging method is via network console, called Netconsole. Netconsole is a Linux feature and a kernel module that sends all kernel log messages to a remote computer over the network. It does not have overwhelming user space messages and is convenient when a serial console is unavailable. Netconsole is an alternative to a serial console and is convenient when the computer does not have a serial port, as is the case with many computers in the field, such as laptops.
Of all the available methods, we choose to use Netconsole for debugging because it can be used when no serial console is available and, importantly, it is a powerful kernel debugging tool. Netconsole sends the kernel log messages (dmesg) to a different computer over UDP packets and helps in kernel debugging when the system hangs.
LUV is used to test systems for any firmware issues. There is a possibility that the system may hang during the execution of LUV. It is very important that LUV gets the debug information to narrow down the causes and provide support to its users.
To use the Netconsole feature in LUV, you must understand how it is used.
The current release of luv-netconsole supports Linux and Microsoft* Windows* hosts and remote computers. However, it is currently only possible to make changes in the luv.cfg file using a Linux system. This file is available in the second partition of the image mounted or flashed in a USB stick.
Choose the IP address and port number to which you want all messages sent. Once decided, you can replace the dummy IP address and port number given in luv.cfg (available in boot partition) as LUV_NETCONSOLE=10.11.12.13,64001 (the IP address and port number respectively).
The luv.cfg file is located in boot partition. Edit the file to meet the needs of netconsole.
To get the ip address and port number on a Linux computer, open a terminal and run the following command:
$ ifconfig
The interfaces, shown as eth0, eth1, lo, and wlan0, are described below:
- Ethernet interfaces are eth0, eth1, etc.
- The loopback interface, which the system uses to communicate with itself, is lo.
- The wireless interfaces are wlan0, wlan1, etc.
There can be more interfaces, other than the ones mentioned here, which are usually named by the user while setting up.
Look for the IP address 'inet addr' in the available interfaces other than lo. To use a wireless network, look in the 'wlan' interfaces. To use an ethernet network, look in the 'eth' interfaces.
Choose port numbers which are not reserved for any kernel activities, such as 4000's and above. Port 64001 is chosen in luv-netconsole.
To get the ip address and port number on a Windows computer, open a command prompt and enter the following command:
ipconfig
Look for the IPV4 Address.
On the remote computer, use netcat or nc to listen to all the messages that are being sent:
$ netcat -l -u <port number> (or) nc -l -u <port number>
Here is an example:
$ netcat -l -u 64001
You can also use socat for this same purpose:
$ socat - udp-recv:<port>
In our example:
$ socat - udp-recv:64001
In this way, all the messages are available on a machine to which you are listening. Happy debugging!