This page describes different methods of debugging ReactOS and the steps necessary to debug ReactOS.
- 1 Introduction
- 2 Available debugging methods
- 2.1 Debugging through text messages
- 2.1.1 Serial Port
- 188.8.131.52 QEMU Virtual Machine: Redirect to a file
- 184.108.40.206 QEMU Virtual Machine: Redirect to the console
- 220.127.116.11 VMware Virtual Machine: Redirect to a file
- 18.104.22.168 VMware Virtual Machine: Redirect using a named pipe
- 22.214.171.124 VirtualBox Virtual Machine: Redirect using a named pipe on a Win32 Host
- 126.96.36.199 VirtualBox Virtual Machine: Redirect to a Console on a Linux Host
- 188.8.131.52 VirtualBox Virtual Machine: Redirect to a File on a Linux Host
- 184.108.40.206 Real computer: Physical serial cable
- 2.1.2 Debug text output to file
- 2.1.3 Debug text output to screen
- 2.1.1 Serial Port
- 2.2 KDBG
- 2.3 GDB
- 2.1 Debugging through text messages
- 3 Generating even more output
To be able to help ReactOS development, whether this be participating in the development of the source code or taking part in crucial testing, you are going to need knowledge of how to generate useful debug logs.
Useful debug logs are essential pieces of information which the developer needs to quickly pinpoint and identify exactly what the operating system is doing. Many people know how to get default debug output from the operating system, but this is generally not particularly useful for locating problems, especially bugs.
This article aims to give users knowledge not only on how to generate a debug log, but on how to generate a useful debug log which can be used directly to assess what the operating system is doing.
Available debugging methods
There are various methods to debug ReactOS, some require more knowledge than others. These are listed below.
Debugging through text messages
This is the easiest method for receiving debug information from ReactOS.
The serial port is the most common method used for receiving debug messages from ReactOS. The method used for receiving data from the serial port depends on whether you run ReactOS in a virtual machine or on a real computer.
QEMU Virtual Machine: Redirect to a file
If you're using the QEMU virtual machine and want to redirect the serial output to a file, add the following to your QEMU command line:
Replace FILENAME with the name of a file, into which the debug output shall be written.
QEMU Virtual Machine: Redirect to the console
The method above redirects the output to a file. If you want to redirect it to the console, there are two different methods. They depend on the operating system you run QEMU under.
- Under Windows, add the following to your QEMU command line:
- Under a Unix-based operating system, add the following to your QEMU command line:
VMware Virtual Machine: Redirect to a file
If you're using a VMware product and want to redirect the serial output to a file, click the Edit virtual machine settings link.
If your virtual machine does not yet have a virtual serial port, add one using the Add button. Then select it and in the Connection box, choose the option Use output file and enter the name of the file, into which the debug output shall be written.
VMware Virtual Machine: Redirect using a named pipe
You can use the same methode as with VirtualBox, below.
VirtualBox Virtual Machine: Redirect using a named pipe on a Win32 Host
You need to download the VmwareGateway application. Start it with /r option to make it run as a service. Next you need to start the service, using SC command:
sc start vmwaregateway
Sort out any firewall popup if applicable. Finally, use your favourite telnet client to connect to localhost on port no. 567.
To configure your VM, set it to Host Pipe with the following pipe adress:
Make sure you do not mark the "create pipe" box. vmwaregateway has already created it. Debug output should appear in your telnet client.
VirtualBox Virtual Machine: Redirect to a Console on a Linux Host
You have to use some application like the Win32 Hyperterminal, such as Minicom. Firstly install it
sudo apt-get install minicom
Then you will have to configure a pipe. Type Ctrl-A, then O. Select Configure serial ports, and in the device type something like
pipe_path is a file which will be used as pipe between the VM and minicom. For example, in my pc i use /tmp/vboxlog
Please note you must type pipe_path exactly on both minicom and VirtualBox
In VirtualBox, create a new Virtual Machine, then go to the Serial Ports configuration. Select Host pipe, type the pipe_path and select Create Pipe.
VirtualBox Virtual Machine: Redirect to a File on a Linux Host
When running minicom, type Ctrl-A then L. Type the name of the capture file.
Real computer: Physical serial cable
You will need a physical serial cable if you want to receive debug messages from a real computer through the serial port. This method also requires two computers (one on which you test ReactOS and another one for receiving the debug messages).
The cable needed for this debugging method is a Null-Modem serial cable. You should find it in many computer shops for less than 10 dollars. If you don't have one ready, you can also build one like this:
DTE1_______________________________________________DTE 2 9pol 25pol (female)_________________________________25pol 9pol (female) 5 7 ---GND---------------------GND------- 7 5 2 3 ---RxD--------. ,----------RxD------- 3 2 X 3 2 ---TxD--------' `----------TxD------- 2 3 7 4 ---RTS--------. ,----------RTS------- 4 7 X 8 5 ---CTS--------' `----------CTS------- 5 8 4 20 ---DTR--------. ,----------DTR------- 20 4 X 6 6 ---DSR--o-----' `-------o--DSR------- 6 6 | | 1 8 ---DCD--' `--DCD------- 8 1
Connect the cable to the first serial port of both computers.
Then use a Terminal application like PuTTY or Windows HyperTerminal on the computer for receiving the debug messages. Set it up to listen to the first serial port (COM1 [3F8/IRQ4]) and a baud rate of 115200.
After that, boot ReactOS on the test computer and you should receive debug messages. If this doesn't work, check your hardware and your freeldr.ini configuration.
Debug text output to file
Edit freeldr.ini to contain an entry like the following:
[ReactOS_Debug] BootType=ReactOS SystemPath=multi(0)disk(0)rdisk(0)partition(1)\ReactOS Options=/DEBUG /DEBUGPORT=FILE /SOS
Debug text output to screen
Edit freeldr.ini to contain an entry like the following:
[ReactOS_Debug] BootType=ReactOS SystemPath=multi(0)disk(0)rdisk(0)partition(1)\ReactOS Options=/DEBUG /DEBUGPORT=SCREEN /SOS
The ReactOS build environment now includes gdb by default. In order to use it you must create config.rbuild from config.template.rbuild and change it so that:
<property name="KDBG" value="0" /> <property name="DBG" value="1" /> <property name="GDB" value="1" />
After you recompile(specially, kdcom and ntoskrnl), start Qemu as you normally would, but add the following command line parameters:
This is done so that Qemu starts in the STOPPED state, and allow you to connect using gdb. Now its time to get GDB of the ground
- (Assuming you are in the rosbe command line), enter “gdb” to start gdb.
- Enter “file ./output-i386/ntoskrnl/ntoskrnl.exe” to tell gdb where to load information about the kernel.
- Enter “set disassembly-flavor intel” for good measure.
- Enter “target remote localhost:1234” to connect gdb to what we are debugging.
- Enter “c” (for “continue“) to have gdb instruct Qemu to start/continue execution of the emulation.
- To manually pause execution, make sure your gdb window has focus and simply enter <CTRL>+<C>
Generating even more output
In order to get meaningful debug output it is sometimes necessary to enable extra verbosity.
Turning on verbosity at compile time
Nearly all ReactOS modules use the built in "ReactOS style" debugging functionality. This style is characterized by:
- Verbosity level is usually defined per file.
- Only 2 message levels:
- always enabled (DPRINT1)
- only enabled when NDEBUG is not defined (DPRINT)
Files that follow this style can easily be spotted by this code:
#define NDEBUG #include <debug.h>
To enable full verbosity just comment out the "#define NDEBUG", and remember to uncomment it when submitting patches.
sample line to enable debug channels in usermode applications, in file:
; Debug channels HKLM,"SYSTEM\CurrentControlSet\Control\Session Manager\Environment","DEBUGCHANNEL",0x00020000,"+ole,+rpc"
Turning on verbosity at runtime
The easiest way to turn on debug verbosity on any particual component is to use DEBUGCHANNEL enviroment variable. For example, to get all debug messages from MSI, simply run in CMD:
Then, run the app you wish to test. You can add debug messages from multiple components, for example:
and so on. After closing down CMD, debug verbosity will change to default.
<Describe details on how to set verbosity level, turn on debug from all components.>
Breaking into the debugger
Bugchecks occur when the operating system can no longer operate safely and to avoid corrupting data, it halts operation. This will normally throw up a Blue Screen Of Death, but if you have the kernel debugger activated, it will drop you into the prompt giving you access to explore the system state. By default ReactOS debug builds have the integrated kernel debugger(kdbg) enabled. Release builds do NOT have this feature enabled and will only display a blue screen.
There are two ways for forcing a bugcheck, each one employing a different method:
If you have a debug build and want to halt the system for any given reason and break into the the kernel debugger, you can force a bugcheck from the keyboard by simply typing:
Remember that kdbg output goes out through the serial port, but it receives input from the console by default.
To receive input through the serial port as well, add the command /KDSERIAL to the freeldr.ini boot options. It could look like this:
Options=/DEBUG /DEBUGPORT=COM1 /BAUDRATE=115200 /SOS /KDSERIAL
Breaking on user mode Exceptions
For each type of exception known by KDB you can set the condition when KDB should be entered individually for first and last chance. The possible settings for the conditions are never, umode, kmode and always. never means that KDB will never be entered when the exception is raised, umode means that it will be entered when the exception was raised in usermode, kmode means to enter KDB when the exception was raised in kmode, and always means to always enter KDB.
To change the condition to enter KDB on INT3 to "always" (default is "kmode"), enter the debugger and type:
set condition * first always
Type "cont" to continue normal execution.
This is useful when you want to halt the operating system when it hits a particular area of code you might be debugging. This is especially useful as you can get an immediate back trace to see where the code flow came before the bugcheck was forced.
This is done by using KeBugCheck() or ASSERT() in the code.
Generating a backtrace
In order to generate a backtrace you must break into the KDBG prompt.
Enter 'bt' and hit <return>, you should see something similar to the following:
(drivers\filesystems\vfat\rw.c:809) <\ReactOS\system32\kernel32.dll> Entered debugger on embedded INT3 at 0x0008:0x800935f2. kdb:> bt Eip: <ntoskrnl.exe:935f3 (lib\rtl\i386\debug_asm.S:31 (DbgBreakPoint@0))> Frames: <vfatfs.sys:97de (drivers/filesystems/vfat/misc.c:111 (VfatDispatchRequest))> <vfatfs.sys:9b25 (drivers/filesystems/vfat/misc.c:167 (VfatBuildRequest))> <ntoskrnl.exe:3ab23 (ntoskrnl/io/iomgr/irp.c:1088 (IofCallDriver))> <ntoskrnl.exe:36206 (ntoskrnl/io/iomgr/iofunc.c:686 (IoSynchronousPageWrite))> <ntoskrnl.exe:59daa (ntoskrnl/mm/section.c:6330 (MmspWriteDataSectionPages))> <ntoskrnl.exe:244c6 (ntoskrnl/ex/work.c:162 (ExpWorkerThreadEntryPoint))> <ntoskrnl.exe:70e90 (ntoskrnl/ps/thread.c:134 (PspSystemThreadStartup))> <ntoskrnl.exe:7b142 (ntoskrnl\ke\i386\ctxswitch.S:258 (KiThreadStartup@156))> kdb:>
The bt command will show a backtrace of the currently attached thread, so it may be necessary to use the 'thread attach' command, please refer to the kdbg manual for more details.
Examining this in more detail, we can see that the bugcheck occured via the INT3 operation which then dropped us into kdb. The next line shows us Eip which is the instruction pointer, and this points to the last address before the system halted.
Following on from that are the frames. This is the important part of generating our backtrace and it contains all the function addresses in the buildup to the bugcheck. This is the crutial information developers need to understand the codeflow before the bugcheck.
Occasionally, there will come a time when you will need to manually translate addresses. When kdbg is not enabled and a bugcheck occurs, you will be presented with a stack trace similar to the following:
(subsystems\win32\csrss\win32csr\conio.c:1101) Console_Api Ctrl-C
*** Fatal System Error: 0x00000001 (0x80079279,0x00000000,0x0000FFFF,0x00000000)
<\SystemRoot\System32\NTOSKRNL.EXE: 29bb> <\SystemRoot\System32\HAL.DLL: 4749> <\SystemRoot\System32\NTOSKRNL.EXE: 54cb4> <\SystemRoot\System32\NTOSKRNL.EXE: 582bf> <\SystemRoot\System32\NTOSKRNL.EXE: 583fd> <\SystemRoot\System32\NTOSKRNL.EXE: 89956> <\SystemRoot\system32\drivers\videoprt.sys: 2417> <\SystemRoot\system32\drivers\vbemp.sys: 17f5> <\SystemRoot\system32\drivers\vbemp.sys: 19cf> <\SystemRoot\system32\drivers\videoprt.sys: 1c48> <\SystemRoot\System32\NTOSKRNL.EXE: 34c17> <\SystemRoot\System32\NTOSKRNL.EXE: 21e0> <\SystemRoot\System32\NTOSKRNL.EXE: 2908> <\SystemRoot\System32\NTOSKRNL.EXE: 29bb> <\SystemRoot\System32\NTOSKRNL.EXE: 85fa8>
As you can see, this is largely the same as the information presented when issuing a 'bt' command in kdbg. The problem here however, is that only the addresses are given. As these addresses are different for everyones builds, this information is useless for anyone trying to follow what events occured in the lead up towards to bugcheck.
The solution here is to translate the addresses into human readable function names. This is done via a tool named 'raddr2line' which is a modified version of the unix tool 'addr2line'. This tool will translate the addresses given to it into file names and line numbers. It does this by using debug information in the executable files to accociate the address with this human friendly info and outputs it into the console. This information can be pasted into the above debug log alongside the addresses providing the developers with a detailed stack trace.
radr2line is included in the Reactos Build Environment. It is invoked in the following way :
raddr2line <file> <address>
So taking the bottom address in the above stack trace :
C:\Users\Ged\MyFiles\ReactOS\clean_source>raddr2line ntoskrnl.exe 85fa8 C:\Users\Ged\MyFiles\ReactOS\clean_source\output-i386\ntoskrnl\ntoskrnl.exe obj-i386\ntoskrnl\ex\zw.S:253 (ZwClearEvent)
we can see here that the address translation for 0x85fa8 is line 253 in file ntoskrnl\ex\zw.S (this will differ if you try it on your build)
This information can now be added into the above stack trace as follows :
<\SystemRoot\System32\NTOSKRNL.EXE: 29bb> <enter next one here> <\SystemRoot\System32\NTOSKRNL.EXE: 85fa8> obj-i386\ntoskrnl\ex\zw.S:253 (ZwClearEvent)