This page describes different methods of debugging ReactOS and the steps necessary to debug ReactOS.

Introduction

To be able to help ReactOS development, whether this be participating in the development of the source code or taking part in crucial testing, you are going to need knowledge of how to generate useful debug logs.

Useful debug logs are essential pieces of information which the developer needs to quickly pinpoint and identify exactly what the operating system is doing. Many people know how to get default debug output from the operating system, but this is generally not particularly useful for locating problems, especially bugs.

This article aims to give users knowledge not only on how to generate a debug log, but on how to generate a useful debug log which can be used directly to assess what the operating system is doing.

Available debugging methods

There are various methods to debug ReactOS, some require more knowledge than others. These are listed below.

Debugging through text messages

This is the easiest method for receiving debug information from ReactOS.

Serial Port

The serial port is the most common method used for receiving debug messages from ReactOS. The method used for receiving data from the serial port depends on whether you run ReactOS in a virtual machine or on a real computer.

QEMU Virtual Machine: Redirect to a file

If you're using the QEMU virtual machine and want to redirect the serial output to a file, add the following to your QEMU command line:

-serial file:FILENAME

Replace FILENAME with the name of a file, into which the debug output shall be written.

QEMU Virtual Machine: Redirect to the console

The method above redirects the output to a file. If you want to redirect it to the console, there are two different methods. They depend on the operating system you run QEMU under.

Under Windows, add the following to your QEMU command line:
```
-serial file:CON
```
Under a Unix-based operating system, add the following to your QEMU command line:
```
-serial stdio
```

VMware Virtual Machine: Redirect to a file

If you're using a VMware product and want to redirect the serial output to a file, click the Edit virtual machine settings link.

If your virtual machine does not yet have a virtual serial port, add one using the Add button. Then select it and in the Connection box, choose the option Use output file and enter the name of the file, into which the debug output shall be written.

VMware Virtual Machine: Redirect using a named pipe

You can use the same methode as with VirtualBox, below.

VirtualBox Virtual Machine: Redirect using a named pipe on a Win32 Host

You need to download the VmwareGateway application. Start it with /r option to make it run as a service. Next you need to start the service, using SC command:

sc start vmwaregateway

Sort out any firewall popup if applicable. Finally, use your favourite telnet client to connect to localhost on port no. 567.

To configure your VM, set it to Host Pipe with the following pipe adress:

\\.\pipe\vmwaredebug

Make sure you do not mark the "create pipe" box. vmwaregateway has already created it. Debug output should appear in your telnet client.

VirtualBox Virtual Machine: Redirect to a Console on a Linux Host

You have to use some application like the Win32 Hyperterminal, such as Minicom. Firstly install it

sudo apt-get install minicom

Then you will have to configure a pipe. Type Ctrl-A, then O. Select Configure serial ports, and in the device type something like

unix#pipe_path

.

pipe_path is a file which will be used as pipe between the VM and minicom. For example, in my pc i use /tmp/vboxlog

Please note you must type pipe_path exactly on both minicom and VirtualBox

In VirtualBox, create a new Virtual Machine, then go to the Serial Ports configuration. Select Host pipe, type the pipe_path and select Create Pipe.

VirtualBox Virtual Machine: Redirect to a File on a Linux Host

When running minicom, type Ctrl-A then L. Type the name of the capture file.

Real computer: Physical serial cable

You will need a physical serial cable if you want to receive debug messages from a real computer through the serial port. This method also requires two computers (one on which you test ReactOS and another one for receiving the debug messages).

The cable needed for this debugging method is a Null-Modem serial cable. You should find it in many computer shops for less than 10 dollars. If you don't have one ready, you can also build one like this:

DTE1_______________________________________________DTE 2

9pol 25pol (female)_________________________________25pol 9pol (female)
5    7  ---GND---------------------GND-------  7   5

2    3  ---RxD--------. ,----------RxD-------  3   2
                       X
3    2  ---TxD--------' `----------TxD-------  2   3

7    4  ---RTS--------. ,----------RTS-------  4   7
                       X
8    5  ---CTS--------' `----------CTS-------  5   8

4   20  ---DTR--------. ,----------DTR------- 20   4
                       X
6    6  ---DSR--o-----' `-------o--DSR-------  6   6
                |               |
1    8  ---DCD--'               `--DCD-------  8   1

Connect the cable to the first serial port of both computers.

Then use a Terminal application like PuTTY or Windows HyperTerminal on the computer for receiving the debug messages. Set it up to listen to the first serial port (COM1 [3F8/IRQ4]) and a baud rate of 115200.

After that, boot ReactOS on the test computer and you should receive debug messages. If this doesn't work, check your hardware and your freeldr.ini configuration.

Debug text output to file

Edit freeldr.ini to contain an entry like the following:

[ReactOS_Debug]
BootType=ReactOS
SystemPath=multi(0)disk(0)rdisk(0)partition(1)\ReactOS
Options=/DEBUG /DEBUGPORT=FILE /SOS

Debug text output to screen

KDBG

kdbg

GDB

Needed Items:

The ReactOS build environment now includes gdb by default. In order to use it you must create config.rbuild from config.template.rbuild and change it so that:

<property name="KDBG" value="0" />
<property name="DBG" value="1" />
<property name="GDB" value="1" />

After you recompile(specially, kdcom and ntoskrnl), start Qemu as you normally would, but add the following command line parameters:

-s -S

This is done so that Qemu starts in the STOPPED state, and allow you to connect using gdb. Now its time to get GDB of the ground

(Assuming you are in the rosbe command line), enter “gdb” to start gdb.
Enter “file ./output-i386/ntoskrnl/ntoskrnl.exe” to tell gdb where to load information about the kernel.
Enter “set disassembly-flavor intel” for good measure.
Enter “target remote localhost:1234” to connect gdb to what we are debugging.
Enter “c” (for “continue“) to have gdb instruct Qemu to start/continue execution of the emulation.
To manually pause execution, make sure your gdb window has focus and simply enter <CTRL>+<C>

Generating even more output

In order to get meaningful debug output it is sometimes necessary to enable extra verbosity.

Turning on verbosity at compile time

ReactOS Style

Nearly all ReactOS modules use the built in "ReactOS style" debugging functionality. This style is characterized by:

Verbosity level is usually defined per file.
Only 2 message levels:
- always enabled (DPRINT1)
- only enabled when NDEBUG is not defined (DPRINT)

Files that follow this style can easily be spotted by this code:

#define NDEBUG
#include <debug.h>

To enable full verbosity just comment out the "#define NDEBUG", and remember to uncomment it when submitting patches.

WINE Style

sample line to enable debug channels in usermode applications, in file:

boot/bootdata/hivesys.inf

	 ; Debug channels
  	 HKLM,"SYSTEM\CurrentControlSet\Control\Session Manager\Environment","DEBUGCHANNEL",0x00020000,"+ole,+rpc"

Turning on verbosity at runtime

The easiest way to turn on debug verbosity on any particual component is to use DEBUGCHANNEL enviroment variable. For example, to get all debug messages from MSI, simply run in CMD:

set DEBUGCHANNEL=+msi

Then, run the app you wish to test. You can add debug messages from multiple components, for example:

set DEBUGCHANNEL=+msi,+rpc,+ole

and so on. After closing down CMD, debug verbosity will change to default.

Breaking into the debugger

Bugchecks occur when the operating system can no longer operate safely and to avoid corrupting data, it halts operation. This will normally throw up a Blue Screen Of Death, but if you have the kernel debugger activated, it will drop you into the prompt giving you access to explore the system state. By default ReactOS debug builds have the integrated kernel debugger(kdbg) enabled. Release builds do NOT have this feature enabled and will only display a blue screen.

There are two ways for forcing a bugcheck, each one employing a different method:

Dynamic

If you have a debug build and want to halt the system for any given reason and break into the the kernel debugger, you can force a bugcheck from the keyboard by simply typing:

tab+k

Remember that kdbg output goes out through the serial port, but it receives input from the console by default.

To receive input through the serial port as well, add the command /KDSERIAL to the freeldr.ini boot options. It could look like this:

Options=/DEBUG /DEBUGPORT=COM1 /BAUDRATE=115200 /SOS /KDSERIAL

Breaking on user mode Exceptions

For each type of exception known by KDB you can set the condition when KDB should be entered individually for first and last chance. The possible settings for the conditions are never, umode, kmode and always. never means that KDB will never be entered when the exception is raised, umode means that it will be entered when the exception was raised in usermode, kmode means to enter KDB when the exception was raised in kmode, and always means to always enter KDB.

To change the condition to enter KDB on INT3 to "always" (default is "kmode"), enter the debugger and type:

set condition * first always

Type "cont" to continue normal execution.

Static

This is useful when you want to halt the operating system when it hits a particular area of code you might be debugging. This is especially useful as you can get an immediate back trace to see where the code flow came before the bugcheck was forced.

This is done by using KeBugCheck() or ASSERT() in the code.

Generating a backtrace

In order to generate a backtrace you must break into the KDBG prompt.

Enter 'bt' and hit <return>, you should see something similar to the following:

 (drivers\filesystems\vfat\rw.c:809) <\ReactOS\system32\kernel32.dll>
 Entered debugger on embedded INT3 at 0x0008:0x800935f2.
 kdb:> bt
 Eip:
 <ntoskrnl.exe:935f3 (lib\rtl\i386\debug_asm.S:31 (DbgBreakPoint@0))>
 Frames:
 <vfatfs.sys:97de (drivers/filesystems/vfat/misc.c:111 (VfatDispatchRequest))>
 <vfatfs.sys:9b25 (drivers/filesystems/vfat/misc.c:167 (VfatBuildRequest))>
 <ntoskrnl.exe:3ab23 (ntoskrnl/io/iomgr/irp.c:1088 (IofCallDriver))>
 <ntoskrnl.exe:36206 (ntoskrnl/io/iomgr/iofunc.c:686 (IoSynchronousPageWrite))>
 <ntoskrnl.exe:59daa (ntoskrnl/mm/section.c:6330 (MmspWriteDataSectionPages))>
 <ntoskrnl.exe:244c6 (ntoskrnl/ex/work.c:162 (ExpWorkerThreadEntryPoint))>
 <ntoskrnl.exe:70e90 (ntoskrnl/ps/thread.c:134 (PspSystemThreadStartup))>
 <ntoskrnl.exe:7b142 (ntoskrnl\ke\i386\ctxswitch.S:258 (KiThreadStartup@156))>
 kdb:>

The bt command will show a backtrace of the currently attached thread, so it may be necessary to use the 'thread attach' command, please refer to the kdbg manual for more details.

Examining this in more detail, we can see that the bugcheck occured via the INT3 operation which then dropped us into kdb. The next line shows us Eip which is the instruction pointer, and this points to the last address before the system halted.

Following on from that are the frames. This is the important part of generating our backtrace and it contains all the function addresses in the buildup to the bugcheck. This is the crutial information developers need to understand the codeflow before the bugcheck.

Translating Addresses

Occasionally, there will come a time when you will need to manually translate addresses. When kdbg is not enabled and a bugcheck occurs, you will be presented with a stack trace similar to the following:

(subsystems\win32\csrss\win32csr\conio.c:1101) Console_Api Ctrl-C 

*** Fatal System Error: 0x00000001
                       (0x80079279,0x00000000,0x0000FFFF,0x00000000) 

<\SystemRoot\System32\NTOSKRNL.EXE: 29bb>
<\SystemRoot\System32\HAL.DLL: 4749>
<\SystemRoot\System32\NTOSKRNL.EXE: 54cb4>
<\SystemRoot\System32\NTOSKRNL.EXE: 582bf>
<\SystemRoot\System32\NTOSKRNL.EXE: 583fd>
<\SystemRoot\System32\NTOSKRNL.EXE: 89956>
<\SystemRoot\system32\drivers\videoprt.sys: 2417>
<\SystemRoot\system32\drivers\vbemp.sys: 17f5>
<\SystemRoot\system32\drivers\vbemp.sys: 19cf>
<\SystemRoot\system32\drivers\videoprt.sys: 1c48>
<\SystemRoot\System32\NTOSKRNL.EXE: 34c17>
<\SystemRoot\System32\NTOSKRNL.EXE: 21e0>
<\SystemRoot\System32\NTOSKRNL.EXE: 2908>
<\SystemRoot\System32\NTOSKRNL.EXE: 29bb>
<\SystemRoot\System32\NTOSKRNL.EXE: 85fa8>

As you can see, this is largely the same as the information presented when issuing a 'bt' command in kdbg. The problem here however, is that only the addresses are given. As these addresses are different for everyones builds, this information is useless for anyone trying to follow what events occured in the lead up towards to bugcheck.

The solution here is to translate the addresses into human readable function names. This is done via a tool named 'raddr2line' which is a modified version of the unix tool 'addr2line'. This tool will translate the addresses given to it into file names and line numbers. It does this by using debug information in the executable files to accociate the address with this human friendly info and outputs it into the console. This information can be pasted into the above debug log alongside the addresses providing the developers with a detailed stack trace.

radr2line is included in the Reactos Build Environment. It is invoked in the following way :

raddr2line <file> <address>

So taking the bottom address in the above stack trace :

C:\Users\Ged\MyFiles\ReactOS\clean_source>raddr2line ntoskrnl.exe 85fa8

C:\Users\Ged\MyFiles\ReactOS\clean_source\output-i386\ntoskrnl\ntoskrnl.exe
obj-i386\ntoskrnl\ex\zw.S:253 (ZwClearEvent)

we can see here that the address translation for 0x85fa8 is line 253 in file ntoskrnl\ex\zw.S (this will differ if you try it on your build)

This information can now be added into the above stack trace as follows :

<\SystemRoot\System32\NTOSKRNL.EXE: 29bb>    <enter next one here>
<\SystemRoot\System32\NTOSKRNL.EXE: 85fa8>   obj-i386\ntoskrnl\ex\zw.S:253 (ZwClearEvent)

Debugging

Contents