WinDBG (pronounced "Wind bag"), is Microsoft's advanced debugging tool. ReactOS, being very compatible with windows and able to be compiled with Microsoft's compiler, is able to be debugged in Kernel and user mode using WinDBG.
To begin, one needs to compile ReactOS from source using Microsoft's tools. This creates executables and Program DataBase symbols (PDBs), which WinDBG uses to step through the code. One requires the minimal cmake and ninja download (see Build Environment), and a version of Microsoft's compiler (which is available with desktop versions of visual studio).
One should create a bootcd using Microsoft's tools, then boot it using a virtual machine. The first time, you probably won't see anything apart from a black screen, this is usual behaviour. The ReactOS kernel is waiting for WinDBG to attach to it before it starts the boot process proper. This can be achieved for a virtual machine using a named pipe. The debugging page has instructions on how to set this up. In WinDBG, go to File->Kernel Debug (or ctrl+K), COM, turn on the pipe flag (I set reconnect too), and put the name of the pipe as the port, e.g., \\.\pipe\ros_pipe.
ReactOS should connect and start booting for the installation, so install and reboot for second stage. To use WinDBG, select ReactOS (Debug) from the freeloader menu (again it will hang until connection). Note that when WinDBG is connected, ReactOS boots and performs significantly slower.
To break into the kernel, mash ctrl+break in the WinDBG gui (or ctrl+c if one is using the command line version). Alternatively, one can press tab+k in ReactOS to break. From here, one can set breakpoints and step through source code, but there's a few more things to set up first. The command 'g' is used to restart the machine.
Recent versions of WinDBG don't work in ReactOS due to unimplemented layered windows, and the installers from Microsoft are particularly troublesome. However, a standalone version of windbg is available on the net, and the command line versions cdb.exe and ntsd.exe do work very well. This allows for debugging of user mode applications in ReactOS, and also for deferred debugging of user mode code on a host windows system running WinDBG as kernel mode debugger. This can be done with the -d flag when launching the command line versions, for example ntsd.exe -d c:\Reactos\explorer.exe will start the program explorer.exe and break out to windbg at the initial break point. This is particularly useful if one's symbols and source code reside on the host machine.
Sybmols must be used for WinDBG to be really useful, and one can set them for one WinDBG instance or using environment variables. Let's look at environment variables first.
Set _NT_SYMBOL_PATH to C:\path\to\reactos\output-VS11-i386\reactos
Sometime's it's useful to also have Microsoft's symbols available as well, this is fine and doesn't interfere in any untowards way, so one can instead use
To break into a step through proper C code, go to File->Open Source File (ctrl+o) and grab a kernel code file. User space connections are iffy, but covered at the bottom of this page.
Research on Windows
ReactOS aims to be compatible with Windows (current target server 2003) for essentially all apps, drivers, and even dlls, so a lot of research is required. WinDBG can assist in this by giving hints about how Windows performs certain operations.
For example: Currently copy and paste functionality in ReactOS is handled using the default context menu, in Windows however, the call path goes through the context menu, but is quickly pushed into the drag and drop implementation. This "dries" the code by avoiding "wet" code (write everything twice). So let's have a look about how to discover this for one's self.
Fire up WinDBG in Windows and ensure you've the microsoft symbols server loaded (see above). Now, attach to the explorer process (F6, then select explorer). Now we know that if we copy and paste a file using the context menu (or ctrl+c, ctrl+v) that a file operation is going to be performed as one of the last things done. A quick trip to MSDN shows that SHFileOperation was used prior to Vista, while IFileOperation::CopyItems was used for Vista and later. SHFileOperation lives in shell32 and has two versions: unicode and ansi, so set a breakpoint on the unicode version using
For Vista it's a little bit more tricky, as IFileOperation is an interface, and we need a class to set the breakpoint on. Using x SHELL32!*::CopyItems tells us that the class is (somewhat logically) called CFileOperation, so we can set a breakpoint on this one with
Now restart explorer with 'g' and copy and paste a file. The process should break and one can use the 'k' command to see the backtrace. By tracking backwards, one can find out the calling helper class, see it's run in async manner, and set additional breakpoints to find out how that was called too.
|kp||generate a backtrace|
|ta||trace to address|
|x *!||full modules list (atm lm is only showing basic modules)|
|!drvobj <driver_object address> 0x7||dumps driver object details - start/unload + irp handlers|
|!gflag +soe||catch all exceptions (first exception handling) apart from STATUS_PORT_DISCONNECTED or if the exception code is not an error code|
|!gflag +hpa||enable page heap flag turns on page heap verification, which monitors dynamic heap memory operations|
|!process 0 0||list all process with basic info|
|!process <addr> 0x1e||list detailed info of attached process, with its threads|
|!process <addr> 0x1f||list stack traces for all the threads in the process|
|.process <addr>||attach to the process of a given address|
|!thread <addr>||list info about the thread|
|.thread <addr>||attach to the thread of a given address and set the default thread context in kernel mode|
|.reload /user||reloads user symbols and enables resolving of usermode|
With WinDbg attached as a kernel debugger, it is not trivial to set breakpoints on user-mode code. Since every process has it's own address space, the debugger does not know which process to set the breakpoint in. The
.process command does not solve this problem sufficiently (
.process /i does, but ReactOS doesn't support that yet).
A workaround is to place a kernel-mode breakpoint (those are always reliable) that is likely to be executed in the context of the right user-mode process.
As an example,
will break whenever a window is created*. By executing
.reload /user after this breakpoint is hit, you will be able to look at a backtrace (that hopefully includes the right user-mode symbols) and judge whether you are in the right process at the right time. Once that is achieved, you can place user-mode breakpoints in the context of this process for further examination.
* when debugging issues during OS boot, be sure to break late enough so that win32k is loaded, though — e.g. you can be sure of this by breaking right after the display mode change that makes the desktop's background color appear.