Debugging for Beginners in Windows and VirtualBox

Got a ReactOS tutorial to share? Drop it in here

Moderator: Moderator Team

Post Reply
RaptorEmperor
Posts: 247
Joined: Mon Jun 26, 2006 12:04 pm
Location: Baltimore, Maryland, United States

Debugging for Beginners in Windows and VirtualBox

Post by RaptorEmperor »

Tutorial for Debugging ReactOS with VirtualBox, com0com, and HyperTerminal in Windows
By RaptorEmperor


To the ReactOS rookie, the thought of debugging ReactOS can be intimidating. Almost as intimidating is figuring out how to get it done. I finally worked up the nerve to try debugging when encountering a particularly nasty bug, so I'll share my newfound knowledge in a simple 20-step process. To my knowledge, there aren't any step-by-step tutorials on your first debugging experience, so I'll write it for you.

For this tutorial, we'll be working with Windows XP, and the included HyperTerminal application. We'll also be using the open-source VirtualBox and com0com applications. Once you grow comfortable with debugging, you can use different applications of your choosing, but, to get you started, we'll use these programs. I'll be assuming you already have some familiarity with VirtualBox and ReactOS. If not, feel free to test the waters and get a feel for the software before debugging.

1. Download VirtualBox, if you haven't already. http://www.virtualbox.org/
2. Download a ReactOS ISO. Trunk builds are preferred for debugging purposes. http://www.reactos.org/getbuilds/
3. Create a ReactOS virtual machine on VirtualBox.
4. Download com0com. http://sourceforge.net/projects/com0com/files/
5. Install com0com. For the sake of simplicity, have it on your start menu.
6. Change the default names of your virtual serial ports. They'll be something like "\\.\CNCA0" or whatever. You can do this by going to the "Setup" program. If you've installed com0com to the Start menu, the file path is Start>All Programs>com0com>Setup. If you see a small window with a bunch of checkboxes and red, black, and green dots connected by lines, you're there. For this tutorial, name your first serial port COM4 and your second port COM5.
7. Go into the settings for your ReactOS virtual machine in VirtualBox.
8. Right-click your ReactOS virtual machine and select the "Settings" option. (Alternately, highlight your ReactOS virtual machine and click the "Settings" icon at the top of the screen.)
9. Under Settings, you should see a serial ports icon. Select it and you'll be presented with a menu. Under the "Port 1" tab, check the "Enable Serial Port" option. This will allow you to click the options underneath.
10. Set the "Port Number" drop box to "COM1". "Port Mode" should be "Host Device". For "Port/File Path", type in "COM4" (without the parentheses, of course). This is the first virtual serial port we made earlier.
11. Click "OK".
12. Open HyperTerminal. From the Start menu, the filepath is All Programs>Accessories>Communications>HyperTerminal.
13. You will be presented with a message box entitled "Connection Description". The name and symbol aren't really that important. I called my first connection "ReactOS Debug" and chose the atom with the thingys flying around it (far, far right) as my Icon, since it seemed appropriate for ReactOS, but it's your call.
14. The next message box will be entitled "Connect To". Under the "Connect using" dropbox, select "COM5". This is the second virtual serial port you created with com0com. Then click "OK".
15. You will now see a menu that says "COM5 Properties". Set the "Bits per second" drop box to "115200" and click "OK". Everything is fine as-is.
16. HyperTerminal is now running. You should have a blank screen in HyperTerminal.
17. Back in VirtualBox, right-click your ReactOS virtual machine and select "Start". (This can also be achieved with the "Start" icon at the top of the VirtualBox main menu.)
18. The virtual machine should start. When FreeLoader boots up, select the ReactOS (Debug) option.
19. ReactOS will start booting up, and you should start recieving output in HyperTerminal. Take a minute and pat yourself on the back for a job well done, and maybe post on your Facebook about how l33t you are, like I did.
20. When you're done debugging, you can shut down HyperTerminal or ReactOS. You won't break anything by shutting one or the other down first. Welcome to the world of ReactOS debugging!

Once you've gotten the hang of getting output from your serial port, now the next step is learning how to file bug reports! (To be continued...)

I used the following articles on the ReactOS wiki for information.
http://www.reactos.org/wiki/Debugging
http://www.reactos.org/wiki/Com0com
http://www.reactos.org/wiki/VirtualBox# ... bug_output

All credit goes to the writers of those articles. I'm just compressing them in an easy-to-read format.Also, kudos to hto, Haos, and vicmarcal for guiding me in the right direction on debugging.

I don't currently run Vista or Windows 7, so if someone wants to correct me or add more information, let me know what to add. If you're trying to debug on Linux, and you already know how to use Linux, you're already more l33t than me, so I can't help you. :P
Last edited by RaptorEmperor on Thu Jan 07, 2010 7:49 am, edited 2 times in total.
RaptorEmperor
Posts: 247
Joined: Mon Jun 26, 2006 12:04 pm
Location: Baltimore, Maryland, United States

Re: Debugging for Beginners in Windows

Post by RaptorEmperor »

I accidentally posted this into "Support" instead of "Tutorials" like I meant to. Someone with moderator access please place this in the right forum. Thanks! :oops:
MadWolf
Posts: 688
Joined: Sat Dec 31, 2005 4:19 am
Contact:

Re: Debugging for Beginners in Windows and VirtualBox

Post by MadWolf »

hi
have a look at my ReactOS in virtualbox debugging Tutorial
http://sites.google.com/site/madguids09 ... ng-reactos
edit My New Tutorial
https://sites.google.com/site/madguids0 ... ng-reactos
Last edited by MadWolf on Fri Nov 05, 2010 4:58 am, edited 1 time in total.
steveh
Posts: 271
Joined: Sat Dec 18, 2004 10:02 pm

Re: Debugging for Beginners in Windows and VirtualBox

Post by steveh »

Hi,

com0com is a program for the windows host, on which virtualbox is running, is it?
What do when virtualbox is running on a linux host?
RaptorEmperor
Posts: 247
Joined: Mon Jun 26, 2006 12:04 pm
Location: Baltimore, Maryland, United States

Re: Debugging for Beginners in Windows and VirtualBox

Post by RaptorEmperor »

steveh wrote:Hi,

com0com is a program for the windows host, on which virtualbox is running, is it?
What do when virtualbox is running on a linux host?
Personally, I've never tried debugging on Linux, since I use Windows as my primary OS and I run into problems when I do use Linux on my computer.

I know other users on here use Linux, if anyone else on here on the forums knows how to do this on Linux, please post a link to another tutorial or explain it or something.
MadWolf
Posts: 688
Joined: Sat Dec 31, 2005 4:19 am
Contact:

Re: Debugging for Beginners in Windows and VirtualBox

Post by MadWolf »

steveh wrote:Hi,

com0com is a program for the windows host, on which virtualbox is running, is it?
What do when virtualbox is running on a linux host?
hi
have a look at http://www.reactos.org/wiki/VirtualBox
Haos
Test Team
Posts: 2954
Joined: Thu Mar 22, 2007 5:42 am
Contact:

Re: Debugging for Beginners in Windows and VirtualBox

Post by Haos »

RaptorEmperor wrote: 10. Set the "Port Number" drop box to "COM1". "Port Mode" should be "Host Device". For "Port/File Path", type in "COM4" (without the parentheses, of course). This is the first virtual serial port we made earlier.
Does it really work on XP? I really doubt that... Anyway, on Vista/7 in "Port/File Path", you should type "\\.\COM4" (without the parentheses, of course) instead of "COM4". Trying your way results here in "Fail to open host device COM4" error.
MadWolf
Posts: 688
Joined: Sat Dec 31, 2005 4:19 am
Contact:

Re: Debugging for Beginners in Windows and VirtualBox

Post by MadWolf »

Haos wrote:
RaptorEmperor wrote: 10. Set the "Port Number" drop box to "COM1". "Port Mode" should be "Host Device". For "Port/File Path", type in "COM4" (without the parentheses, of course). This is the first virtual serial port we made earlier.
Does it really work on XP? I really doubt that... Anyway, on Vista/7 in "Port/File Path", you should type "\\.\COM4" (without the parentheses, of course) instead of "COM4". Trying your way results here in "Fail to open host device COM4" error.
yes i have always set the Port/File Path to COM4 windows xp host
Post Reply

Who is online

Users browsing this forum: No registered users and 13 guests