02 Jun 2014

39250

0

Leak-leaking leaks

I began the week by fixing the issue where the right button wasn’t detected by the code managing the start menu and submenus.

I had been told that there was a JIRA issue related to the shutdown dialog, and that it had been implemented in some way in trunk already. I considered the means of calling that dialog from the shell32 function the start menu calls, but I dropped the idea some hours later, when I decided the code in msgina wasn’t designed to be called externally.  At some point someone will have to copy the needed code over to shell32, and replicate the dialog there.

Then I set my sights on the navigation history, the feature that implements the back/forward buttons. I wasn’t sure how hard it was from working, but after some debugging and poking around, I eventually found a strange call in one of the functions that was working in Windows but failing in ReactOS. Checking the parameters, it seemed that this call was never even meant to be there at all, and I’m still unsure why Windows is happy to accept it. Regardless, removing the call, and passing NULL in the parameter it used to fill, works as described in MSDN, in both Windows and ReactOS, and thanks to that, we can now use the history feature properly.

Many of the following hours are fuzzy. I fixed some minor issues, and did a lot of staring at the screen. Basically, I dedicated most of them trying to debug the issue where explorer BSODs after a dozen or so navigation actions. Eventually, after countless hours spanning multiple days, I traced the primary symptom to an USER handle leak. USER handles are handles to objects created by user32, such as windows, menus and dialogs. There are two other types of handles: Kernel handles, which represent files, registry keys, and other kernel32-provided features, and GDI handles, which represent objects such as bitmaps, fonts, brushes, pens, and everything else provided by gdi32. The leaks in explorer appear to be mostly related to HMENU handles.

Trying to replicate the issue, in an attempt to create a smaller test case, I managed to cause a leak of HMENU handles by seting up a nested menu, and then destroying the root menu. This should in theory cause the child menus to be destroyed first, and then the parent, but it didn’t seem to work as expected in ReactOS. The issue was quickly fixed in trunk, but after I applied the changes to the branch (as part of a trunk sync), I noticed the issue hadn’t changed much at all: it still leaked approximately the same number of menu objects per click. So I was stumped. All those hours hadn’t been wasted. The bug was real, yet it wasn’t the same bug that still plagues the file browser window.

I was told by Giannis that Process Hacker is able to show the number of USER objects per process, so I used it to monitor the filebrowser window while it runs in Windows. To my surprise, even after disabling the use of the rshell menus, it still leaks! Which means the leak exists solely within browseui, and browseui alone is the one responsible to avoid it.

So here I am, at the beginning of a new week, writing (late) the weekly report, while a big question mark fills most of my head: Where the [censored] is the leak – or leaks –?

Lastly, before closing off, the obligatory screenshot. This time, of the navigation history showing the Back menu within ReactOS:

Image

Good day/night and until next week!

P.S.: I will try to write the report on friday this week, so that it's early instead of late. ;P

Discussion: https://www.reactos.org/forum/viewtopic.php?f=2&t=13394

This blog post represents the personal opinion of the author and is not representative of the position of the ReactOS Project.