File Browser Menus
Most of the actual work in managing the file browser menus is done in the WM_INITMENUPOPUP message handler. This message is initially received by the shell browser object, which does its own processing first, and then forwards it to the shell view. The message would normally contain the index of the menu item that opened the submenu that is being initialized. Because the shell uses custom menus based on toolbars, the submenus don’t know the index of the item that opened them, so the shell browser has to check which menu is opening, and then set the appropriate index in the right parameter, before forwarding.
Both the edit menu and the view menu are initialized only once, when the view activates for the first time.
The next step was to implement the File menu. Unlike the edit and view menus, which are only initialized once, the file menu is cleaned up and the items of the current selection are added to it every time the menu is shown. For this, Windows has a special function that gathers the available actions for the current selection, and builds a special menu for those items. If we don’t have this function yet, then the code just obtains the items of the context menu, and adds them to the File menu as-is. This may need future improvements, but serves its purpose for now.
The favorites menu now uses the CMergedFolder.
Since Windows keeps the selection shown on windows (but not the desktop), there is a flag that is only set when not in desktop mode.
Explorer Command Line
Immediately after the results are obtained, SHExplorerParseCmdLine relays the structure as-is, to SHCreateFromDesktop. The browseui component has two different behaviours depending on if the /SEPARATE flag was specified in the commandline.
If it was, it creates a special window it calls “Desktop Proxy”, by using the CProxyDesktop, and this hidden window it used in place of the actual desktop as a “host” for the folder windows.
If the flag was not specified, it looks for an existing desktop, and then it sends a few special messages, using shared memory to pass on some data. What is the data? Well that’s where the complicated part comes.
Here are some details about the IPC mechanism used by browseui to open new windows in the existing process.
Investigating the values used in the shared memory buffer, it seems there are some matches with the input given to the function. Observation shows some of the matched parameters, and shows that at the end of the buffer, the contents look remarkably similar to the data seen while debugging ITEMIDLIST objects.
The buffer does indeed contain some sort of header or struct, followed by the data of three ITEMIDLIST objects, and finally the path string that would be the path of the folder to open, in case there was no ITEMIDLIST for it.
Then is was a matter of investigating the other end of the IPC: the message handlers for messages 1037 and 1035.
Message 1037 is related to rooted idlists and appears to be sent by browseui when looking for the root window in order to find the target for message 1035. The contents of this message is a bit unclear, other than it’s send through a call to EnumWindows (presumably in the callback). The contents of the message may involve the target ITEMIDLIST, but this may need verification. There is a chance this message may also be used to find an existing window for the folder, so that it can activate the window instead of opening it. For now, handling of message 1037 is stubbed.
Message 1035 is the one that uses the information mentioned earlier, which presumably means “open new window with this information”. Not all the details of the shared buffer are known yet, but it appears that WPARAM is set to 0/NULL, and LPARAM is set to the shared memory handle. This says that the shared memory may be obtained with the PID of the target instead of the PID of the caller. A previous call to GetWindowThreadProcessId (presumably with the HWND obtained by the results of the root window search) appears to confirm so.
The message is sent to one of two windows: either the actual desktop window (that is the “ProgMan” window that lies at the root of the explorer hierarchy), or to the first “Proxy Desktop” window it can find.
The proxy is used in one of two conditions: either if the /SEPARATE flag is specified in the command line, or if the “Open folders in a separate process” option is enabled in the settings. If we still don’t have such an option implemented yet, then we implement that check using a global variable.
It is confirmed that the shared memory is allocated in the context of the target window, and that it was using the “SH*Shared” set of functions (an abstraction of the low-level shared memory) as a means to use shared memory.
Then for the case of the separate process, which needs a proxy window, there is code that creates this hidden proxy window and tells it to open a new folder. In the proxy implementation, there's a handler for the two messages, even though only 1035 will be implemented for now. The other is a stub in case one day we want to add rooted folders, which may need message 1037 to be windows-like.
Then what was left was to investigate what happens when message 1035 is received by the desktop window. This message calls SHOnCWMCommandLine, which is takes the shared memory handle, decodes the parameters from the packet, and calls SHOpenFolderWindow with the information.
In windows, SHOpenFolderWindow does a whole lot of work, including deciding if it should activate an existing window or open a new window. Since we don’t really care about that just yet, we implement this function based on the existing implementation of SHOpenNewFrame. Some of the code was taken from there and moved to SHOpenFolderWindow, making SHOpenNewFrame call this function instead.