The ReactOS installer is probably one of the oldest extant chunks of code in ReactOS that has not undergone significant overhauls for a very long time. Besides some tuning by Alex Ionescu to reduce resource usage and adding initial support for removable devices by Cameron Gutman, it has not seen any real major changes since Eric Kohl put it together all those years ago. For the developers, this was not too much of an issue since the installer did its job perfectly fine for the most part. Bugs that cropped up would get squashed and as a whole there was no driving urgency to overhaul the installer since it just worked. Elements of the wider community however were considerably more vocal about the need for an overhaul though.At the most basic level, most of the complaints levied at the installer was that it was not shiny enough.
Installers for Windows and most other Linux distros became considerably fancier with a point and click GUI instead of purely keyboard console style interfaces. Another issue brought up was how the liveCD could not be used to install ReactOS. Internally, these complaints were regarded with fairly low priority since the developers were more focused on things like making sure ReactOS did not crash after it had been installed. They are technically still fairly low priority considering the amount of issues still to be resolved in more important components, but a few new core requirements have offered the opportunity to combine functional improvements with architectural ones.
The most direct way to address complaints about the installer not looking nice enough would be to just migrate the first stage installer into a GUI application that is part of the liveCD. The question of how difficult this is to do depends on how compartmentalized the code responsible for handling user input is from the code that does the actual work of installing ReactOS. The answer in this regard is actually quite promising so kudos to Eric for his initial design work. The interface code is pretty strictly separated from the code that does the heavy lifting and it is mostly a matter of writing a new interface that will then plug in the underlying installer code. Mostly, but not entirely.
If the project is going to overhaul the setup code however, the overhaul needs to resolve several issues to be worthwhile. After all, having a shinier installer is a side benefit and not the prime objective here. The first thing to note is that the installer is a native NT application, not a win32 application. As such it is directly calling various NT functions instead of going through the corresponding win32 API. This in and of itself is not a problem since a native NT application can run perfectly fine even when the win32 subsystem is running. The only issue though is if there is a desire to make the installer fancier and have it do things that are easier to do with the win32 API than the native NT API. Whether migrating to the win32 API is even feasible if there is a desire to share code between a stripped down bootcd installer and a fancier GUI livecd installer is also a point that will influence any such decision. For the short term at least, there does not appear to be a need to migrate the setup code to the win32 API and its usage of the native NT API will do just fine.
The second issue deals with where ReactOS deals with partitions. Or rather, the second group of issues. The partitions as created by the installer currently are not properly recognized by the Windows installer or gparted. This is a bug on ReactOS' part and one Eric is in the process of dealing with. Other limitations include not being able to handle extended partitions. Another issue that most testers will recall that ReactOS needs to be installed on the first partition of a disk. At this point no one on the team remembers the exact cause of the limitation, but most assume it to be due to either hardcoding or some bug in the code. The partitioning code in the installer is also something of a mess, one that Eric has been working diligently to clean up, not without some bumps along the road. It is also probable that the bug is not just in the installer but also in the bootloader, so unraveling this bug will be a multi-part process.
Another issue is the overall flow of the partitioning operation. Right now due to the order of operations the installer cannot create multiple partitions at the same time because creating a partition and choosing its filesystem is done in two separate steps. This also has the consequence of in the case of installations that stop after a partition was created but before the partition is formatted, a reboot will not properly see the previously created partitions. The solution of course is to create a partition and select its filesystem type at the same time.
One of the things ReactOS does not have proper support for right now is installing on SSDs. Yes, ReactOS can be installed on one, but the inefficient nature of the current partitioning scheme does not do a SSD's lifetime any good. Right now partition alignment is based on cylinder boundaries with 63 sectors per track. This unfortunately does not align with the 4K blocks that SSDs use so there is some inefficiency/waste going on here. One solution would be to detect whether ReactOS is being installed to a SSD versus a harddrive and align accordingly. The other solution would be to always use megabyte alignment for partitions.
The next issue is fairly complicated since definitively fixing it requires changes from the kernel to the storage stack to the bootloader in addition to the installer code. Master Boot Record partitions have been with us since the early 1980s but they are slowly being superseded by GUID Partition Tables. ReactOS at present does not support GPT based partitioning, though this limitation is based on missing functionality in more than one place. The kernel saw some work by Pierre Schweitzer so it is aware of and can work with GPTs but the storage stack and anything that uses it is not. And since the storage stack does not support GPTs, the installer that relies upon it therefore also does not. One of the missing pieces in the stack is in the IOCTL_DISK_SET_DRIVE_LAYOUT_EX structure. While ReactOS has an implementation, our implementation only works with MBR and not GPT.
This issue is not related to partitions, but it is related to what GPTs are used for, UEFI support. GPT's relationship with UEFI is a bit more tenuous since it is possible to perform a UEFI boot from MBR partitions, though Windows does not support this (or booting from 32bit UEFI for that matter, but that's a separate point). Whether ReactOS will is a matter that will depend on if anyone is willing to exert the necessary effort to make it happen. That said, installing ReactOS to boot natively on a UEFI system will almost certainly require a somewhat different code path. UEFI has somewhat different expectations when it comes to placement of bootloaders and the like and the installer must be updated to fulfill those requirements.
The last issue, or at least the last one that I will cover today, is mostly a code organization point and not really an 'issue' per se. If the installer code is split and the actual installation code is pulled out into something that can be shared, the team will ultimately need to decide whether that code will live as a static lib or a DLL. This one just boils down to whether people want to bother setting up the DLL exports to create a DLL versus just providing headers and a lib file for linking in all the code. Space savings are going to be pretty small anyway so it more or less falls into, who wants to do the work for a DLL version.
Addendum: Soliciting information and feedback for this post resulted in a twenty-five long email chain between various developers, probably the most feedback I have gotten on any single topic. Color me amused.