Difference between revisions of "ARM Port"

From ReactOS Wiki
Jump to: navigation, search
m (See Also: Added a bunch more information about running x86 programs on ARM (and other architectures))
(Do not dump extraneous speculation on this page.)
Line 1: Line 1:
 +
{{obsolete}}
 +
 
'''Also see the section on the [[ReactOS_ports#ARM_.28.22arm.22.29|ReactOS Ports]] page'''
 
'''Also see the section on the [[ReactOS_ports#ARM_.28.22arm.22.29|ReactOS Ports]] page'''
  
Line 145: Line 147:
 
*[NTOS]: Fix issue when trying to make both ROS_SECTION_OBJECTs and NT's SECTION co-exist. We relied on the *caller* knowing what kind of section this is, and that can't be a good idea. Now, when the caller requests an ARM3 section vs a ROS section, we use a marker to detect what kind of section this is for later APIs.
 
*[NTOS]: Fix issue when trying to make both ROS_SECTION_OBJECTs and NT's SECTION co-exist. We relied on the *caller* knowing what kind of section this is, and that can't be a good idea. Now, when the caller requests an ARM3 section vs a ROS section, we use a marker to detect what kind of section this is for later APIs.
 
*[NTOS]: For section VADs, we were storing the ReactOS MEMORY_AREA in the ControlArea... however, the mappings of one individual section object share a single control area, even though they have multiple MEMORY_AREAs (one for each mapping). As such, we overwrote the MEMORY_AREA continously, and at free-time, double or triple-freed the same memory area.
 
*[NTOS]: For section VADs, we were storing the ReactOS MEMORY_AREA in the ControlArea... however, the mappings of one individual section object share a single control area, even though they have multiple MEMORY_AREAs (one for each mapping). As such, we overwrote the MEMORY_AREA continously, and at free-time, double or triple-freed the same memory area.
*[NTOS]: Moved the MEMORY_AREA to the "Banked" field of the long VAD, instead of the ControlArea. Allocate MMVAD_LONGs for ARM3 sections for now, to support this. Also, after deleting the MEMORY_AREA while parsing VADs, we now use a special marker to detect double-frees, and we also use a special marker to make sure we have a Long VAD as expected.  
+
*[NTOS]: Moved the MEMORY_AREA to the "Banked" field of the long VAD, instead of the ControlArea. Allocate MMVAD_LONGs for ARM3 sections for now, to support this. Also, after deleting the MEMORY_AREA while parsing VADs, we now use a special marker to detect double-frees, and we also use a special marker to make sure we have a Long VAD as expected.
 
 
 
 
=ReactOS on ARM, running x86 binaries without modification=
 
*[http://www.reactos.org/forum/viewtopic.php?f=2&t=11078 ReactOS on ARM discussion]
 
*[http://wiki.winehq.org/ARM WINE wiki ARM page]
 
**run the assembler parts of x86 in qemu and call function in Wine compiled for ARM, which might speed up some things (darwine tried that before, but they stopped because the byteswapping (big endian <-> little endian) was too much work, but ARM is mostly little endian, so we have a chance if everything works fine with packed structures. I already gathered some information how darwine tried to do it)
 
**Also has some good extra links at the bottom
 
*[http://forum.openhandhelds.org/viewtopic.php?p=13499#p13499 ''VERY'' useful forum topic about running x86 programs on ARM]
 
*[http://blogs.msdn.com/b/b8/archive/2012/02/09/building-windows-for-the-arm-processor-architecture.aspx Building Windows 8 for the ARM Processor Architecture (Microsoft)]
 
*[http://getwired.com/2011/09/20/win32-the-reports-of-my-death-are-greatly-exaggerated/ Win32 on ARM]
 
*[http://stackoverflow.com/questions/8254846/does-windows-8-arm-have-the-winapi-win32 Does Windows 8 ARM have the WINAPI?] (Yes)
 
**Note that this just means the API is there for (potentially) QEMU to redirect to, and an indicator that the API shouldn't change when porting ReactOS to ARM
 
*[http://forum.winehq.org/viewtopic.php?p=65885#65885 I want to run x86 binaries in QEMU and redirect the win32api to the wine as a BACKEND on ARM/MIPS platform]
 
**win32 applications(X86) -> qemu (translate)-> wine (win32 api)
 
***The big difference is that you aren't running WINE/REACTOS x86 libraries in QEMU, you are running WINE/REACTOS ARM libraries in ARM REACTOS
 
***This gives you a large speed benefit
 
**[http://forum.winehq.org/viewtopic.php?p=66047#66047 Transitive technology has such technology. Years ago they worked together with Transgaming to run games on non-x86 architectures. Later on Transitive also provided this technology to Apple for running PowerPC OSX apps on x86 I believe.]
 
*[http://forum.winehq.org/viewtopic.php?p=63221#63221 There is a component of qemu, which name is TCG, and it is able, for example, to translate x86 assembly instructions into ARM ones at runtime.]
 
 
 
*[http://forum.winehq.org/viewtopic.php?p=67218#67218 Qemu usermode emulation is one of the most advanced forms of paravirtualization. It emulates at the syscall to kernel point. So every time application does a syscall the syscall is processed natively. Do all the NT syscalls, have them run arm, returning to the x86 emulated user-space, would help performance a little. Also this would add the means to call gate from the x86 code out to the arm version of Winelib. So reducing overheads of emulation since less x86 code would have to be emulated.]
 
*[http://forum.winehq.org/viewtopic.php?p=67235#67235 Currently Qemu userspace operation is that it receives a system call request from a running application. Arch convert the request and send it straight on to the native Linux kernel then does the same to the kernel answers. Basically there is no guest kernel in qemu usermode emulation. Its the highest level of paravirtualization you can do. Ie no kernel and no devices emulated.]
 
 
 
*I believe someone would need to make a QEMU [http://qemu-buch.de/cgi-bin/moin.cgi/UserModeEmulation User Mode Emulation] module for Win32, since all the QEMU user mode modules are targetting Linux.
 
*[http://wiki.qemu.org/download/qemu-doc.html User mode emulation: In this mode, QEMU can launch processes compiled for one CPU on another CPU.]
 
 
 
*[http://apple.slashdot.org/comments.pl?cid=10046066 Darwine]:
 
**When the project is complete, OS X users will be able to open EXE files with Darwine. Darwine will use QEMU to execute the x86 instructions, however when the program makes calls to the Windows API, Darwine calls those functions in WineLib compiled natively for PPC.
 
**So, where Bochs and VirtualPC and others like them emulate the entire operating system environment (Emulated BIOS, emulated hardware, emulated Windows, and finally the emulated x86 application ... overhead city), Darwine allows applications to run essentially linked to native code - Wine/WineLib for PPC.
 
**Thus for most Windows applications, the GUI and event handling and everything else the Windows API is good for will be executed in native PPC code. QEMU will then emulate an x86 processor for all the compiled code in the application.
 
**Imagine some internal corporate application that uses all standard Windows widgets to let a user interact with some data: all those widgets plus the menu and root pane will be handled by the native WineLib code except when the programmer has included some special functions or number-crunching routines that are emulated on QEMU's fake x86.
 
*[http://talk.maemo.org/showthread.php?t=43480 A discussion about experiences doing this with Wine on ARM]
 
Darwine and QEMU
 
*[http://darwine.cvs.sourceforge.net/viewvc/darwine/qemu/darwin-user/ CVS for Darwine QEMU work]
 
*[http://git.qemu.org/?p=qemu.git;a=tree;f=darwin-user Darwin-User] user-mode emulation target source branch
 
*[http://git.qemu.org/?p=qemu.git;a=tree;f=linux-user Linux-User] user-mode emulation target source branch
 
*[http://git.qemu.org/?p=qemu.git;a=tree;f=bsd-user BSD-User] user-mode emulation target source branch
 
 
 
  
 
[[Category:Documentation]]
 
[[Category:Documentation]]

Revision as of 19:50, 20 September 2013

Template:Obsolete

Also see the section on the ReactOS Ports page

Building

Obtain the React OS Build Environment for ARM (RosBE-ARM). The latest version is 1.0 and available from: http://reactos.colinfinck.de/download/RosBE-Windows/RosBE-ARM-1.0.exe.

Launch the RosBE-ARM command prompt, and type "makex" on the command line. After some minutes, you should have the whole tree built.

Making a RAM disk

Currently, the ARM port will only boot from a RAM disk, either in NOR/NAND flash on a real device, or an emulated one like QEMU. A RAM disk is a simple flat image file that is loaded into memory.

To generate one of these images, you may use a tool such as qemu-img, the Mac OS Disk Image utility, or any other tool capable of generating a raw image file that corresponds to a hard disk image. You should generate a RAM disk of 32MB or less.

You then need a tool (on Windows), or the proper set of commands, to mount this virtual file/image as an actual hard drive on your machine. This step is required as the ARM port does not have an "installer" of any sorts, and cannot partition your disk, so you must do so yourself.

On a system such as Linux, you can use a local loopback mount, and then launch mkfs_fat on the mounted device. You can use either FAT16 or FAT32, as both have been tested with success. On OS X, the Disk Utility will automatically ask you to partition/format the image. Make sure you select "MS-DOS" partitioning when asked. On Windows, you may need to obtain a tool that can mount such images, such as vdkmount. Alternatively, you may use this image as an x86 ReactOS hard disk in QEMU, and then launch the x86 ReactOS setup program. Continue through the installer pages until you reach the formatting step, and allow ReactOS setup to format the disk. After that, you should be able to quit QEMU and have a formatted image file.

IMPORTANT: Usually, your raw image file will look like a real "disk". It will have an MBR sector at 0x00000000, followed by a boot sector for your first partition somewhere else on the disk. Sometimes this is at sector 63, 2, 1, 60, etc. You will need to know the offset of where the partition starts. Often this will be 0x7E00 (32256), although on Mac OS X .dmg images it will be 0x200. This offset is easy to find, either with a tool that understands your image file when mounting it, or by using a hex editor and locating the boot sector (look for "FAT").

Installing

Now that you have a mountable, formatted, FAT image file, go ahead and mount it, and set your ROS_INSTALL environment variable to the \reactos folder on the virtual drive. For example, if you mapped it to X:\, this would be ROS_INSTALL=X:\reactos.

Now issue a "makex install" from the RosBE command line. This will copy all the files to your new RAM disk.

The last thing you'll need is a freeldr.ini. You can write one from scratch, or simply take the usual one that the x86 ReactOS setup program installs on your x86 ReactOS partition/image file. You'll want to keep the ReactOS entry that's marked as "(RAMDisk)".

Preparing to boot

Now that you have a RAM disk, you are ready to build the final components. The ARM port basically uses an LLB (Low-Level Bootloader) to initialize the hardware, and then pass control to FreeLDR, the ReactOS Boot Loader. FreeLDR will then use the RAM disk to search for freeldr.ini, present the OS options, etc, and finally load the kernel.

Therefore, there are three components that the system needs: the LLB, the loader, and the RAM disk. QEMU has command-line options and support for only loading two image files, and so the LLB and the loader must be fused together into one binary file. Additionally, the RAM disk image file you created must be slightly modified for technical reasons beyond the scope of this document.

Thankfully, a tool exists to perform these operations: nandflash. You simply need to run nandflash (from your output-arm/tools/nandflash directory) with the "1" option to instruct it to generate the necessary files. The other use of nandflash (without the parameter) is to generate OMAP3 ROM NAND flash files, but that usage is beyond the scope of this document.

Note that nandflash expects your image file to have been called ramdisk.img, so make sure you rename it to match this. After running nandflash, you will have two new files: ramdisk.bin, which is a modified version of your ramdisk image, and reactos.bin, which is the fused LLB and loader together.

Booting

You can now instruct QEMU to boot ReactOS for ARM. You will need to obtain the ARM version of QEMU (qemu-system-arm) or build it yourself. Currently, testing is done with the "Versatile/PB" platform, but with a modified CPU target being the Cortex A8, an ARMv7-a core, which is being used internally for BeagleBoard and ZoomII-MDK work. Although the original ARM port of ReactOS was designed for ARMv5 and ARMv4, these cores have an MMU that is incompatible with ReactOS/Windows requirements, and lacks instructions needed for synchronization done in user-mode. To specify these two options, use "-M versatilepb -cpu cortex-a8" on your command line.

You now need to specify the two files that nandflash created for you. You may do this with the following two options: "-kernel reactos.bin -initrd ramdisk.bin".

Finally, you need to tell the LLB that we are booting from a RAM disk, and you need to input the boot sector offset we talked about earlier. You can do this with the following options: "-append boot-device=RAMDISK,rdoffset=0x200". In general, "-append" is used to send command-line parameters to the LLB, and you can specify additional options by comma-separating them.

Here is an example final command line for QEMU: "qemu-system-arm -M versatilepb -cpu cortex-a8 -kernel reactos.bin -initrd ramdisk.bin -append boot-device=RAMDISK,rdoffset=0x200"

Past this point, the experience should be familiar. Note that the only thing you can do for now is select the OS from the boot selection menu, and see it load a couple of drivers and hang somewhere in the kernel.

Status

http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/mm/ARM3/?view=log

http://svn.reactos.org/svn/reactos/trunk/reactos/hal/halarm/?view=log

http://svn.reactos.org/svn/reactos/trunk/reactos/boot/armllb/?view=log

http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/config/arm/?view=log

http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/ps/arm/?view=log

http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/rtl/arm/?view=log

http://svn.reactos.org/svn/reactos/trunk/reactos/boot/freeldr/freeldr/arch/arm/?view=log

http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/include/internal/arm/?view=log

http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/ke/arm/?view=log

http://svn.reactos.org/svn/reactos/trunk/reactos/hal/halarm/?view=log

http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/mm/ARM3/?view=log

http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/mm/arm/?view=log

r56035: Two Part Patch which fixes ARM3 Section Support (not yet enabled). This had been enabled in the past for testing and resulted in bizare crashes during testing. The amount of fixing required should reveal why:

Part 1: Page Fault Path Fixes

  • [NTOS]: As an optimization, someone seems to have had changed the MiResolveDemandZeroFault prototype not to require a PTE, and to instead take a protection mask directly. While clever, this broke support for ARM3 sections, because the code was now assuming that the protection of the PTE for the input address should be used -- while in NT Sections we instead use what are called ProtoType PTEs. This was very annoying to debug, but since the cause has been fixed, I've reverted back to the old convention in which the PTE is passed-in, and this can be a different PTE than the PTE for the address, as it should be.
  • [NTOS]: Due to the reverting of the original path, another optimization, in which MiResolveDemandZeroFault was being called directly instead of going through MiDispatchFault and writing an invalid demand-zero PDE has also been removed. PDE faults are now going through the correct, expected path.
  • [NTOS]: MiResolveDemandZeroFault was always creating Kernel PTEs. It should create User PTEs when necessary.
  • [NTOS]: MiDeletePte was assuming any prototype PTE is a forked PTE. Forked PTEs only happen when the addresses in the PTE don't match, so check for that too.

Part 2: ARM3 Section Object Fixes

  • [NTOS]: Fix issue when trying to make both ROS_SECTION_OBJECTs and NT's SECTION co-exist. We relied on the *caller* knowing what kind of section this is, and that can't be a good idea. Now, when the caller requests an ARM3 section vs a ROS section, we use a marker to detect what kind of section this is for later APIs.
  • [NTOS]: For section VADs, we were storing the ReactOS MEMORY_AREA in the ControlArea... however, the mappings of one individual section object share a single control area, even though they have multiple MEMORY_AREAs (one for each mapping). As such, we overwrote the MEMORY_AREA continously, and at free-time, double or triple-freed the same memory area.
  • [NTOS]: Moved the MEMORY_AREA to the "Banked" field of the long VAD, instead of the ControlArea. Allocate MMVAD_LONGs for ARM3 sections for now, to support this. Also, after deleting the MEMORY_AREA while parsing VADs, we now use a special marker to detect double-frees, and we also use a special marker to make sure we have a Long VAD as expected.