Personally, I don't like the way PSX is implemented at present in Windows (SFU 3.5). The main problem is mixing Windows and Unix environments in a non-clean way. You never know where you are. Also security is a bad mix. PSX should be IMHO a pure Unix/Linux implementation under the ROS executive and use /etc/passwd for simple unsecure logins and a PAM module to autenticating with LSA. It should be possible to make PSX the dominant subsystem, if Win32 is not needed at all. -ea

We've talked about this at the LinuxWorld Expo. For the benefit of everyone else: the idea behind ReactOS POSIX is that any POSIX-like, or even any command line application should run under it. The main design goal is to make the likes of UnxUtils a thing of the past. So you won't have to ask yourself whether you're running the "real thing" or the slightly incompatible (but better integrated) Win32 port - the problem you describe won't exist, if the design goal is met. Indeed, the concept could be taken to a further extreme, by only using Win32 for the GUI and relying on the much better POSIX API for anything else

What needs to be done to implement POSIX and UNIX features (braindump):

General principles

Developer usability: should be super-easy to port code, reasonably mix Win32 and POSIX code, reuse existing tools (compilers, debuggers, memory checkers, etc.), etc. (the "Microsoft lesson")
Minimum effort: if Windows doesn't support a feature at all, POSIX-on-Windows shouldn't either (compare: Cygwin)
Modular and lightweight: it should be possible to use as many parts of it as possible as libraries in Win32 code with minimum overhead

fork

First, kernel support in NtCreateFile and the Mm. User-mode support a lot harder, if we want to support forking of Win32 processes. Speaking of the very basic infrastructure:

Ldr: needs to notify DLLs before and after fork (see pthread_atfork) - new messages for DllMain (DLL_PRE_FORK, DLL_POST_FORK), new function to register fork-aware DLLs (LdrEnableForkLibraryCalls) and fork callbacks (LdrRegisterForkCallback). Needs to set the new cached CLIENT_ID in the TEB
Rtl: needs to convert CRITICAL_SECTIONs so they are usable in the fork child (cfr. POSIX mutexes). Would be nice to have a RtlForkUserProcess function to nicely wrap the annoying sequence of system calls
Dbg: needs to raise image load events for the DLLs already loaded, or the debugger could get confused
Io: needs to "dup" files instead of merely duplicating the handles - i.e. create new file objects with the same FCB and a new CCB - or the two processes will needlessly serialize for synchronous I/O and, much more seriously, share the same file position. *VERY HARD* - all current filesystems need to be rewritten/wrapped. Possibly required a major redesign of the I/O subsystem, and extensive testing

Speaking of Win32, the hardest parts are:

USER: duplicate all windows of the caller thread? don't? fail? (how does X11 handle it?)
RPCRT: duplicate client connections? close them? (the server knows nothing about forking, so they can't be really "forked") (how does Sun RPC handle it?)

NOTE: Cygwin doesn't fork Win32 DLLs, for good reasons...

exec

copies all string parameters (command line, etc.) and pertinent fields of the PPB (stdin, stdout, stderr, paths, etc.) to kernel mode
opens the executable. Failure beyond this point terminates the process with an unhandled exception (NtRaiseException with an ExceptionRecord and Context allocated from user-mode memory, ExceptionRecord->ExceptionCode set to the failure status and SearchFrames set to FALSE), so that an attached debugger will be notified
closes all non-inheritable handles
frees all private virtual memory
unmaps all data section views (there's an odd section view at 0x7FFB0000, always mapped, even before any threads are created, which *is* unusual - what should we do with this?)
unmaps all image section views
reallocates and reinitializes the PEB and the PPB
remaps ntdll
maps the new executable
??? maps the loader ??? (do we do this in the kernel side of exec or in the initialization code? how do other systems implement it?)
creates the initial thread, copying the parameters to its stack
runs the initial thread

[... more?]

signals

The logic is very portable and self-contained, and could be taken almost as-is from any free POSIX system, like FreeBSD. Implementation on Windows/ReactOS should be like this:

thread switching through special kernel-mode APCs (successfully experimented, very low latency) or suspend/get context/set context (experimented by other projects, requires many system calls but allows to implement the kernel as an user-mode process)
calling signal handlers through trap frame manipulation and a callback modelled after KiUserApcDispatcher (successfully experimented)
no interference with user-mode APCs - programs should be able to mix signals and APCs freely
only interrupt interruptable waits with an alert test - *don't* add hooks in the dispatcher (some POSIX systems have non-interruptable waits too)
restartable system calls a feature of the system call stub (e.g. loop on ERESTART/carry flag)

Open issues:

interaction between SEH and signals:
- block all signals when calling Win32 code? (Win32 almost never considers reentrance)
- reset signals to the default action when calling Win32 code? (POSIX signals would "steal" SEH exceptions)
- invoke signal handler as a last resort?
- Tru64 had SEH: how did it handle this?
- how does Cygwin handle it?
some signals pull in a lot of infrastructure - need to implement sessions, process groups, terminals, etc. for some signals

Filesystems and I/O

Locking

Windows file locking is mandatory (POSIX locking is advisory) and slightly incompatible. Easily solved: since POSIX locking is advisory, it will only be checked for explicitely

Asyncronous I/O

Needs kernel-mode support (must be able to cancel single requests, not possible with NtCancelIoFile). All POSIX files need to be opened internally for asyncronous I/O (Windows can't convert files opened for syncronous I/O into files opened for asyncronous I/O - any I/O on them will always be serialized), because any file descriptor is eligible for asyncronous I/O

sockets

Need a dedicated filesystem (i.e. not AFD)

pipes

See above. Raise SIGPIPE in Win32 processes? would be useful (forces POSIX pipeline behavior on Win32 commands when running in a POSIX shell)

/dev

See above

VFS

Not really needed, but makes many things much easier, especially pathname resolution when using chroot (chroot isn't standard but widely supported on existing systems). Ideas:

single device, called \Device\VFS and representing the caller's root directory
doesn't support direct opens. \Device\VFS can only be opened to query and set information about the device (like the security descriptor)
a reference to \Device\VFS\POSIX$ is kept open globally. POSIX open() directly performs an IoCreateFile relative to this file. The path (relative or not) is passed verbatim - no conversion to Unicode, normalization, etc. - not to lose any information
\Device\VFS\DOS$ used as a base for DOS-style paths into the VFS tree. Only used to translate POSIX paths into DOS paths. Paths passed this way are always considered absolute

Open issues:

should be entirely self-contained (makes an user-mode implementation possible) or should act like a filter device? the former may be more efficient, the latter neatly integrates with the whole system and existing tools

open issues

paths

What's the best way to use Win32 paths in POSIX applications? ideas:

translate //<device>/<path> into \\.\<device>\<path> (with a special case for drive letters - //<letter>/<path> => <letter>:\<path> - and the special escape //./<raw text> => \\.\<raw text>. UNC paths can be specified with //unc/<path>). // paths are reserved by the standard for implementation-specific behavior, and the //<letter>/ syntax to escape Win32 paths is widely used in existing POSIX compatibility environments
euristics to recognize "bare" Win32 paths as such
case-insensitive lookup for Win32 paths and // paths (does the standard allow this kind of implementation-specific behavior for // paths?). Make it configurable both in the environment and programmatically. Use a preloaded module to impose policies based on more or less complex euristics (EXE identity, command line, etc.)
how to translate characters in POSIX paths that are never valid in Win32 paths (control characters, \, ", <, etc.)? idea: map them to codepoints in the Unicode user-defined range
/ paths point to the VFS root (how to handle chroot on // paths?) and have full POSIX semantics: best of both worlds. / paths are translated into Win32 paths like this: /<path> => \\.\VFS\DOS$\<converted path>
relative paths have semantics that depend on the current directory
ability to mount Win32 paths under the VFS tree, to impose POSIX semantics on certain paths

STREAMS devices

Need to look into this a bit deeper...

Virtual memory

mmap: needs to map at page boundaries (i.e. can't use NtAllocateVirtualMemory, must use NtMapViewOfSection with AT_ROUND_TO_PAGE). Needs to either reserve a large enough range (mmap heap), or to track pages to avoid unmapping private virtual memory not allocated with mmap, or to merrily unmap said private virtual memory if allocated a page at a time
brk/sbrk not in the standard anymore, but needed for backwards compatibility

Threading and real-time

many POSIX threads implementations for Windows - need to have a look
can we map POSIX real-time features to Windows real-time features?

Security

Mapping uids/gids to SIDs

Can either use S-1-N-<uid/gid> (some recent systems allocate uids/gids in parallel) or S-1-X-<uid>/S-1-Y-<gid> (with uid 0 mapping to S-1-5-18 and gid 0 to S-1-5-32-544), or more sophisticated approaches (table-based or interoperating with the native Windows security)

Mapping masks to ACLs

Pretty easy - every mask maps to exactly three ACEs

Mapping ACLs to ACLs

Could be impossible - no easy way to map POSIX mask ACEs to Windows ACEs

uid/gid policies

... such as those implemented by kill. Should be limited by Windows security as well (which one to check first? POSIX policies or Windows security?)

Open issues

Full implementation of setuid/setgid functions and the setuid/setgid file attribute nearly impossible

More to come...

[KJK::Hyperion]

PSX

Contents