It's been a long time since I edited this page. Here's where I am:
What needs work
Paste From discussion with Arty in #ReactOS
1) fixing irp cancellation would make the situation much better also, afd could use refactoring in select in drivers/network/afd/afd/main.c, sort out DestroySocket DestroySocket is ungraceful because we could have a mid-flight receive that needs to be cancelled and completed by tcpip.sys we should wait for that, rather than pending and shutdown that way removing this pseudo-refcount code from afd that would allow the completion code in select and read to be simplified
2) drivers/network/tcpip/tcpip/dispatch.c <-- here's what's needed: I put in IRPFinish but it's not that great after all get rid of it TCP itself doesn't need recmutex if the winding of different tasks can be sorted out so do that gracefull in tcp.c remove recmutex from tcpip make cancellation local by allowing a tcp request to exist without a backing file object when the file object has been closed use this fact to withdraw and complete cancelled reads immediately make the tcp worker hard close cancel-closed sockets internally
3) blocking recv on the packet path doesn't cancel properly either but it's easier to fix the main offender in the packet case is DGRemoveIRP in lib/drivers/ip/transport/datagram/datagram.c probably the first one to hit is tcp.c (lib/drivers/ip/transport/tcp/event.c) since fixing it won't break anything else specifically, TCPSleep needs to be implemented by passing a baton between worker threads there's still some of casper's refcount detritus in a few places, also remove that
Yet Newer News
We're basically good on functionality at this point, plus or minus some details. The biggest problem right now is stability around socket close time. I've embarked on reading some stuff and doing some thinking about IRP cancellation and completion.
Well I'm making slow progress on tcpip.sys. I feel more confident with it than before after cleaning up and clearing up many small allocations and much misuse of PC(Packet)->DLComplete and kin. Now each function which sends a packet takes a callback and context from the parent, and stores it someplace guaranteed not to overlap somebody else's callback.
There's no swapping of callback info into ndis packet context area any longer. It's used by datalink/lan.c to call the neighbor, and arp back *only*.
There were a ton of ways a packet could get out to the adapter. Some were in neighbor, arp, icmp, datagram, transmit. I changed this:
- arp and neighbor may send a packet on the wire using the interface transmit function.
- Everyone else uses IPSendDatagram, which uses the same code path for all datagrams.
- Neighbor takes a small hit and uses the same code path for every datagram as well.
After doing that, I have not seen any bugchecks on output, and I'm confident that any remaining errors can be spotted.
I will do more. Currently, the system stops either on a deadlock that i haven't nailed down yet or on an exc14 from ne2000, WordMode receive. I'm not sure what the bugcheck is about but I need to talk to tamlin since he seemed to think it was a qemu (and not real hardware) phenomenon.
oskittcp is now available as a separate userland library, and has an application in apps/tests/oskittcp that uses it. Our goal is to make it work right in all cases. Royce Mitchell has volunteered to work on it in userland some. How to set up to have it communicate with the special qudp is in here: Oskit TCP QUDP Setup.
TCPIP.SYS is coming along. I've separated out LAN.SYS (not checked in yet) which encapsulates access to the ndis adapters of all flavors. It also isolates TCPIP.SYS from having to know very much about media types and other things that are a pain in the semprini. I'm rewriting lan.c and arp.c in tcpip to take advantage of it.
Next task: separate the files in network into one or two userland libraries (route.a and ip.a) which will also be linkable in userland apps. This will give us almost everything that can be separated out. We will just need to finish worrying about io completion, double free, and other sorts of bugs in tcpip which may not be sorted out yet.
I rewrite AFD.SYS in the intervening time. The rewrite is very simple. No timers, no callbacks. Everything is either a TDI request or an io completion. Almost the whole thing runs at PASSIVE_LEVEL.
I've started working with oskit's TCP part, integrating it into our tcpip.sys. This will finally enable TCP, without us having to finish it ourselves. Primarily, I am focused on getting addresses, packets and information into and out of TCP. As I go, I am finding and fixing lots of bugs in tcpip, afd and ndis. As these are fixed, we'll get better results.
My ultimate goal is to be able to update the reactos sources from svn on reactos, as well as use lynx, curl and wget.
Stuff to do with oskittcp and reactos (stream of consciousness):
There is a bit of confusion regarding what the Header member of IPPacket points to. In one, and as far as I can tell, only one instance, it's used to refer to a link layer header. I'd like to avoid that somehow. Ultimately, I'd like to remove all instances of link layer header from tcpip, but need to think before leaping. It works now, in that packets make their way to oskit and are recognized and handled.
I need the information in the struct socket and CONNECTION_ENDPOINT to match. These are really the same data structure. I'm going to migrate out all of the in_pcb functions to be handled by reactos, at least.
Last night, I got the first RST packet after a SYN,SYN/ACK. The RST was because the connection wasn't found in the in_pcb hash table. I don't like that hash table very much. At the very least, I can make reactos search for a CONNECTION_ENDPOINT, then return a fake inp, or alternately, store the inp in the CONNECTION_ENDPOINT.
I've been doing lots of work trying to isolate functionality in tcpip.sys:
- Addresses and functions that work on them.
- Info -> IOCTL_TCP_SET_INFORMATION_EX
- Verifying more of ndis.
- Isolating things that should be libraries to tcpip.sys (list, cache management)
- Building and dispensing ndis packets and buffers library.