[ros-dev] Re: proper nt-style authentication (reactos, wine,
davidjohnson.johnson at gmail.com
Fri Sep 2 04:34:20 CEST 2005
I love for this great idea and I put it to the ROS DEV.....
Please consider doing every bit of it.
On 9/1/05, Luke Kenneth Casson Leighton <lkcl at lkcl.net> wrote:
> [please note: due to its cross-discipline and cross-project nature,
> this message is going out to SEVERAL related project mailing lists.
> when replying, please take EXTRA caution not least because some of
> these lists are subscriber-only. also, please note: i _would_
> post this also to the samba mailing lists but due to the fascist
> censorship in place since the 15th dec 2004, i am unable to do so.
> this is their decision and it is their loss. i am not asking you
> to respect that decision i am simply making you aware of it.]
> out the woodwork i pop - not necessarily ready to chew anything because
> i know just how much work's involved, but what i did want to do was
> say "hi, i'm still here" and do a brain-dump of how authentication
> ideally needs to be implemented in reactos.
> at 2,600 words and 16k, this message is quite long and so i have
> placed a copy at http://lkcl.net/software/reports/reactos-auth.txt,
> just in case it doesn't make it past various mailing-list limits
> (i'll find out in a couple of minutes... :)
> it breaks down into a number of sections:
> 1) i describe the timescales and ways to cut them,
> along with some warnings and stuff.
> 2) amonst other stuff i outline some background as to
> why i am posting this to so many lists.
> 3) i outline a project plan and the dependencies
> and "optional" steps,
> 4) i describe a recommended implementation
> roadmap starting with the "minimum" requirements,
> and expand on some technical info and references
> i found, which would help with some of the
> "nice-to-have" steps.
> so. first.
> please do not be scared by how much work is involved, and how much code
> there is. it all hinges one ONE function and that one function... you
> are _not_ going to believe how much code that one function drags in,
> kicking and screaming, behind it.
> please also please i beg you DO NOT consider going "bleedin 'ellfire,
> that's so much frakkin work we can't POSSIBLY do it the way you
> describe, we MUST do it our own way, starting with what we know, love
> and have already started tackling, and can't possibly back out of what
> we've already done".
> should you choose to exercise the "NIH" option, i frakkin
> guarantee you that you will waste about five to eight years
> of your (collective) lives reinventing the wheel.
> "The Plan" outlined here will shave that down to about 12 to 18 months,
> utilising some SIX HUNDRED THOUSAND lines of pre-existing code, and
> later in this message i also outline a prioritisation of the necessary
> work to "cut down" the time to maybe about ... mmm... three or so
> months, by leaving out some of the "nice-to-have" stuff. actually...
> _all_ of the nice-to-have stuff :)
> that will at least "GetItWorking(tm)" and the rest of the bits
> can be considered at leisure once people go "god that's awful,
> we can't possibly leave it like that" and hopefully hopefully
> things will actually progress from there.
> remember - please: this stuff is sufficiently complicated such that
> you really can't afford the niceties such as "It Must Be Perfect (tm)"
> before it can be accepted. i've seen that shit before, and it's
> a baad mxxxxxxxer path to go down, especially with such complex
> and heavily interdependant reverse-engineering projects as reactos,
> wine, samba and freedce.
> fyi - before i begin, i should mention a couple of things:
> 1) this message also goes out the the apache devel mailing
> list (specifically the APR one) because the NT style
> authentication thing is the sort of thing that really _should_
> be in a (special?) APR util library, along with "NT Named Pipes" -
> see http://en.wikipedia.org/wiki/NamedPipes
> one of the reasons why NT-style NamedPipes is _not_ in an APR util is
> because it is believed that NT-style NamedPipes do not fit the
> "least common denominator". by having the infrastructure outlined
> below, it is possible to move things "up one level" to the point
> where unix _has_ that denominononmination.
> also, if APR still has support for that god-awful program-running
> program called Win95, it would, with not much extra effort, be
> possible to port some reactos components to Win95 (!) such that
> ThePlan outlined here would make Win95 have proper NT-style
> authentication (!) now there's a horrible thought that will give
> MS and free software people nightmares both: upgrading Win95 by
> installing free software components at it. muhahahahah ahem.
> 2) i have filled in a number of pages on wikipedia.org. see
> http://en.wikipedia.org/wiki/User:lkcl. please take the time to
> review these pages, PLEASE help me with my totally biased "POV"
> (point-of-view) comments, by either editing the pages direct or by adding
> comments on the "talk" pages as appropriate. or alternatively dumping
> me in the nearest pond if ever you meet me.
> wikipedia is supposed to be encyclopedia-ic and some of my
> comments are anything but that.
> 3) please due to the quite broad distribution of this message across
> multiple mailing lists, please do NOT ask stupid questions like
> "what the hell do we need to use samba tng for, why can't we use samba
> in reactos"? and "why is this guy bothering us with this insane stuff?"
> and "how do i use killfiles?" or terry pratchett's - more specifically
> the librarian's favourite - "where has all the pie and custard gone?"
> please do your research: see if nothing else
> http://en.wikipedia.org/wiki/Samba_TNG_software and then start googling.
> roughly in order of dependence, with "nice-to-have" status added as
> 1) port FreeDCE to Win32 - more specifically add in autoconf support for an
> alternative threading model - the one in reactos. no, this is not a joke:
> MSRPC is a critical part of NT infrastructure it's just that very few
> are actually aware of this (including the US DoJ and the EU commission...)
> see http://www.google.co.uk/search?q=lkcl+wine+freedce for discussion
> links as to why - it's a long story and it took me several dozen
> messages spanning over many days to outline it in sufficient detail
> for it to be understood (mostly me getting things straight in my
> status: "nice to have" but you will soon wish that you _did_ have it!!!
> there is also an opportunity to support Wine, here: essentially
> it's the same job, for DCOM, for proper "authentication"
> purposes in Wine just like in ReactOS... it's a long story.
> 2) remove dependence on "unix" security model from Samba TNG's services
> starting with samrd (perhaps by finishing off samrtdbd?)
> status: "essential".
> a) in Samba TNG, rework winbindd such that it is a REQUIRED service,
> and it has "modes" that, instead of "inventing" links between
> unix uids and nt sids, it reads smbpasswd and correlates unix uids
> with nt sids _just_ like is done now - but hard-coded through
> the awful code that has had to dieee since... about two weeks after
> i wrote it, back in 1999.
> this one's a long story but it's not actually required for
> reactos but _might_ be required for wine...
> status: "pretty much essential" but not for reactos but for making life
> easy to share tng development between reactos and unix.
> b) in Samba TNG, make the authentication code (smb-side and rpc server
> back-end-wise) contact winbindd BUT ONLY for resolving unix uids and
> unix gids - which is actually a VERY small task involving a few
> hundred lines of code.
> this goes back to the "SURS" stuff i wrote up in 1999 - but instead
> of a library it hands off the responsibility for sid<->uid+gid
> lookups to winbindd.
> it _may_ also be essential for Wine to operate correctly
> with authentication (which is a "stub" at the moment just
> like it is in reactos), to interoperate with other programs,
> to cache login credentials in order to do smb client-side
> operations, that sort of thing, under which circumstances,
> it will be necessary for Wine to have the same sort of thing.
> it's no big deal :)
> status: same as for 2a).
> c) in Samba TNG, abandon all use of hard-coded MSRPC stuff and utilise
> FreeDCE instead, and do a rewrite of all services, one by one
> (and because samba tng is modular, this _can_ be done one-by-one,
> checking MSRPC interoperability along the way between the "old"
> services, the "new" services, and also with NT itself).
> in and of itself, this critically depends on making
> DCEThreads reliable, though, or adding support (finally!) for
> POSIX threads.
> status: "non-essential" but you would soon end up wishing that you had :)
> 3) complete LsaLogonUser, add LsaLogonUserEx, in NTOSKRNL.DLL, and
> my favourites. the LSASS stuff.
> these functions are quite simple: they are "redirectors" - a vector
> table of functions is required to be passed in, which includes things
> like "authenticate with my lovely service" and "free some memory".
> there are many LSASS sub-services: one of them is Netware, one of
> them is MSV1_0.DLL, one of them is Kerberos.
> there are others.
> LSASS is based on _exactly_ the same technique that dlopen does e.g
> in libdvdcss, and in freedce's ntlmsspauth.so, and in... mmm...
> the freedce transport module infrastructure and _many_ other
> projects ... except of course it's NT-based table of
> higher-order-functions not unix "dlopen()" ones.
> i'll stop trying to teach people to suck eggs, now *embarrassed* :)
> status: "essential".
> a) write an MSV1_0.DLL which registers with the LSASS service.
> this will utilise MSRPC functions that call into the Samba TNG
> "NETLOGON" service.
> there already exists "basic" functions inside Samba TNG that...
> well... it's a bit messy, but they work.
> the key function to be calling is cli_nt_login_interactive,
> and you will notice _very_ quickly from the arguments hey, some
> of those look... kinda... familiar!!
> status: "essential."
> the "porting" bit of this code to FreeDCE i would classify as
> "non-essential" but again, you will soon wish that you _were_ using
> that's basically it: there are other sub-projects such as
> turning the "Registry" code in ReactOS (or Wine) into an MSRPC
> service (yes, i did say pretty much everything that's anything
> critical in NT is an MSRPC service, didn't i? :) but they aren't
> i say basically it, but that ONE stupid function, cli_nt_login_interactive,
> drags in quite literally HALF A MILLION lines of code - 250,000 lines of
> it in FreeDCE, alone (which, like i said, could possibly be avoided but
> the hard-coded MSRPC stuff in samba tng is sufficiently awkward to make
> a rewrite utilising FreeDCE very attractive).
> what i would suggest is the following:
> 1) write an MSV1_0.DLL "test stub" in combination with completing the
> LSASS functions, which supports a username "test", domain of "test" and
> password of "test".
> code up a hard-coded "blob" - a NET_USER_INFO_3 structure - that contains
> the response / essential information of groups, gids, sids, user session
> key etc.
> (note: i _did_ say it would be nice to utilise FreeDCE didn't i? well,
> the NET_USER_INFO_3 structure in that header file is a "messy" version that
> i recreated from off-the-wire back in about ... mmmm... 1997. there
> _do_ exist IDL-generated versions of this data structure, thanks to
> matthew chapman - NETLOGON.idl for example).
> once that works...
> 2) write a lovely insecure method of "outsourcing" the username,
> domain and password to an external server - Samba TNG - which performs
> the authentication on your behalf and gets back "real" data.
> this could be done simply with a TCP connection, throw the data
> in-the-clear over to a simple temporary shim service blah blah,
> bob's your uncle.
> 3) port samba tng's netlogond, samrd and lsarpcd to ReactOS.
> this is quite straightforward: about the only really essential
> "missing" bit is to tie the services in to "NT Named Pipes" rather
> than unix domain sockets.
> _but_ - they only need to be plugged in to a back-end transport
> which all services - any of samba tng's MSRPC services - would
> use. so there's a key bit of work needed - probably under
> 400 lines of code - and the rest should fall into place.
> this is where it would be SO much easier to be utilising FreeDCE.
> all that's needed would be to write a FreeDCE "transport"
> plugin for ReactOS - ncacn_np - and then you'd be DONE.
> it's a long story...
> hey, has someone implemented TransactNamedPipe() and CreateNamedPipe()
> in ReactOS?
> if not, it's quite straightforward to do, and it involves dropping
> opaque data blobs onto smbd (again, smbd ported to reactos...)
> i _did_ say it's a long story, didn't i? :)
> 4) finally: track down how "LsaLogonUserEx" works. LsaLogonUser
> utilises cleartext passwords. LsaLogonUserEx i BELIEVE utilises NT and
> LM password hashes, or some brain-dead encrypted variant thereof.
> once 1) is completed, then TA-DAAAAA!!! lib/secur/lsa.c's "LsaLogonUser"
> function actually gets REAL DATA!
> 1) regarding FreeDCE. see http://en.wikipedia.org/wiki/FreeDCE
> freedce is an interoperable version of MSRPC that is derived from
> EXACTLY the same source code (DCE 1.0 reference implementation) that
> is present in Windows NT.
> DCE/RPC was co-opted / adopted / borg-ified by microsoft to form
> the basis of NT domains because paul leach, a co-founder of
> apollo computers and originator of NCS which became DCE/RPC,
> ended up working for microsoft back at the time when dave
> cutler was doing the original NT 3.1.
> using FreeDCE is non-essential for small projects of less than about
> 8,000 lines of code. Samba TNG's MSRPC code comprises OVER ONE HUNDRED
> THOUSAND lines of code.
> that i carried on hand-crafting MSRPC packets for three years simply
> demonstrates quite how bloody stupid i am.
> that you - the wine team - continue to reinvent an non-interoperable
> version of MSRPC, for binary-level "DCOM" interoperabiltiy ONLY,
> demonstrates quite how just as bloody stupid you are being. that _can_
> be taken as a compliment, as i genuinely i mean it with the greatest of
> 2) regarding a strategy to "minimise" the amount of time needed to
> "Get Things Working".
> a) utilise samba tng as-it-is, porting it as-is to ReactOS (mingw32)
> b) utilise cli_nt_login_interactive() as-it-is.
> c) complete MSV1_0.DLL.
> this will be SUFFICIENT and would only take a few months. enhancements
> come later, roughly in this order:
> a) complete a port of FreeDCE to ReactOS, by beating DCEthreads to
> death and replacing it with Win32 (NT) threads - #ifdef style,
> autoconf style.
> b) locate NETLOGON.idl, samr.idl and lsarpc.idl from matthew chapman's
> stuff, or from "todd sabin's" http://razor.bindview.com, or from
> the samba web site.
> actually... see: http://www.bindview.com/services/razor/utilities/
> compile up CLIENT-SIDE ONLY, using dceidl, and create a
> library for use inside MSV1_0.DLL.
> i know todd developed interoperable versions of these idl files,
> because he was using them, compiled with MIDL.EXE, to do security
> tests against NT 4.0. remember: these IDL files are what microsoft
> DOESN'T want anyone to have, and there are very good _legitimate_
> reasons for it: they are "behind-the-scenes" APIs which, if some
> idiot inside microsoft went and wrote a tool which became publicly
> used and relied on, they would be stuffed: they have ENOUGH apis
> which date back 15 years and they don't need any more. that
> having been said, i hate the fact that they are forcing people
> to pay $50k up-front and then $100 per-client for frakkin
> _information_. bastards.
> c) replace cli_nt_login_interactive() with a function that utilises
> the above library that utilises netlogon, samr and lsarpc client-side
> stuff from b) above.
> basically, what it boils down to is that the functions inside
> cli_nt_login_interactive i wrote BY HAND after examining MSRPC
> traffic. those functions neeeed too dieeee and they can easily
> be replaced one-by-one with the "proper" versions from doing
> "dceidl -client lsarpc.idl". it takes a couple of seconds and
> you get a header file lsarpc.h and a lsarpc.o and you're DONE.
> question. why did i spend so long doing hand-marshalling of the
> nt domains code. perhaps because i would not be able to give
> microsoft absolute hell for three frakkin years?
> d) consider porting, one-by-one, the Samba TNG services lsarpcd,
> netlogond and samrd (or better samrtdbd) to FreeDCE runtime
> this is NOT essential but it is very worthwhile. every project
> needs to do at least two throw-aways and with the samba tng
> code, there has been approximately one and a half throwaways
> so far... time for a major one, ESPECIALLY in light of the reactos
> e) consider writing that winregd service but actually picking up REAL
> registry hive files. someone did write a "reader" of registry files,
> i do not recall who it was.
> [note: YESSSS!!! yeeeeeehawww, todd i could KISS you!!]
> todd wrote a registry-hive-reader as a linux filesystem driver!!!!!
> todd, you are bloody mad. *smooch*.
> f) consider writing a samr service with the FreeDCE runtime
> that utilises "registry" functions RegXXXX - _properly_.
> anyway, if you got this far: congratulations and welcome to a brief
> nightmare history of the development behind NT Domains Authentication.
> all of the infrastructure above _already_ exists - in Samba TNG.
> it's just... it works. it's a bit creaky at the edges, but it works.
> how about it?
> p.s. it almost goes without saying, but i will hint at it once again:
> you will not find the samba team's goals and strategic direction to be
> compatible with reactos. as a team, they lack the insight, vision and
> project management skills to be able to cope with such inter-dependant
> insane gibberish. this i say with much sadness because they are highly
> respected individuals in the free software community, and yet they are
> taking the samba project in very different - and ultimately
> time-consuming - directions from what is really really needed. due
> to pride they cannot admit this in public but they have been known
> to admit it in private. much much disappointment and sadness and there's
> absolutely nothing i can do about it.
> <a href="http://lkcl.net">http://lkcl.net</a>
> Ros-dev mailing list
> Ros-dev at reactos.com
More information about the Ros-dev