[ros-dev] RosFS

[Jasper van de Gronde]

Rick Langschultz wrote:

I'd first like to state I'm a big proponent of any advances in metadata 
handling and think it's a great idea to do something like this for ROS 
(and possibly other systems).

First off, what kind of data model are you thinking of? Something 
graph-like (like RDF), something more tree-like (no multiple-parents), 
something even flatter, or perhaps something entirely different. Also, 
what kind of (meta)data are you thinking of storing? Just "system" data 
(things like modified times), just "user" data (title/description) or 
just anything one would like to store?

> RosFS is a small service running on system initialization by default
> that indexes files, folders, users, devices, and network resources can
> hold metadata via MySQL, sleepycat DB, TCL, Apache, and Perl; which will
> all be embedded within the service itself.

Could you explain in more detail how you envision the structure of this 
service? For example, why MySQL AND Sleepycat DB? Is it meant to have a 
way to attach different database backends relatively easily? (perhaps 
not necessarily at runtime at first, but at least at compiletime)
Why TCL? Or is this just one possible scripting language you plan on 
supporting?
What storage format will be used? And what will applications use to get 
data from the storage?

> RosFS will hold encrypted
> resources based off of a key that is created with the first startup of
> ReactOS, it changes every 30 days to ensure security.

What do you mean by resources in this case? And why do they need to be 
encrypted? Is the encryption meant to be used as some form of access 
control?

> The RosFS can be
> accessed online via the built in web server with can be an encrypted
> connection - SSL, or TLS.

What kind of interface will the web server expose? Something like 
WebDAV? Multiple interfaces, depending on plugins of some sort?

> User ACLs can be set on a machine level or a
> domain level.

I assume you mean a domain as used with Windows networking.
What will the ACLs protect, the entire database or individual records? 
If they protect the entire database, what will be used to determine who 
is allowed to change/read/create/etc. values in the database.

> RosFS will support a SQL-like querying language that can
> be written easily - it will have triggers, procedures, functions, and
> saved queries - which are all logged.

Might I suggest writing a relatively flexible query engine which will 
allow you to use several query languages with it. One possible reason 
for this could be that you'd like to support both very compact query 
languages and very verbose query languages. Another reason could be that 
you'd want to change the query language later on or support a new great 
query language that doesn't exist yet (there are a lot of query 
languages for these kinds of purposes in development).

> Triggers will be supplied by a
> TCL-like language with features of Perl.  An service will provide users
> with current system updates, patches, etc stored on a domain server -
> this option will only be installed on a powerful server with pretty good
> hardware - ie: not for home use!!!

With "this" option I assume you're referring to the patch update stuff?

> A few problems I know that will arise in developing such a project are:
> 1) Size

Of what? The application itself or the data it stores? The application 
itself could indeed grow big, but it could be kept somewhat limited by 
modularizing it in such a way that not everyone would need all parts.

> 2) Security

That depends on what kind of accesses you allow, if you just allow 
access to the DB from the "local" computer and make sure everything is 
properly protected by ACLs or something like that than it should be 
reasonably safe.

> 3) Where to keep the files on the system

Martin Fuchs's suggestion of splitting it across drives seems like a 
reasonable suggestion. And I don't know what change logs are in NTFS, 
but if they're anything like transactions I'd recommend them.

> 4) The RosFS query language will be trouble to write and maintain
> depending on if a user uses such Databases like ORACLE, MySQL, MSSQL,
> etc.

That's another reason to make a clear distinction between the query 
parser and query engine, that way you'd be able to write the parse code 
once and then write different backends for different databases. You may 
want to have a look at what the Redland project did (I'm not convinced 
their implementation is ideal, but it shows a possible way to go):
http://librdf.org/

> 5) Explorer Integration - Will Explorer.exe need to be rewritten to
> allow the addition of Metadata structures, etc.

That depends on what you want to do (and what the current ROS explorer 
supports, as I have no idea what it does support at the moment). The MS 
Windows explorer supports quite a few extensions (especially in WXP) 
meant for handling metadata (a simple example is a column handler, which 
can add columns to explorer's details view).

> 6) Web integration - Perl, PHP, TCL, CGI, ISAPI???

If you mean language bindings, sure, but I'd first get the basic system 
more or less working before even thinking of something like this.

I'd also like to mention a project I'm working on, MDDB, which at least 
has some similarities to what you're trying to do. It also has some 
important differences (it has a clear distinction between a file and a 
document for example, which is probably not entirely what you had in 
mind). The current site is available at:
http://home.hccnet.nl/th.v.d.gronde/dev/mddb/

In general you may want to look at some RDF based projects out there, 
there are some great things being done with RDF at the moment and if 
nothing else it might at least give you some ideas.