File vs block caching

Here you can discuss ReactOS related topics.

Moderator: Moderator Team

Post Reply
tverweij
Posts: 39
Joined: Wed Jun 18, 2008 7:17 pm
Contact:

File vs block caching

Post by tverweij »

Windows has a File cache - works reasonable.
But I am now working with a block level cache in Windows. My old SATA disks suddenly perform like they are SSD's.

So my tip to make ReactOS better than Windows: implement a block level cache instead of a file cache.

ThFabba
Developer
Posts: 291
Joined: Sun Jul 11, 2010 11:39 am

Re: File vs block caching

Post by ThFabba »

That's pretty much impossible. The cache controller is an integral part of the kernel and drivers expect it to work a certain way.

Also, your evidence is anecdotal. If you want your suggestion to be taken seriously, you would at least have to explain why you think one is better than the other and/or show some benchmarks. Ideally a scientific paper would agree with your conclusion.

tverweij
Posts: 39
Joined: Wed Jun 18, 2008 7:17 pm
Contact:

Re: File vs block caching

Post by tverweij »

Impossible? I would say : try it.
There are several products available. You only need 4Gb+ Free memory for it (I am running it on Windows 2016 HyperV production servers (on the host), one with 8Gb cache (4 VM's), one with 32 Gb cache (10 VM's)).
Edit: And for "Scientific" evidence: This is the way every cached raid-controller works.

ThFabba
Developer
Posts: 291
Joined: Sun Jul 11, 2010 11:39 am

Re: File vs block caching

Post by ThFabba »

Not impossible as in "third-party software can't do something about it". More like, impossible as in "you can't modify the internal workings of the operating system at that level without breaking stuff".

But of course, whatever software you use to achieve it on Windows should ideally also work on ROS. For that matter, such software is extremely unlikely to replace Windows's caching, it most likely supplements it -- meaning you're comparing file level caching against file AND block level caching. Of course the latter will be faster.

Rosh
Posts: 97
Joined: Thu May 03, 2018 12:44 am

Re: File vs block caching

Post by Rosh »

Do a JIRA request (assuming it is possible and logical). ;)

User avatar
mrmajik45
Posts: 344
Joined: Sun Sep 24, 2017 1:21 am
Location: USA, Indiana
Contact:

Re: File vs block caching

Post by mrmajik45 »

All I expect to happen if you added Block level cacheing: Bsod that Bsod's that Bsod's that Bsod's (Etc.)
ReactOS Donator ~ $5.00 | Linux Mint Donator ~ $1.00 in BTC

tverweij
Posts: 39
Joined: Wed Jun 18, 2008 7:17 pm
Contact:

Re: File vs block caching

Post by tverweij »

All I expect to happen if you added Block level cacheing: Bsod that Bsod's that Bsod's that Bsod's (Etc.)
Why do you expect that?
Based on what?
Not impossible as in "third-party software can't do something about it". More like, impossible as in "you can't modify the internal workings of the operating system at that level without breaking stuff".
It's not about changing the inner workings of the os. It's about adding a kernel driver.

User avatar
mrmajik45
Posts: 344
Joined: Sun Sep 24, 2017 1:21 am
Location: USA, Indiana
Contact:

Re: File vs block caching

Post by mrmajik45 »

You really think it would work like that? I know that there will always be atleast 1 problem
ReactOS Donator ~ $5.00 | Linux Mint Donator ~ $1.00 in BTC

anthracen
Posts: 43
Joined: Thu May 10, 2018 2:28 pm

Re: File vs block caching

Post by anthracen »

Block caching is much dumber than file caching. it was used in much older systems - Win95 I guess, old Unixes, VMS. It cannot bring any advantages over file caching. For example it's impossible to use a read-ahead technique, when cache manager could cache data that might have been requested later, based on working on file views rather than just disk blocks.
What you see is achieved by huge wasting of memory - your HDDs haven't been sped up, you just wasted a lot of RAM for dupicating disk content. It is in the File cache, but also that driver you mention, caches disk blocks as well.
This is not only very wasteful with respect to memory usage, but also can have a lot of troubles in form of data loss, since you cannot garantee that that driver is so smart to deal with proper flushing data back to disk. if power goes down, what programs and OS thought is on disk, might be still in memory and, as a result - disappear. It's definitely a cheap trickery, disrespecting the fine structure of the internal OS mechnisms (breaking them) and bringing "performance" goals by the brute force of enourmous wastage and risk of data loss and instabilities.

There is registry value LargeSystemCache under HKLM\CurrentControlSet\Control\Session Manager\Memory Management key, that, being set to 1, can direct OS to let more room for the system cache.
RAID contorllers work on blocks because they cannot work on files obviously. In no way this can solidify logical block caching (sector caching) over virtual block caching (file caching). the former is an outdated approach.

ThFabba
Developer
Posts: 291
Joined: Sun Jul 11, 2010 11:39 am

Re: File vs block caching

Post by ThFabba »

tverweij wrote:
Not impossible as in "third-party software can't do something about it". More like, impossible as in "you can't modify the internal workings of the operating system at that level without breaking stuff".
It's not about changing the inner workings of the os. It's about adding a kernel driver.
Absolutely. And if ROS works correctly that driver should work just as well as it does on Windows. However in your initial post you suggested we use block caching _instead of_ file caching, which is where I got the idea that you wanted to change OS internals.

Xbase
Posts: 32
Joined: Thu Apr 19, 2018 2:08 pm

Re: File vs block caching

Post by Xbase »

tverweij wrote:Windows has a File cache - works reasonable.
But I am now working with a block level cache in Windows. My old SATA disks suddenly perform like they are SSD's.
my Samsung SSD have a Software RAPID Mode and it boost Performance up to 10 X ... :o
as i'm working in Network and have e.g. this Situation.

1.) qwery Server database
2.) get result and create temporary local file
3.) work on local file
4.) update Server database from local files
5.) delete local files and start next with 1.)

it does work 99,9% in normal Mode but with RAPID Mode after 5.) i can still see files in Explorer even press F5 :(

so for share files in network external Cache often make more Problem than without :!:

---

as we talk about Cache i guess ROS does support SMB1 Opportunistic Locking :?:

ThFabba
Developer
Posts: 291
Joined: Sun Jul 11, 2010 11:39 am

Re: File vs block caching

Post by ThFabba »

ROS does not support SMB. Oplocks are implemented but I don't know if anyone's really tested them.

As for your "rapid mode"... it seems that feature is simply buggy then. If you add a cache and notice any difference in behavior other than speed, that means the cache is broken.

"There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors." -Leon Bambrick

Xbase
Posts: 32
Joined: Thu Apr 19, 2018 2:08 pm

Re: File vs block caching

Post by Xbase »

ThFabba wrote:ROS does not support SMB. Oplocks are implemented but I don't know if anyone's really tested them.
so how to share file based database like DBF files on ROS as Server :?:
ThFabba wrote:As for your "rapid mode"... it seems that feature is simply buggy then. If you add a cache and notice any difference in behavior other than speed, that means the cache is broken.
sorry forgot to say that it happens with OS > NT where we have SMB2 e.g. Win 10 1709/1803

Server have Cache and Client have Cache and SMB is to "communicate" between.
when flush Cache Command is send it work fine with OS Cache but it must pass to 3-PP Cache and that take time
To be more precise file-System
IRP_MJ_CREATE/IRP_MJ_SETINFROMATION/IRP_MJ_CLEANUP with a file-disposition DELETE=true leads to a an IRP_MJ_DIRECTORY_CONTROL querydirectory operation result of true for the DELETEd file.
This is different behaviour than in previous Windows Version
p.s. i also work with PostgreSQl and MySQL which both do not need SMB

ThFabba
Developer
Posts: 291
Joined: Sun Jul 11, 2010 11:39 am

Re: File vs block caching

Post by ThFabba »

Xbase wrote:
ThFabba wrote:ROS does not support SMB. Oplocks are implemented but I don't know if anyone's really tested them.
so how to share file based database like DBF files on ROS as Server :?:
Well... you can't.
Alpha and all...

Post Reply

Who is online

Users browsing this forum: Ahrefs [Bot] and 2 guests