Virtual File System Sample in .NET, C#

In this article

Virtual File System Sample in .NET, C#

This sample implements a virtual file system with synchronization support, on-demand loading, selective offline files support, upload and download progress, and error reporting. It synchronizes files and folders both from remote storage to the user file system and from the user file system to remote storage. It is written in .NET Core, C#.  

To simulate the remote storage, this sample is using a folder in the local file system on the same machine. 

The purpose of this sample is to demonstrate the major features of the IT Hit User File System for .NET and provide patterns for its programming. You will use this sample as a starting point for creating a OneDrive-like file system for your DMS/CRM/ERP and will reprogram it to publish data from your real storage instead of the local file system. This sample provides a bare minimum code and functionality for demo purposes. For advanced features, such as thumbnails support, Microsoft Office editing support, automatic locking support and custom properties support see the Virtual Drive sample.

You can download this sample and a trial license in the product download area. You can also clone it and browse the code on GitHub

Requirements

  • .NET 3.1 or later.
  • Microsoft Windows 10 Creators Update or later version.
  • NTFS file system.

Configuring the Sample

By default, the sample will use the \RemoteStorage\ folder, located under the project root, to simulate the remote storage file structure. It will mount the user file system under the %USERPROFILE%\VFS\ folder (typically C:\Users\<username>\VFS\).

To specify the folder that will be used for remote storage simulation edit the "RemoteStorageRootPath" parameter in appsettings.json. This could be either an absolute path or a path relative to the application root.

To specify the user file system folder edit the "UserFileSystemRootPath" parameter in appsettings.json.

Setting the License

To run the example, you will need a valid IT Hit User File System Engine for .NET License. You can download the license in the product download area. Note that the Engine is fully functional with a trial license and does not have any limitations. The trial license is valid for one month and the engine will stop working after this. You can check the expiration date inside the license file. Download the license file and specify it's content in License field in appsettings.json file.

You can also run the sample without explicitly specifying a license for 5 days. In this case, the Engine will automatically request the trial license from the IT Hit website https://www.userfilesystem.com. Make sure it is accessible via firewalls if any. After 5 days the Engine will stop working. To extend the trial period you will need to download a license in a product download area and specify it in appsettings.json

Running the Sample

To run the sample open the project in Visual Studio and run the project in a debug mode. In the debug mode this sample provides additional support for the development and testing convenience. When starting in the debug mode, it will automatically create a folder in which the virtual file system will reside, register the user file system with the platform and then open two instances of Windows File Manager, one of which will show a user file system and another a folder simulating remote storage:

 Virtual File System Sample Initial Run

You can start managing and editing files in the \RemoteStorage\ folder immediately after the Windows File Managers open and will see all changes being propagated to the user file system. You can also edit documents and manage file structure in the user file system and all changes will automatically appear in the remote storage folder. Note that this sample does NOT support Microsoft Office document editing, for the Microsoft Office document editing and other advanced features see the Virtual Drive sample.

This sample implements on-demand loading. After running the sample all files and folders are marked with a cloud icon Offline File, which means that the content of this file or folder is not available locally but instead resides in the remote location. Even though the file shows the correct size in the Size column, the file occupies zero bytes on the disk. You can open the file Properties dialog to check the "Size on disk" value. You can see in the console log that only root folder files and folder placeholders are being created when Windows File Manager listed the root folder content. Folders located deeper in the hierarchy are not loaded until their content is being requested by the platform file system calls. 

When any application is accessing the file, located under the \VFS\ folder, opening it for reading or writing, the operating system redirects the call to this sample, which loads file content from the remote storage folder. The file becomes marked with a green check-mark on a white background iconLocal File, which means the file content is present on the local disk:

Local Cloud File. Green check-mark on a white background icon means the file content is present on the local disk.

The Windows File Manager provides the "Always keep on this device" and "Free up space" context menus, which are standard menus provided by Windows OS. If you select the "Always keep on this device" the file or entire folder structure will be recursively loaded to the local disk and files content will be loaded to the local disk and will become available offline. All files and folders are marked with a pinned file iconPinned File. Pinned files will not be deleted from the drive even if it runs low on space.

Always keep on this device menu will load all files to a loacal disk and will keep them from purging on low disk space.

To remove content from the local disk select the "Free up space" menu. It will restore the cloud icon Offline File.

When a large file is being downloaded from the remote storage, this sample submits progress reports to the operation system, to show a standard Windows "Downloading" dialog. At the same time, the Windows File Manager also shows progress in the files list view:

Large file download from the remote storage show progress reports in Windows File Manager and Files View

Stopping the Sample

You can exit the sample running in the debug mode in 3 ways:

  • If you press 'Q' the user file system will be unregistered and all files will be deleted. This simulates uninstall with complete cleanup. 
  • If you press 'q' the user file system will be unregistered. All files and folders downloaded on disk will be converted into regular files and folders. The offline files and folders will disappear. This simulates your application's uninstallation process.
  • If you press any other key, the application will exit without unregistering the user file system. All files and folders placeholders, their attributes, and states remain in the user file system, even though most of the functionality, such as downloading file content or listing folder content for offline folders are unavailable. You can start this sample again to continue managing documents. This simulates the machine reboot or application failure.

How the Sample Works

On-Demand Loading

Initially, when you start the application, the user file system does not contain any file of folder placeholders, except the sync root folder. The content of the folders is populated only when any application is listing folder content. The content of files is loaded only when an application is opening a file for reading or writing.

Platform File System Operations Handling

Folder content listing, file reads, move/rename and delete are performed by the VirtualEngine , VirtualFile and VirtuaFolder classes.  

The initial folder content listing is done inside VirtualFolder.GetChildrenAsync() method call. When any application is listing folder content, the VirtualEngine class creates the VirtualFolder class instance that corresponds to the requested folder inside the VirtualEngine.GetFileSystemItemAsync() factory method. Then the VirtualEngine class calls GetChildrenAsync() method on the returned folder item.

As soon as the folder content is populated with placeholders during the first access, the GetChildrenAsync() is never called again for this particular folder. Instead, to update the folder content, this sample monitors changes in the remote storage and applies them to the user file system. See the "Remote Storage to User File System Synchronization" section below.

Hydration/Dehydration via Windows File Manager Commands

When the user selects "Always keep on this device" or the "Free up space" in the Windows file manager, the file is marked with a Pinned or Unpinned attribute respectively. The Engine detects these flags triggering hydration or dehydration if needed. To hydrate the file the VirtualFile.ReadAsync() method is called.

Remote Storage to User File System Synchronization

The remote storage is simulated by the dedicated folder in the file system (\RemoteStorage\ by default). The sample monitors changes in it using RemoteStorageMonitor class that generates events and applies corresponding changes in the user file system. The RemoteStorageMonitor class is using a FileSystemWatcherQueued class which is similar to FileSystemWatcher which can monitor the creation, update of file content and attributes, deletion, and rename. But unlike the FileSystemWatcher, the FileSystemWatcherQueued class can process a large number of events and is tested on the highly loaded file system. Note that the FileSystemWatcherQueued class (as well as FileSystemWatcher class) can NOT monitor the move event. In your real-life application, you will replace the FileSystemWatcherQueued class with web sockets or any other technology that will send events from your server to all connected clients.

User File System to Remote Storage Synchronization

The user file system to remote storage synchronization is performed by VirtualFile and VirtualFolder classes. When a file or folder is created in the file system, the VirtualFolder.CreateFileAsync() and VirtualFolder.CreateFolderAsync() methods are called. When the file content is modified on the client the VirtualFile.WriteAsync() method is called. When a file or folder is moved or deleted the VfsFileSystemItem.MoveToAsync() and VfsFileSystemItem.DeleteAsync() methods are called.

Next Article:

Virtual File System Sample for Mac in .NET, C#