furumi-ng/ARCHITECTURE.md

Furumi-ng: Distributed Virtual File System

Project Overview

This document describes the architectural vision and technical requirements for a modern, high-performance distributed virtual file system built in Rust. The system allows a remote client to connect to a server with authorization, list directories recursively, and mount the remote directory as a local virtual file system. It is designed as a faster, more reliable, and modern alternative to WebDAV.

Core Requirements

1. Cross-Platform Support: Initial support for Linux and macOS. The backend server and client mounting logic must be logically separated so that new OS support (e.g., Windows) can be added solely by writing a new mount layer without altering the core backend or client networking code.
2. Read-Only First: The initial implementation will support read-only operations, with an architecture designed to easily accommodate write operations in subsequent phases.
3. Memory Safety & Reliability: The entire stack (server, shared client core, and mount layers) will be implemented in Rust to leverage its strict compiler guarantees, memory safety, and high-performance asynchronous ecosystem.

High-Level Architecture

The architecture is divided into three main components: Server, Shared Client Core, and OS-Specific Mount Layers.

1. Transport & Protocol (gRPC)

• Protocol: gRPC over HTTP/2.
• Why gRPC: Provides strong typing via Protobuf, multiplexing, and robust streaming capabilities which are essential for transferring large file chunks efficiently.
• Security: Requires TLS (e.g., mTLS or JWT via metadata headers) to secure data in transit.

2. Server (Linux Backend)

The server role is to expose local directories to authorized clients safely and asynchronously.

• Runtime: tokio for non-blocking I/O.
• Security Validation: Strict path sanitization (protection against Path Traversal). The server restricts clients strictly to their allowed document root.
• VFS Abstraction: Backend logic will be abstracted behind a Rust trait. This allows future swapping of the storage backend (e.g., Local Disk -> AWS S3, or In-Memory for testing) without changing the gRPC transport layer.

3. Client Architecture

To maximize code reuse and maintainability, the client is split into two layers:

A. Shared Client Core (Cross-Platform)

A Rust library containing all OS-agnostic logic:

• Network Client: Handles gRPC connections, request retries, backoff strategies, and error handling.
• VFS Cache: An in-memory cache for metadata (TTL-based) to dramatically reduce network latency for high-frequency stat / getattr calls generated by file managers or terminals.
• VFS Translator: Maps VFS operations into remote gRPC RPC calls.

B. OS-Specific Mount Layer

Thin executable wrappers that consume the Shared Client Core and handle OS integration:

• Linux: Uses the fuser crate (binds to libfuse) to mount and handle events from /dev/fuse.
• macOS: Acts as a lightweight local NFSv3/v4 server (nfssrv or similar crate). The system natively mounts localhost:/ via the built-in NFS client, avoiding any need for third-party kernel extensions (like macFUSE) or complex FileProvider bindings.
• Windows (Future): Will wrap libraries like WinFSP or dokany to integrate with the Windows internal VFS.

API Design (Read-Only Foundation)

The initial Protobuf specification will involve core remote procedure calls (RPCs) to support read-only mode:

syntax = "proto3";
package virtualfs;

message PathRequest {
  string path = 1;
}

message AttrResponse {
  uint64 size = 1;
  uint32 mode = 2; // Permissions and file type
  uint64 mtime = 3; // Modification time
  // ... other standard stat attributes
}

message DirEntry {
  string name = 1;
  uint32 type = 2; // File or Directory
}

message ReadRequest {
  string path = 1;
  uint64 offset = 2;
  uint32 size = 3;
}

message FileChunk {
  bytes data = 1;
}

service RemoteFileSystem {
  // Get file or directory attributes (size, permissions, timestamps). Maps to stat/getattr.
  rpc GetAttr (PathRequest) returns (AttrResponse);
  
  // List directory contents. Uses Server Streaming to handle massively large directories efficiently.
  rpc ReadDir (PathRequest) returns (stream DirEntry);
  
  // Read chunks of a file. Uses Server Streaming for efficient chunk delivery based on offset/size.
  rpc ReadFile (ReadRequest) returns (stream FileChunk);
}

Future Expansion: Write Operations

The design ensures seamless expansion to a read-write file system. Future RPCs such as CreateFile, MkDir, Remove, Rename, and WriteChunk (utilizing Client Streaming or Bi-directional Streaming in gRPC) can be added without restructuring the foundational architecture.