FUSE - Filesystems in User Space

Linux has a long tradition of supporting multiple filesystem types (30+), to maintain compatability with legacy systems, and support new technologies without major changes to existing systems. Many of the filesystem types are not traditional disk-based hierarchies of files, but are views of networks, devices or memory presented as a hierarchy of virtual files (e.g. /proc). Filesystems may support only a subset of the available file and directory-related system calls. Supporting a new filesystem type tradtionally requires kernel support via a device driver, which is typically a difficult programming task and can lead to system errors or security problems if not written carefully.

1. FUSE Overview
   The FUSE device driver is a general purpose filesystem abstraction layer, which loads as a kernel module and presents a virtual device (/dev/fuse) to communicate with a user (non-kernel) program via a well defined API. The user code need not run with root priviledge if it does not need to access protected data or devices, and can implement a virtual filesystem much more simply than a traditional device driver. Because of this a large number of contributed filesystems are now available to run on top of FUSE. FUSE was originally developed to support AVFS (A Virtual File System).
   • Open Source developer community; Mac port by Google.
   • System requirements:
     • Linux 2.4 or later kernels
     • Mac OSX 10.4 or later
   • Licensing: GPL (free download)

2. Application and relevance to Research Computing
   FUSE is an enabling technology. By itself it gives no useful new capabilities to a system, but it is required to run a wide variety of innovative virtual filesystems now available. Most of these are free (usually GPL) and some are commercial. Some of the available filesystems are potentially directly useful to research groups, especially those using applications which they are unable to modify. For example, an application expecting to open a plain data file could be used directly with compressed files, encrypted files, or files which are included in archives (.zip, .tar)

   Developing a new FUSE-based filesystem may also be an effective strategy for groups doing data acquisition, to isolate the hardware-dependent code from the rest of the data processing.

3. Assessment
   The FUSE source code was downloaded, configured and built using GCC on a Red Hat (RHEL4) system. The FUSE module is written on C, and built without errors or warnings. As a device driver, it requires that the kernel headers package be installed (kernel-devel on RHEL4). It installs one SUID binary (fusermount) and several shared libraries, all below /usr/local. A boot-time init script loads the FUSE kernel module and makes the virtual device /dev/fuse available (which must be give public read/write permissions for unpriviledged virtual filesystems to be used). FUSE-based programs are linked against the provided shared libraries, and communicate with the kernel via /dev/fuse. They call the fusermount utility to instantiate the virtual filesystem and make it visible to the rest of the system.

   MacFUSE was downloaded from the Google site and installed with a typical Mac OSX installer. No user configuration or building was needed.

4. Advantages
   Applications or user codes which expect to read data from simple files may be able to support data in various forms of compressed archives, local or remote, or on devices not directly accessible by normal means, without modifying the applications.
5. Disadvantages
   
   • While FUSE seems to be stable and have a good record of stability and security, many of the potentially interesting contributed filesystem drivers are not production quality.
   • Not supported by Red Hat Enterprise Linux (as of RHEL5). As a 3rd-party kernel module, it must be rebuilt any time a kernel update is installed.
   • A smaller number of FUSE-based filesystems are available yet for MacOSX.
6. For more information
   FUSE home page: fuse.sourceforge.net
   Mac OSX port of FUSE: code.google.com/p/macfuse/
7. Some FUSE-based filesystems

   FUSE allows the user to install and run a wide variety of virtual filesystem types, or develop new ones. A small selection of the available filesystems is listed below:

   • sshfs -- Mount any subset of a remote system accessble via SSH. Access to remote files is the same as an interactive SSH login would give you. The remote system needs no additional software installed. (Tested on RHEL4 and OSX)
   • AVFS -- Mount a variety of compressed files and archived file collections. This was the mechanism used by the Midnight Commander file manager to "open" various file formats, including tar, zip, gzip, bzip2, ar, rar, rpm, and grew into the FUSE project. There is an AVFS FUSE daemon, plus handlers for each file format (which may be implemented in perl, shell, python etc.) AVFS also supports remote files via ftp and http. (Tested on RHEL4).
   • Run Time Access -- A library to expose internal program structures and arrays as database tables and as virtual files
   • GlusterFS -- A clustered filesystem, using Infiniband or TCP/IP to aggregate storage units into a parallel filesytem scaleable to petabytes.
   • CVSFS -- Mount a CVS (Concurent Versioning System) tree as a filesystem and see old versions directly as files.
   • FunionFS -- An aggregation of a read/write filesystem (e.g. RAM disk) and read-only (e.g. CDROM).
   • TagFS -- A filesystem for audio files, presenting files organized by the ID3 tags in MP3 and OGG files.
   • wikipediafs -- Mount a Wikipedia page as a file, to allow editing directly with the HTML editor of choice.
   • BlogFS -- Mount a WordPress blog and edit posts directly.
   • FUSEPod -- Mount an Apple iPod as a filesystem
   • mysqlfs -- A filesystem in which the file data are stored in a MySQL database.