Unix File System - History and Evolution

History and Evolution

Early versions of Unix filesystems were referred to simply as FS. FS only included the boot block, superblock, a clump of inodes, and the data blocks. This worked well for the small disks early Unixes were designed for, but as technology advanced and disks grew larger, moving the head back and forth between the clump of inodes and the data blocks they referred to caused thrashing. Marshall Kirk McKusick, then a Berkeley graduate student, optimized the BSD 4.2's FFS (Fast File System) by inventing cylinder groups, which break the disk up into smaller chunks, with each group having its own inodes and data blocks.

The intent of BSD FFS is to try to localize associated data blocks and metadata in the same cylinder group; and, ideally, all of the contents of a directory (both data and metadata for all the files) in the same or nearby cylinder group, thus reducing fragmentation caused by scattering a directory's contents over a whole disk.

Some of the performance parameters in the superblock included number of tracks and sectors, disk rotation speed, head speed, and alignment of the sectors between tracks. In a fully optimized system, the head could be moved between close tracks to read scattered sectors from alternating tracks while waiting for the platter to spin around.

As disks grew larger and larger, sector level optimization became obsolete (especially with disks that used linear sector numbering and variable sectors per track). With larger disks and larger files, fragmented reads became more of a problem. To combat this, BSD originally increased the filesystem block size from one sector to 1K in 4.0BSD; and, in FFS, increased the filesystem block size from 1K to 8K. This has several effects. The chances of a file's sectors being contiguous is much greater. The amount of overhead to list the file's blocks is reduced, while the number of bytes representable by any given number of blocks is increased.

Larger disk sizes are also possible, since the maximum number of blocks is limited by a fixed bit-width block number. However, with larger block sizes, disks with many small files will waste space, since each file must occupy at least one block. Because of this, BSD added block level fragmentation, also called block suballocation, tail merging or tail packing, where the last partial block of data from several files may be stored in a single "fragment" block instead of multiple mostly empty blocks (Allen 2005).

Neither UFS nor other currently widespread file systems — such as Ext (which was a port of UFS), XFS, JFS, NTFS — nor Hardware RAID provide sufficient protection against Data integrity problems. The modern ZFS adresses these issues. While it is also faster than UFS or DragonFly BSD's HAMMER file system, it can be seen as the successor to UFS. Research indicates that ZFS protects data better than earlier solutions. The new graphical installer of PC-BSD can handle even / (root) on ZFS and RAID-Z pool and Gnome installs and disk encryption using Geli right from the start in an easy convenient way (GUI). The current PC-BSD 9.0+ 'Isotope Edition' has ZFS filesystem version 5 and ZFS storage pool version 28. The UFS2 Filesystem in FreeBSD has background fsck. The modern faster ZFS in FreeBSD, FreeNAS and PC-BSD has no "fsck" repair tool, instead, it has a repair tool called "scrub". Additionally ZFS uses Copy-on-write, intelligent Data scrubbing and Resilvering (resyncing). While fsck must be run on an offline filesystem, scrub is designed to be used on a working, mounted alive ZFS filesystem. While fsck usually only checks metadata, such as the journal log, but never checks the data itself, scrub checks everything, including metadata and the data. It is also possible to simply swap a harddisk drive to a larger drive and resilver (repair) the zpool of disks. The ZFS file system is designed from the ground up with a focus on data integrity and protection against silent data corruption caused by bit rot, bugs in disk firmware, ghost writes, etc..