Disks, partitions and file systems: a complete guide

Storage is one of the fundamentals of the modern Information Technology. We use the word storage to describe persistent data storing, something that doesn’t disappear in case of a power shortage.

The history of storage

Storing data has always been important in our history since we as human beings want to pass down our knowledge, want to share our experiences, want to avoid danger together. In the early years of the human kind, humans used large rock walls to store experience in form of pictures: rock art. But that’s not data, that’s experience: subjective and not really measurable. We come closer to the concept of data when the human kind developed language. Language is complex code that uses voice to give thoughts a physical form to convey to another individual. That’s what language used to be until we developed writing: the art of taking thoughts, transforming into language, and ultimately storing them in a persistent way. Today writing and language are almost one concept, but remember that before the invention of writing, there was no way to store persistently a concept, the only known way was to pass it down from father to son. And that lead us to the first type of storage human kind has ever used, and probably the most powerful still today: books. Books can store pages and pages can store characters (which are part of a language), that’s a persistent storage. Books have been the primary means for thousands of years, then came punched cards/tapes, then Williams tubes (and derived), then magnetic drum, magnetic tapes, floppy disks, optical discs, and finally hard disks. It is with the latest however that storage starts to become what it is today.

Disks and partitions

As I told in the last paragraph, (hard) disks are a form of storage and today are the most used mean to store data. But alone, disks are just a physical mean to store data, what lacks is organization. That organization comes in the name of Partition. A partition is a logical form of boundary, it is used to divide the disk in logical units so that can be used in order to fulfil the disk’s true purpose: storing data. Partitions store data, but where are partitions stored? Partitions are stored in what’s called a partition table. Partition tables store the data associated with partitions, where a partition starts, where a partition ends and such things. However partitions are not enough to store data in an ordered manner. To do that we need another system, a file system. A file system takes care of storing pieces of data (files) that would have no sense if they were striped together. All of that in order to get to the last form of order: files. Files themselves are just a bunch of data that are stored through the file system, which resides in a partition, which is recorded in a partition table, all of this, inside your hard disk.

Partition table types

There are many partition tables out there, we will however look at the two most common ones: MBR and GPT. Which one to use is entirely dependent on what the needs are.

The old way: Master Boot Record

The MBR is the old way of doing things, it has been widely used in the past and it is still widely used today, however it has many limitations, mostly:

• You can have only 4 partitions. (You can overcome this)
• Partitions are limited to 2TB.

As we said MBR is quite used, however when UEFI was proposed, it came clear something new was needed.

The new way: GUID Partition Table

That’s where GPT comes in. GPT overcomes the limitations of MBR:

• You can have up to 128 partitions.
• Partitions can be as great as 9.4 ZB.

Yes, that’s a lot, and all, but usually operating systems, like Windows, impose a stricter limit. For example, Windows impose a maximum partition size of 256 TB.

File systems

File systems (or filesystems) are the last piece of the puzzle and probably the widest one. Filesystems are mostly designed to fulfil a specific need. There are many out there, let’s look at the principal ones:

• FAT/FAT32/exFAT (File Allocation Table) family was originally Windows file system, however it has limitations (depending on the type: 16/32/ex). For example FAT32 can’t handle more than 2GB in a single file. So even if you have a USB stick with 4GB, you won’t be able to store more than 2GB in a single file. FAT support is almost universal.
• NTFS (New Technologies File System) is one of the most famous (probably the most famous) file system. It is associated with Microsoft and was developed to replace FAT. NTFS is supported (like FAT) almost universally. In the past years it was prone to fragmentation, today that issue has been highly reduced.
• EXT/EXT2/EXT3/EXT4 (Extended File System) is historically the Linux filesystem, it is highly flexible, supports ACLs and journals (Ext3/4). It is supported on UNIX-like operating systems, but has no real support on Windows (though there are software to overcome this).
• HFS/HFS+ (Hierarchical File Systems) is the official Apple filesystem. It is primarily used on Mac computers.
• ISO 9660/UDF, are filesystems used on optical disks. The latter has nothing to do with the first one and was developed in order to replace the former.
• XFS is a relatively new filesystem designed to handle huge amount of data. It is primarily used by enterprises.
• BTRFS is a really promising filesystem that has snapshots, high performance and can possibly span across multiple drives. As XFS it is primarily used by enterprises and it is still somewhat unfinished, though it is not unstable and almost ready for production.

Conclusion

This is the structure of what you use probably every day. But what about SSD and USB sticks? The same apply.

• Author
• Recent Posts

mark

The IT guy with a slight look of boredom in his eyes. Freelancer. Current interests: Kubernetes, Tensorflow, shiny new things.

Latest posts by mark (see all)

• 2020 A year in review for Marksei.com - 30 December 2020
• Red Hat pulls the kill switch on CentOS - 16 December 2020
• OpenZFS 2.0 released: unified ZFS for Linux and BSD - 9 December 2020