Pardon, What’s a Filesystem?

I apologize for the long delay between posts - I have so much I want to tell you all, and I’ve made a lot of progress on the GridTree project! I have another post in the works that I’ve been working on (off and on) for a long time, and it grew to be… a lot. So I’m going to simplify my life and actually post something for once by excising this portion here out of that longer post. This is for those who could use an explanation of computer filesystems. I’ve made it very accessible and I genuinely hope you feel smarter after reading it! Or, if you know what a filesystem is, feel free to skip this one - though if you do, you’ll miss out on man-eating books and mechanical monsters….

What do I mean by a filesystem? A filesystem, or file system to be more proper, is the software construct within your computer Operating System (OS) for organizing and storing files, and it provides the algorithmic foundation for quick access to them. It’s what handles how and where they are physically represented on the hardware, so that you don’t have to, instead showing you a tidy tree of folders to put things in.

Some years back, it used to be that if you walked into an office building, or perhaps a library, and asked about the “filing system”, a bored office worker might point you to a wall covered in filing cabinets; or if it’s in a library, the librarian might point you to the Card Catalog. Those cabinets are the location of the stuff, yes, as are the shelves in a library, but the actual system is how they organize the stuff in those cabinets and and shelves. When you open a drawer, do you find things neatly in alphabetical order? Or perhaps conveniently grouped by category? (All the financial documents here, customer forms there, etc.) Or, when you look inside, are you violently assaulted by a bunch of piles crunched in there willy-nilly, that make a bid for freedom as soon as you show open the drawer, or a forest of Jenga towers of books with a homicidal appreciation for gravity? (I sincerely hope not. Although, that would make libraries a lot more dangerous and interesting, and people might actually find going there more of an adventure! Bring a hard hat. And sign a waiver.) In reality, the librarians would have likely been using the Dewey Decimal System, which is a method of organizing books by categories and subcategories, and then alphabetically within each of those. And they are neatly, safely put on tidy shelves, where they have a much harder time of harming any passersby who sneeze or look at them askance.

A method, an ordering system, is at the core of what a computer filesystem is, too. Like the folders in a wall of filing cabinets, digital files have a solid place where they are physically stored on a disk. Just as the cabinets might have an index which shows which cabinet holds which items, the filesystem of a computer also knows where to find a given item. However, unlike the filing cabinets, which are likely entirely sorted, the files on a computer are totally reliant upon a virtual hierarchical indexing system to know where to find things, and the actual items on the disk are sort of shoved in the physical cabinets willy-nilly after all, in whatever empty space the OS could find. The user sees the neat and tidy tree, rather than a phone-book style list of which shelves or cabinets are where.

There’s a few reasons the files are not tidily sorted physically on the disk. One of the big ones is that while office workers can drag cabinets apart to insert an extra when one grows too big, you can’t do that on the disk without physically moving every file from there to the end of the disk to make room - in a library, literally moving about half the library’s books! In fact, the hierarchical index on a computer, the “filesystem”, is actually intended to simulate the cabinet-style mobility virtually - one folder might be inside another folder, and those inside a certain drawer, which is in a certain cabinet. So, instead of the cabinets representing the physical disk items, the cabinets are the index. Moving a drawer or a cabinet or a folder is easy. Similarly, in a computer filesystem, it is as easy as rewriting an entry in the index. Since the user doesn’t even see where the items on the disk are physically, the user doesn’t care where they are physically, see?

This is especially important if, say, you want to move a thousand items at once - rather than busy your entire office staff moving folders between two rows of cabinets for an hour, or dragging cabinets across the (formerly) nice linoleum, you could just paste over the stickers on them that describe what their contents are, right?

This filesystem is called a “layer of abstraction”, hiding what’s underneath and presenting something more user-friendly to the user. As a side note, another useful abstraction provided by your filesystem is chunking larger files into smaller segments behind the scenes (a physical folder stuffed with a ten-inch-thick pile of loose-leaf papers might not all fit in one available space in a cabinet, so the system puts pieces of it wherever they fit and links them up for you). On the other hand, one downside of the hierarchical index is that if you can’t find it in the index, you literally have to search through every file to find what you lost - scanning every book on every individual shelf in the entire library to find one misplaced title. (If you hate search tools, you should know that they’ve gotten better than this by now - they actually create a second index behind the scenes which stores some information about the contents of every file - if you lose it in one index, you might find it by searching the other.)

Behind the scenes, this “filesystem” in a computer is really just a collection of computer code algorithms telling the computer a specific way of how to organize and access stuff on the physical hard drive or SSD. Blocks of data on the disk have “pointers” to one another to create the tree, or to link up longer files. There are a variety of different organizational systems, good at different things, or invented by different companies, so there are likewise a variety of different filesystem types. Usually your Windows machine has a different one (NTFS, exFAT, FAT32) than Mac (APFS, HFS+), and so on. And Linux (EXT4 and many more). (Linux users sometimes get choosy about these things because different filesystems aren’t always compatible with one another. Anyway.)

Now you know what a filesystem is, and maybe you can imagine how much worse it might be if you didn’t have one. You might open up your file explorer (you know, the thing that shows folders with stuff in them?) on your computer and see instead a simple list of hundreds of thousands of files, all in the order in which they were created, and you have to find things in that mess! While I am fascinated by the idea of a “Flourish and Blotts”-style library-jungle inhabited by strange creatures (and librarians), I do rather like my noggin intact, and being able to find the book I’m looking for without being pounced on by a pack of killer encyclopedias. And I don’t fancy trying to find the next file created after I delete something that’s not at the end of the list, when the system tidily fills the hole with your precious essay. (Let’s face it, you’re never seeing that homework assignment again. No, no, you didn’t need to turn it in after all.)

I guess I’m not done after all. I’ll be brief. If you want to know.

(Hang on, Nathan, I thought you said you weren’t going to talk about how it works? Sorry, I couldn’t resist.)

As I said the space on your drive is broken up into little fixed-size blocks (for various reasons that include it being able to efficiently check which blocks are free, and also so that space doesn’t get fragmented with tiny leftover spaces between things that are unusably small). These blocks are all indexed for whether they are empty or full, but in most systems the index has nothing to do with their contents, so that it can be a really small index at the front of the disk, which the system uses to quickly scan for free space to put things in.

So, since there’s no other indexed info about them, all the file metadata is stored in the block itself, at the start of the file or folder. Folders store pointers to other blocks for files or folders inside them, and file blocks are linked together with pointers as well for files that don’t fit in one block. There’s no (good) way to find something except by following the trail of pointers through the tree - if you you know the “path” (address). (If not, well, good luck.)

When you delete a file, nothing moves, it only marks it as deleted. When you copy a file, it copies it (though some filesystems wait till you edit the “copy” before actually copying it). When you move a file, nothing actually moves, it just rewrites the pointers so it’s accessed by the new parent folder. Unless you move it to a separate drive.

See? Magic. Brilliant algorithms and fascinating constructs! Revel with me in the amazingness of it. (I love this stuff. Yes, I know you think I’m crazy, and I don’t care.)

I hope you learned something you’ll find useful, and maybe it will help you understand the feral, unobedient mechanical thing sitting on your desk just a little bit better. The better you and it understand one another, the more effectively you’ll be able to train it to do your bidding! Or train yourself to use it, as the case may be. Anyway, I hope this helps.