I've worked for a long time in the storage field during my career and ZFS is the piece of engineering I admire most in that space. I wanted a way to pay homage to it and to explore and better understand its internals. So I built Uberblok: a museum of ZFS. A deep dive into copy-on-write, txgs, vdevs, RAID-Z, snapshots, the ARC, metaslabs, the ZIL, send/receive with explanations and animated diagrams.
I'm sure people here know these internals better than I do, so corrections are very welcome.
I built it with my agent swarm organization, Algotitan.
I hope you'll enjoy the deep dive into Uberblok as much as I do.
Museum: https://uberblok.storagebits.workers.dev Repo: https://github.com/algotitanhq/uberblok
QuantumNomad_13 hours ago
I absolutely love ZFS. I use it on FreeBSD for all disks on the whole system, including the ones that FreeBSD itself is installed on, and I use it with some disks on Linux and with most of my external disks with macOS.
In an alternative universe, Mac OS X would have gotten ZFS shipped with the base install back in 2007, and presumably we’d then still have that to this day. In our universe that ended up not happening even though it was supposed to :/ https://mjtsai.com/blog/2025/04/28/revisiting-zfs-for-mac/
I really would have loved to have the whole of macOS running on ZFS. But at least using ZFS from homebrew on macOS to have ZFS on external disks is the next best thing after that, and which we do have.
storagebitsop12 hours ago
I forgot that Apple story, a great "what ifs". It's not in the history wing yet, thanks for the link, it might deserve its own panel!
cbarrick10 hours ago
Genuinely cool, but the LLM-speak started to wear on me.
An editing pass to make the prose less "punchy" would be great.
MithrilTuxedo5 hours ago
That's what that is! I was doing well until I got lost at Map of the Visit. I forgot I was pretending to be in a museum.
drewg12313 hours ago
I'd love ZFS a lot more if it weren't for the ARC. Using ARC means that ZFS caches stuff outside the kernel page cache. That in turn means that sendfile is no longer zero-copy when data originates on ZFS, making it a non-starter for content drives for our CDN workload.
ahofmann12 hours ago
Yes, but this is a Linux/licensing problem. Use one of the BSDs and the ARC doesn't fight with the system cache.
ashkankiani11 hours ago
Building a museum when you self admit you don't understand the internals that well... seems like a conflicting goal? If you're showing off to educate then how can you do that if you don't understand it?
We know the answer, obviously. I can't trust how much of this was something an AI said vs something you actually did seriously look over. At the very least, all of the text on this should've been in your own words, I think. Otherwise this is just show and tell of some prompts.
I wish I could enjoy tech demos anymore, but now they feel so hollow. Which is a real loss from the age of LLMs. I used to enjoy reading the dolphin EMU progress report for years. I wonder what they're up to.
ghm218013 hours ago
One small suggestion: Make the code open to the right instead of a new window, that makes it easy to keep context without distraction.
storagebitsop12 hours ago
Good idea, thanks! A side panel is the way to go.
storagebitsop8 hours ago
Shipped, thanks to you: the copy-on-write room now opens its sources in a side panel. Your comment went from here to production in about four hours. Others rooms to come. The rendering is far better!
AyyEye12 hours ago
> I built
You keep using those words. I don't think it means what you think it means.
storagebitsop12 hours ago
Fair hit. "Directed" is closer: agents wrote the code and most of the text, I chose the subject, steered and reviewed.
Brian_K_Whitean hour ago
Nobody wants to land a "hit". They were hit. The hit was being shown something as though it were valuable, only to discover it is not.
gspr10 hours ago
You were a manager, in other words.
ByteAtATime12 hours ago
Inconceivable!
unethical_ban14 hours ago
As a mere operator, I've been fascinated by ZFS ever since I first heard of it 15+ years ago. It seems so elegant, like it really is the last generalist filesystem one would ever need. Performant, reliable, fundamentals easy to comprehend. I've never tried managing LVM configs because zpools and raidz are just so damned easy. Native encryption. Datasets can be as simple or as complicated as you need them to be. send/recv dataset syncing at the block level vs having to decrypt and sync files individually.
A lot of this goes over my head as someone who isn't familiar with filesystem internals, but I appreciate the effort and the homage to a great tool.
yjftsjthsd-h10 hours ago
> Native encryption.
I think maybe that's the one feature that's iffy? Theoretically great but bad history of bugs. (Unfortunately)
gspr10 hours ago
So many really brilliant people keep saying this, so I don't doubt that it's true. Can you explain to an ordinary user why ZFS isn't everywhere? It seems to be a marvel, yet ext4 lives on.
tom_alexander4 hours ago
Everyone has mentioned the license, which is the reason ZFS isn't everywhere. I just wanted to add: Don't let the license scare you away. The CDDL is actually a nice compromise between copyleft (GPL) and copyfree (BSD). The CDDL is essentially the same as GPL except the GPL applies to everything it touches (include a single GPL *.c file in your project and now your entire project is subject to the GPL). Whereas the CDDL only applies to the CDDL-licensed files themselves. So you can include ZFS in your project without making your entire project subject to the CDDL.
The issue is, the GPL wants to spread to everything it touches (including the CDDL-licensed ZFS files), which would make the ZFS files subject to both the GPL and the CDDL. But the CDDL and the GPL are incompatible, which is why Linux can't include ZFS whereas more permissively licensed projects like the BSDs can.
So it's an unfortunate situation, but ZFS' license is not "bad". It's just the GPL and the CDDL are incompatible.
otterley4 hours ago
Because ZFS is complicated to use and isn’t necessary for most workloads.
The licensing issues don’t help. Consider that most Linux machines are in commercial settings (Google runs more of them than all individuals combined) and they don’t need Oracle on their backs.
jjav2 hours ago
> Because ZFS is complicated to use
There is nothing complicated about having zfs be your filesystem. You can use its extra features like snapshots, send/receive, etc, but only if you wanted to. Otherwise it's just a filesystem, there is nothing to use as such, so nothing complicated.
> and isn’t necessary for most workloads.
Not sure what that means. What workload wouldn't benefit from a reliable filesystem?
otterleyan hour ago
The fact that ZFS isn’t massively popular after close to 30 years, and after nearly 20 years of availability via OSS, suggests that the benefits aren’t really that meaningful to most users.
jjav37 minutes ago
Others have answered this: licensing
Linux is the most common platform and many Linux distributions have an issue with license compatibility (or perceived one), which hinders adoption.
storagebitsop10 hours ago
I would say (in no particular order) that the CDDL license is not innocent, ext4 is good enough for many usages, cost ? ZFS is RAM hungry ... but ZFS is well present where it means most (NASes , Proxmox , backups to tell some)
[deleted]9 hours agocollapsed
unethical_ban8 hours ago
Primarily licensing, IMO. Its license is incompatible with inclusion in the Linux kernel. Ubuntu ships with native support for it anyway; I guess they're not worried about legal issues. It's completely functional in Linux, you just have to take an extra step to install it, which is enough to keep it from mass non-techie adoption.
Totally worth it for NAS. I think there are more systems that utilize its advanced dataset features in the BSD world.
storagebitsop13 hours ago
Thanks, that's exactly the reader I'd like to reach, and you're not the first to say it goes over your head. Honestly, I'm still trying to figure out who it doesn't go over the head of. Matt Ahrens, maybe? ^^ Next is a "start here" route: one plain question per room, answered before any code. Copy-on-write and snapshots are probably the best rooms to start with today.
ghostly_s11 hours ago
If you can't be bothered to copy edit the gratingly AI prose, I can't be bothered to read it.
sprixia2 hours ago
[flagged]