Snow Leopard Will Have ZFS!

Just announced, and chronicled on Robin Harris’  column on ZDNet, Apple’s next operating system will include the Zettabyte File System, now simply referred to as ZFS.

ZFS is a 128 bit file system, which allows for expansion that is, for all practical purposes, unlimited (why it is, is beyond the scope of this article, but a very good description of the entire file system is located here [pdf] .)

This is very bad news for Microsoft, for as we all know, will never be including new technology unless forced. For business, this will be the file system desired by all IT departments, and Mac will have it. Unless the boys at Redmond have been working on this in secret, they have a lot of catching up to do.

from the article

As if Grand Central weren’t enough bad news for Microsoft, now they have ZFS to contend with. Building a reliable, high-performance file system takes years and Microsoft doesn’t have years to respond.

The formal announcement is for Snow Leopard server, which is how Apple introduces new file systems. HFS+ first arrived on a server version as well.

Who cares?
Anyone who stores data should.

Microsoft’s NTFS is 20 year old technology borrowed from DEC. Fine for small disks and puny CPUs. Not so great for today’s data intensive systems and applications.

Few people I know understand this – they think that NTFS is pure Microsoft, and give credit where it is not due.

Silent data corruption is common - only you don’t know it - because the corruption shows up as other problems, like missing DLLs.

ZFS: open source from Sun
ZFS is the first desktop file system with true end-to-end data integrity. Thanks to sophisticated tree-based checksums it detects and corrects silent data corruption anywhere in the data path: disks, cables, interfaces and more.

The checksums are stored with the parent block, so the file system always knows that the child block is both uncorrupted and the correct block. That’s just one of the errors that NTFS and most other commodity file systems - including the Mac’s HFS+ - are prone too.

Sun’s ZFS engineering team started working on ZFS 7 years ago as a clean-sheet design. It combines file system and volume management functionality. Instead of managing individual disks, you manage a pool of blocks. ZFS takes care of the details.

Turning up the heat on Microsoft
For all of Microsoft’s fine talk about innovation they don’t do squat unless someone else does it first. Remember IE 6? ZFS is a modern and innovative file system that solves some difficult data storage and integrity problems.

This will tend to blow the  entire  ‘Windows 7 will be essentially a tune up of Vista’  rot, unless they wish to surrender more of the office market to Mac.

Like these:

No more Disk Warrior
Data corruption on PCs and Macs is a sad and stupid fact of life. Power failures, flaky RAM, poor grounding, (slowly) failing hard drives, driver glitches, phantom writes and more conspire to rot your data.

ZFS eliminates that. All blocks are checksummed and the checksum is stored in a parent block. ZFS always knows if the block is correct and/or corrupt. Every block has a parent block (with one obvious exception that gets special treatment), so the entire data store is self-validating. You’ll never have to wonder if all your data is correct again. It is.

No RAID cards or controllers
ZFS implements very fast RAID that fixes the performance knock-off against software RAID. In ZFS all writes are the fastest kind: full stripe writes. And the RAID is running on the fastest processor in your system (your Mac), rather than some 3-5 year old microcontroller.

Just add drives to your system and you have a fast RAID system. With Serial Attach SCSI and SATA drives you’ll pay for the drives (cheap and getting cheaper), cables and enclosures.

No more volumes
Every time you add a disk to your Mac you see another disk icon on the desktop. If you want to RAID some disks you use Disk Utility (or something) to create the volume. Slow, error-prone, confusing.

ZFS eliminates the whole volume concept. Add a disk or five to your system and it joins your storage pool. More capacity. Not more management.

Kind of like Windows Home Server – except that this actually works, and was built to work that way, not being a kludge put atop another way of handling storage.

Backup made easy
ZFS does something called snapshot copy, which creates a copy of all your data at whatever point in time you want. Copy the snapshot up to a disk, tape or NAS box and you are backed up.

Create a snapshot on every write if you want, so if your database barfs you can go back to just before it choked.

But that’s not all!
For in-depth treatment of ZFS see here and here. Includes links to more technical info and benchmarks.

The Storage Bits take
It would be nice if Microsoft were driving innovation and reliability, but - like General Motors - they prefer to rest of their laurels. And like General Motors, they are facing a long and painful decline if they don’t get their act together.

GM says they are proud that 1 in 4 cars sold in America are GM - but the number used to be 3 out of 5. Microsoft is rightfully proud of their 90% market share. But that share can change - as it has for IE - and they have nowhere to go but down.

As users we benefit from the competition. Kudos to Apple for bringing the latest technology to consumers.

If Mac gets this right, it will immediately own another 10-15% of the corporate market – that’s my own prediction, and that percentage will continue to grow, until Microsoft has their own implementation – reinvention of the wheel will not suffice here – it is nothing to be ashamed of to acknowledge that someone else got to an answer before you, it is only a problem if you fail to acknowledge, or act.

(further reading about ZFS [pdf])

-

Quote of the day:

At times one remains faithful to a cause only because its opponents do not cease to be insipid. - Friedrich Nietzsche

Technorati Tags: ZFS - Apple - Sun - open source - self-healing - journaling file systems