Google Implements ext4 filesystem on their servers
By on January 15th, 2010

Google, the giant search engine who knows a thing or two about speed, has decided to implement the ext4 as the filesystem for their servers.

In a post to the ext4 developer’s mailing list, Senior Engineer Michael Rubin explains that Google had been deciding upon a filesystem upgrade to their existing filesystem, and after a lot of performance testing of the available filesystems, including   xfs, ext4 and jfs, they came to a conclusion that ext4 was best suited to their needs. Although xfs performed close to ext4, the deciding factor on ext4 was that it provided an easy upgrade path from their existing ext2 filesystem to ext4.

ext4 has become the default filesystem for several of the popular Linux distributions including Ubuntu, Fedora and openSUSE and had become an official part of Linux kernel since Linux 2.6.28 kernel. ext4 features several improvements over its predecessor, amongst which include support for files upto 16 tebibytes ( 1 tebibytes equals 1,024 gibibyes , with 1gibibyte equal to 1.074 gigabytes) and a maximum volume size of upto 1 Exbibyte. Other features of the ext4 are backwards compatibility with ext2/ext3 filesystems, and a subdirectory limitation of upto 64,000 subdirectories, and an online defragmentation tool which is still under development stages, however.

Interestingly, one of ext4′s lead developer, Theodore Ts’o is also a Google employee, notes SJVN indicating that Google would most likely use his expertise in further getting even more performance out of an already fast filesystem.

 xfs, ext4, jfs
Tags: ,
Author: Sathya Bhat Google Profile for Sathya Bhat
Sathyajith aka "Sathya" or "cpg" loves working on computers, and actively participates in many online communities. Sathya is a Community Moderator on Super User, a collaboratively maintained Q&A site which is part of the Stack Exchange network. Sathya also contributes to and is a Super Moderator at Chip India Forums. While not writing SQL queries or coding in PL/SQL, Sathya is also a gamer, a Linux enthusiast, and maintains a blog on Linux & OpenSource. You can reach Sathya on twitter.

Sathya Bhat has written and can be contacted at sathya@techie-buzz.com.
  • http://www.fundybuzz.com Williams

    Thats great news that google implements ext4 file systems. I think it will improve performance of google.

  • http://winchesterpc.com Benjamin Vander Jagt

    I wish I could get word to Google. Ext4 has been terribly unreliable and especially bad at recovering from crashes. In my server, it seems to become less reliable the more data and the larger the volume. I had a 6TB volume, which fell apart when I filled it with more than 1.5TB. I shrunk it to a 4.5TB volume, which failed at 3TB. A 3TB volume + two 1.5TB volumes so far seems stable. I'm pretty sure that Google's file systems are much, much bigger.

    Thus far, the only FS's I've seen real reliability out of have been ReiserFS 3.6 and Reiser4. Of course, having been designed by a wife-murderer makes them hard to advertise.

    Be warned, Google. Backup often…and onto a filesystem -other than- EXT4.

    • http://sathyabh.at Sathya Bhat

      Ouch, data loss is bad, no matter what. I trust you had a backup ?
      I trust Google would have redundant power backups and storage devices to counter any dataloss.

      • http://winchesterpc.com Benjamin Vander Jagt

        Hi, I did have a long-term backup. The first time it failed, though, I had a backup onto another mirrored Ext4. I started pulling data back off of that, when it fell apart as well. I lost some data, and therefore some work hours, but nothing irreplaceable was lost. The time loss recovering from data disasters has certainly made any speed improvements meaningless. Thanks for your concern.

        I don't know what Google's planning, but at least this means that Google will probably spend the time and money necessary to make Ext4 more reliable. It seems as though Ext4 is getting -slower- during its development, and I'm running across more critical bugs as I continue applying updates and patches. Reiser4, on the other hand, has never had data loss problems since its official release, and new versions improve speed. I really don't want to bring up the old Ext vs Reiser flame war, but in all of my experience I just can't find proof that Ext4 is as reliable as the rhetoric suggests.

        • http://sathyabh.at Sathya

          Hmm, interesting that. I do agree about recovering from data loss can be a nightmare – more so when you can't seem to find some data when you *really* need it the most.

          Do drop in a comment if you can find an article/post featuring some real world data on Ext4 vs Reiser4, I'm sure it'll make for an interesting read, atleast to me.

          • http://winchesterpc.com Benjamin Vander Jagt

            I would love to, but unfortunately it's unlikely that any authoritative article on the topic exists. Because Reiser4 isn't in the mainline kernel, few mainstream distributions support it, so few real-world configurations use it. There's some word about Reiser4 making it into the kernel somewhere around the second half of this year, but that's not set in stone, either.

            However, a colleague of mine does have some emails from kernel developers that state their dissatisfaction with the reliability of Ext4. I'll ask him at the beginning of the week. (I won't see him for a little while, since we're getting close to three feet of snow tonight.)

            If you're curious about the Reiser4 inroads into the kernel, be sure to check Phoronix, as that's where I read it.

          • http://sathyabh.at Sathya Bhat

            Thanks, Benjamin. I'll hit Phoronix and see what I can find.

  • http://ms1.gotdns.com jonny rocket

    yes! i can see/feel a big diff between ext3 and ext4.

 
Copyright 2006-2012 Techie Buzz. All Rights Reserved. Our content may not be reproduced on other websites. Content Delivery by MaxCDN