Image of the glider from the Game of Life by John Conway
Skip to content

XFS vs Reiser

Last night, Christer and I were playing with the Ubuntu 8.04 installer, trying to break it, and get any last bugs reported before final is released tomorrow.

What I noticed last night, during the install really surprised me. ReiserFS is a screaming filesystem compared to XFS. After doing the install 3 times on 2 identical machines (Dell Precision 490), in every case, ReiserFS was nearly twice as fast completing the install. Didn't matter how the disk was partitioned either. It is a screaming speed machine. Further, launching applications, and playing with KVM, it was also noticeably faster than it's XFS competition.

However, XFS did have much better storage management in the way of sqeezing out every last byte on the drive. In fact, on bare bones installs, XFS gave me 10% more storage space than ReiserFS.

Conclusion? If you're after speed, from what I gathered last night, ReiserFS screams. If you're looking for maximized disk space, XFS is the clear winner there. Was the disk space gain worth the wait of the install? Yeah, probably. But man, post-install, launching applications with ReiserFS was still noticeably faster than XFS.

I'm curious about these, and will be benchmarking some more in the future. Expect follow-up posts on this topic.

{ 20 } Comments

  1. Vadim P. using Firefox 3.0b5 on GNU/Linux 64 bits | April 23, 2008 at 5:45 am | Permalink

    Is the map bug in the installer fixed now?

  2. Vicente using Konqueror 3.5 on GNU/Linux | April 23, 2008 at 5:52 am | Permalink

    I will keep an eye on you waiting for the other benchmarks.
    I usually use ext3 without any reasoned argument.

    Thanks!

  3. Daniel using Konqueror 3.5 on Unknown O.S. | April 23, 2008 at 5:52 am | Permalink

    I saw some benchmarks that showed that ext3 is the fastest filesystem just over xfs and jfs and even ext2 I think this is the page http://bulk.fefe.de/lk2006/

  4. Wolfger using Firefox 2.0.0.14 on Windows XP | April 23, 2008 at 6:04 am | Permalink

    When you say Reiser, you mean Reiser3, correct? Or is this on Reiser4?

  5. jdong using Firefox 3.0b5 on GNU/Linux | April 23, 2008 at 6:12 am | Permalink

    Filesystem benchmarks are always a touchy and controversial subject. I've spent a year of my life chasing after all the Linux filesystems in terms of performance. Reiser is *REALLY REALLY FAST* at metadata operations (i.e. the creation and deletion of millions of new files, such as installing an Ubuntu system) however it has many drawbacks:

    (1) DEAD upstream, DEAD migration path. The founder doesn't care about it anymore, and the only guy maintaining reiserfs in upstream Git is one SUSE developer to support SLES bugfixes. We can only guess what happens when that version of SLES EOLs. There is also no migration to a newer filesystem other than reformat and restore.

    (2) Metadata latency: Metadata operations lock up the whole kernel. Even if you have an entire MP3 in memory, removing a large folder can cause your music to skip, games to be choppy, etc because the kernel does not allow anything else to multitask while that goes on.

    (3) Performance degrades over time: A 1-month-old reiserfs volume can be an order of magnitude slower than before due to tree packing / balancing algorithm issues. This is addressed in the reiser4 rewrite, but heavily affects reiserfs3. Many seasoned reiserfs geeks have scripts to rebuild certain parts of their filesystem with mkreiserfs periodically.

    (4) Heavy file fragmentation with large files. A 4GB DVD torrent was finished in 4,000 or so individual fragments and read back at 8MB/s on a 40MB/s drive. Even copying that file over itself only yielded a small improvement, while under ext3 a copy-back will yield raw disk performance and XFS doesn't even fragment the file to BEGIN with.

    With that said, I think you also hit XFS at its weakness: metadata operations. XFS is very slow at creating files, very slow at deleting files. Well, that's not entirely true: In ext3 and reiserfs, deleting files is an O(n) operation: Time taken to delete a file increases linearly with the size of the file. In XFS, it's nearly O(1): deleting a 50TB file and deleting a 1KB file take the same amount of time (in case you're curious, deleting 50TB worth of 200GB files in ext3 apparently takes 2 weeks on a decent SCSI storage cluster), but that constant amount of time sucks for small sized files. This is the effect that you see during an installation.

    However, XFS does have unique advantages:

    (1) Very nice handling of large files, as mentioned before. If you work with 200+MB sized files regularly, XFS is your filesystem!
    (2) Only filesystem with online repacker. XFS has a command xfs_fsr for "defragging" online filesystems. This can be helpful with slowly torrented or other large files that can become fragmented enough to impact performance.
    (3) Only filesystem with online dump tool: xfsdump is awesome for backups and that alone is why I choose it personally.

    It does suffer from poorer handling of bad blocks (kernel panic) though, so it shouldn't be used on hardware that is prone to such defects.

  6. jimcooncat using Firefox 2.0.0.14 on Ubuntu | April 23, 2008 at 6:25 am | Permalink

    I've been a ReiserFS fan for years, since building my first in-house mail server from Gentoo and Qmail, many years ago. A few tips on its use (the first for ANY filesystem):

    Check your hard disk before committing your personal data to it. Run badblocks shortly after installation, and if there are any, consider trashing the disk. If you can afford the time to set it up, install smartmontools to use modern disks' ability to warn you if the disk is going bad.

    When mounting a partition that boots, (/ or a separate /boot partition), use the -notail option. This prevents small files from staying absorbed in the filesystem, which boot managers don't like. You can also use it for partitions you want maximum performance on at the expense of disk space. The old Gentoo docs used to specify a separate /boot partition because of this. That's too much to maintian for me. I use a / partition with -notail, but a separate /home without the option. Perhaps next time I'll use -notail on /home too, but my systems are working great for now.

    Most filesystems (I don't know about XFS or other esoteric ones) bog down when approaching maximum capacity. I've found (through observation only, no benchmarks) that somewhere between 90% and 95% full the disk screaming stops and my own screaming starts. I set a monitor (I use monit, but a quick cron/bash script would work well) to warn me at 80% full. Then it's time for me to start purging back to 60% or start planning to increase capacity.

    Note that we're NOT talking about Reiser4, we're talking about ReiserFS.

  7. jimcooncat using Firefox 2.0.0.14 on Ubuntu | April 23, 2008 at 6:33 am | Permalink

    John (jdong): That is excellent info! Perhaps it's time for me to start reconsidering what I'm doing.

    Aaron: I'll be watching for your followup. Good luck!

  8. Leandro Peracchi using Firefox 2.0.0.14 on Windows XP | April 23, 2008 at 7:03 am | Permalink

    Hi!

    I´m from Brazil.

    I want to install Ubuntu on a computer with two identical hard drives using RAID1 and encrypted volumes.

    In 7.10 there was a bug that doesn´t allow install this way.

    I tried one build of 8.04 and appears to me that the same bug is still there...

    Do you know something about this?

    P.S.: sorry for my poor english... ;)

  9. Jeff Schroeder using Firefox 3.0b5 on GNU/Linux | April 23, 2008 at 7:06 am | Permalink

    Not long ago, there was a guy on Linux kernel planet that blogged about creating a new filesystem. He did exhaustive benchmarks on the main Linux filesystems to see what he was up against. Here are some non-biased benchmarks for you to look at:

    http://tservice.net.ru/~s0mbre/old/?section=projects&item=fs_contest2

    reiser3 and resier4 get pretty impressive results in a lot of areas. From experience, reiserfs is the WORST filesystem at recovering from hard shutdowns. It is more likely to corrupt itsself and require an fsck than any other filesystem.

  10. Travis using Minefield 3.0pre on GNU/Linux | April 23, 2008 at 7:36 am | Permalink

    Interesting Jeff. In my experience, using reiser for several years, it is the best at recovering. Actually, I've found that it's better at recovering than xfs. I built 2 machines with xfs and had multiple corruptions, especially with my firefox profile for some reason. I've never lost a byte to reiser.

    XFS is not particularly suited for a desktop machine ... man it felt slow.

    Reiser always feels fast.

  11. jdong using Firefox 3.0b5 on GNU/Linux 64 bits | April 23, 2008 at 7:44 am | Permalink

    Err, when your reiser requires a fsck, good luck :) It's about the MOST sensitive fscker to random data on the disk and easily gets side-tracked and rebuilds a trash B-tree and calls it quits.

    XFS is not designed to be a desktop filesystem where a good proportion of your files are small and constantly created/deleted. It is good for a storage partition for your multimedia content though -- possibly the best suited for the job.

  12. Kees Cook using Firefox 3.0b5 on GNU/Linux 64 bits | April 23, 2008 at 9:15 am | Permalink

    I have to agree with most people here.

    XFS: I have seen repeated, disastrous, corruptions with XFS. It has eaten libc upgrades, it truncates files, it wrecks its own metadata. I cannot recommend against it strongly enough.

    ReiserFS: in the general case, this FS is okay. However, recovering it from corruption can take literally hours, and it has a tiny fraction of the number of developers as ext3.

    Since ext3 can grow while mounted, handles recovery well, and has tons of developers, I think it's the best choice unless you have really really specific needs.

  13. Scott James Remnant using Epiphany 2.22 on GNU/Linux 64 bits | April 23, 2008 at 9:19 am | Permalink

    ...and if you're after your data, use ext3

  14. jldugger using Firefox 2.0.0.14 on Ubuntu | April 23, 2008 at 12:14 pm | Permalink

    Unless you have redundant power, I'd stick with with ext3. Like Kees, I've seen XFS corrupt data on power failure. The FAQ on the subject used to say "If it hurts, don't do it" as if the only reason this happened was people forcing power off.

    If you're really after benchmarking etc, it might be informative to the general Linux using public to examine multiple versions of ext4, as we can expect the performance characteristics to change over time.

  15. Scott Wegner using Firefox 3.0b5 on GNU/Linux 64 bits | April 23, 2008 at 3:49 pm | Permalink

    I plan on reformatting my HDD next week and starting anew when stable Hardy comes out. As such, it's one of my goals to try out new filesystems to fit my needs. Previously I've been dual-booting, so I had ext3 for Ubuntu partition, and NTFS for data. But, I'd like to try ReiserFS for the Linux partition, and perhaps XFS or JFS for the data partition. I use MythTV and store a lot of large video files on the data partition, so I've heard XFS and JFS are both good options. Is one better than the other?

  16. Anthony using Firefox 3.0b5 on GNU/Linux | April 23, 2008 at 8:49 pm | Permalink

    I have to agree with jdong and most of the others about XFS. It's certainly not good (possibly very poor) for regular desktop filesystem partitions, but I've been using it for years on my storage drive(s) without any problems. It's varied from 200gb drive up to a ~1 tb RAID 5 array.... holding movies, mp3's, backup tar archives, and other large files it excels greatly at. Never had any corruption issues in the past 5-6 years I've used it. It also excels at very large partitions and guaranteed I/O rate. Supposedly JFS is somewhat decent at this as well but cannot give any real experience.

    I've actually heard ReiserFS was the worst for stability issues although this was at some time ago. Have not used it but know it's fast for large amounts of small files especially with lots of creation/deletions.... /var or /tmp perhaps?

    Personally use XFS on my fileserver RAID array (running NFS) and ext3 for all partitions on all other boxes. Highly recommend against using ext3 for any large partitions, tried it - it's crazy slow.

  17. Trevor using Firefox 2.0.0.14 on Ubuntu | April 28, 2008 at 4:05 pm | Permalink

    Guess this solves the whole debate.

    http://blog.wired.com/27bstroke6/2008/04/reiser-guilty-o.html

    ReiserFS will start to fade away.

  18. jimcooncat using Firefox 2.0.0.13 on Ubuntu | April 29, 2008 at 4:52 am | Permalink

    Update 4/29/08: John Dong (jdong) has enlightened me regarding the waning support of ReiserFS. The website of the developing company, Namesys, has now gone offline. OpenSUSE did have good reasons to stop using it as the default filesystem, and I wished I'd run across their reasoning earlier (link below). I'm reevaluating, and will probably start using Ext3 instead.

    http://blog.linuxoss.com/2006/09/27/suse-102-ditching-reiserfs-as-it-default-fs/

  19. will using Firefox 2.0.0.16 on GNU/Linux | August 25, 2008 at 1:39 am | Permalink

    I like them all. Some require tuning for good performance, expecially xfs, and some don't work well with some kernels.

    my personal
    /dev/hda5 on / type reiserfs (rw,noatime,data=writeback,notail)
    /proc on /proc type proc (rw)
    /sys on /sys type sysfs (rw)
    none on /dev/pts type devpts (rw,gid=5,mode=620)
    /dev/hda8 on /1 type reiser4 (rw,noatime)
    /dev/hdi3 on /2 type xfs (rw,noatime,nodiratime,logbufs=4,logbsize=256k)
    /dev/hda10 on /4 type jfs (rw,noatime,nodiratime)
    tmpfs on /dev/shm type tmpfs (rw)

  20. Alex using Konqueror 3.5 on Unknown O.S. 64 bits | October 23, 2008 at 1:48 pm | Permalink

    As Will stated, XFS require tuning for performance. This webpage is quite interesting

    http://everything2.com/node/1479435

    Personally, I haven't tried XFS, but it sounds quite good...

Post a Comment

Your email is never published nor shared.

Switch to our mobile site