Image of the glider from the Game of Life by John Conway
Skip to content

Encrypted ZFS Filesystems On Linux

This is just a quick post about getting a fully kernel-space encrypted ZFS filesystem setup with GNU/Linux, while still keeping all the benefits of what ZFS offers. Rather than using dmcrypt and LUKS, which would bypass a lot of the features ZFS brings to the table, encryptfs is our ticket. The reason this is so elegant, is because Oracle has not released the source code to ZFS after version 28. Version 32 contains the code to create native ZFS encrypted filesystems. So, we need to rely on a 3rd party utility.

First, create your ZPOOL:

# zpool create rpool raidz1 sdb sdc sdd sde sdf

Then create your ZFS filesystem:

# zfs create rpool/private

Lastly, install the ecryptfs software, and make the encrypted filesystem by mounting it, and follow the prompts:

# mount -t ecryptfs /rpool/private /rpool/private
Select key type to use for newly created files: 
 1) tspi
 2) passphrase
Selection: 2
Passphrase: 
Select cipher: 
 1) aes: blocksize = 16; min keysize = 16; max keysize = 32
 2) blowfish: blocksize = 8; min keysize = 16; max keysize = 56
 3) des3_ede: blocksize = 8; min keysize = 24; max keysize = 24
 4) twofish: blocksize = 16; min keysize = 16; max keysize = 32
 5) cast6: blocksize = 16; min keysize = 16; max keysize = 32
 6) cast5: blocksize = 8; min keysize = 5; max keysize = 16
Selection [aes]: 
Select key bytes: 
 1) 16
 2) 32
 3) 24
Selection [16]: 
Enable plaintext passthrough (y/n) [n]: 
Enable filename encryption (y/n) [n]: y
Filename Encryption Key (FNEK) Signature [53aad9b192678a8a]: 
Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_fnek_sig=53aad9b192678a8a
  ecryptfs_key_bytes=16
  ecryptfs_cipher=aes
  ecryptfs_sig=53aad9b192678a8a
Mounted eCryptfs

Notice that I enabled filename encryption, as I don't want anyone getting any of my USB drives to decipher what I'm trying to hide. This will mount the encrypted filesystem "on top" of the ZFS filesystem, allowing you to keep all the COW and error correcting goodness, while keeping your data 100% safe:

# mount | grep rpool
rpool on /pool type zfs (rw,relatime,xattr)
rpool/private on /rpool/private type zfs (rw,relatime,xattr)
/rpool/private on /rpool/private type ecryptfs (rw,relatime,ecryptfs_fnek_sig...(snip))

Works like a charm.

{ 17 } Comments

  1. Loïc using Firefox 14.0.1 on Ubuntu 64 bits | August 22, 2012 at 2:06 am | Permalink

    Seems great! :)

    But I don’t think dedup and/or compression will continue to work as they did before, no?

  2. Aaron Toponce using Debian IceWeasel 10.0.6 on GNU/Linux 64 bits | August 22, 2012 at 3:24 am | Permalink

    Yes they will. Of course they will. The eCryptfs filesystem resides "on top" of ZFS. So, as far as ZFS is concerned, it's just standard, run-of-the-mill data. It doesn't know any different. So, all of the benefits ZFS brings are still there. No compromises are made. Unlike ZFS on top of LUKS.

  3. Loïc using Firefox 14.0.1 on Ubuntu 64 bits | August 22, 2012 at 8:40 am | Permalink

    Ok then, seems really great :)

  4. Brozer using Safari 534.57.2 on Mac OS | August 24, 2012 at 2:06 pm | Permalink

    Is it really correct to say there are no compromises at all? Granted, any exceptions would be somewhat minimal, and without source access we don't know exactly how Oracle has chosen to implement encryption, BPR etc. But in principle it seems like some ZFS operations would work better if performed on the data as a step before encryption in the chain. It seems likely that dedup, to take Loïc's example, would be more effective if it was done and then the result was randomized rather then after the result was randomized. Compression would be useless (though that of course can get handled farther up) and I don't know if snapshots would be quite as efficient.

    It's true that it's mostly transparent though, but in the end I still hope the illumos team comes through with their own native implementation as well as the improvements suggested by Delphix.

  5. Aaron Toponce using Debian IceWeasel 10.0.6 on GNU/Linux 64 bits | August 25, 2012 at 8:29 am | Permalink

    Deduplication happens at the block level, beneath ZFS. Regardless if the data is encrypted or not, deduplication is still possible. Now, granted, encrypting very similar, yet different data, can result in highly pseudorandom data. This can make data deduplication very difficult, but that doesn't change that with version 32, it's still the same problem, as it uses 256-bit AES. Same is said for compression. So, it's not eCryptFS that is causing these issues, but the disagreement in the technologies. These are the same issues you'll face with the latest version from Oracle. Regardless, the point is that ZFS knows nothing about the data it receives, only what it can do with it after the fact.

  6. Rudd-O using Safari 534.30 on Android | October 22, 2012 at 12:03 pm | Permalink

    I don't understand what compromises you make with LUKS + ZFS. I have used that and there are no compromises. ZFS works very well with it.

  7. Jonny using Google Chrome 20.0.1132.47 on Ubuntu 64 bits | October 23, 2012 at 7:31 am | Permalink

    Deduplication will work in that you can turn it on, but you wont find any duplicate data.

    Encrypting duplicate files/blocks will result in different data even when using the same key, since the Initialisation Vectors will be different.

    In ZFS they got around this by changing the way the IV was calculated when dedup is turned on (see this blog post: https://blogs.oracle.com/darren/entry/compress_encrypt_checksum_deduplicate_with).

    So as far as I can tell using 3rd party encryption will make dedup pointless

  8. Aaron Toponce using Debian IceWeasel 10.0.7 on GNU/Linux 64 bits | October 23, 2012 at 8:20 am | Permalink

    Not quite. The initialization vector is a nonce, and there is only one. It is what is responsible for kickstarting the CBC process. Further, deduplication sits underneath the encryption layer, not on top. At that level, the blocks are nothing more than zeros and ones. Now, granted, encryption algorithms are designed to mimic random data, and it is very hard to compress or deduplicate random data. With that said, it can still be done, and it's works just fine.

  9. Anonymous using Google Chrome 24.0.1312.56 on Windows 7 | February 1, 2013 at 12:09 am | Permalink

    "Now, granted, encryption algorithms are designed to mimic random data, and it is very hard to compress or deduplicate random data. With that said, it can still be done, and it’s works just fine."

    If you can compress your encrypted data, the encryption algorithm is egregiously leaking data. Same goes for deduplication - a crypto implementation that allows one to tell from ciphertext only that the plaintexts are the same is broken

  10. MN using Firefox 19.0 on Mac OS | April 6, 2013 at 1:42 pm | Permalink

    Hi Aaron,
    Can you help clarify what you mean by "using dmcrypt and LUKS, which would bypass a lot of the features ZFS brings to the table"? As I understand it, using LUKS would just change the underlying raw device to be an encrypted device. Since ZFS would then use that encrypted raw device, we would still be able to take advantage of all it's features. I probably am missing something so any clarification will help!

    thanks.

  11. EGL using Google Chrome 26.0.1410.64 on Windows 7 | May 16, 2013 at 6:12 am | Permalink

    Great article! But I'm very curious, as MN, which disadvantages you get when using LUKS below ZFS. As far as I can see, dedup and compression are going to work properly with LUKS as opposed to ecryptfs. What I'm not sure about is are things like 4k sectors and such...

    Thank you for clarification!

  12. Hiki using Opera 9.80 on GNU/Linux 64 bits | June 25, 2013 at 7:09 am | Permalink

    You can't share an ecrypfs mount over the network though, smb, nfs etc won't work (sftp does work).

  13. Michael using Google Chrome 29.0.1547.59 on Android | August 31, 2013 at 3:55 pm | Permalink

    Im very interested in the drawbacks of using LUKS underneath ZFS as well! Please enlighten us as this is somewhat of a dream setup for many linux users. Thanks in advance!

  14. crypt0s using Firefox 26.0 on GNU/Linux 64 bits | December 28, 2013 at 3:35 am | Permalink

    "...which would bypass a lot of the features ZFS brings to the table..."

    for a ZFS mirror:
    - cpu load is doubled
    - extra attack vector (doubled data)

    ...what are the other problems to watch for?

  15. Josh Enders using Safari 9537.53 on Mac OS | February 23, 2014 at 4:00 am | Permalink

    Hey Aaron, thanks for the post and series on ZFS. After testing eCryptfs with ZFS, I have unfortunately found a major trade-off that wasn't mentioned. When filename encryption is enabled, as is suggested, there is a limit posed on pre-encrypted filenames to be less than 143 characters in length. [1] Filenames longer than this produce an encrypted name that exceeds the length a ZFS filesystem can store for a filename (255 bytes) and worse yet, results in an unencrypted file.

    [1] http://unix.stackexchange.com/questions/32795/what-is-the-maximum-allowed-filename-and-folder-size-with-ecryptfs

  16. Matt using Firefox 24.0 on Windows 7 | April 12, 2014 at 12:05 am | Permalink

    Aaron, could you state which features of ZFS are bypassed if you use ZFS on top of a LUKS block device? I can think of none. However, using encryptfs on top of ZFS will prevent effective use of ZFS compression and/or deduplication.

  17. Aaron Toponce using Google Chrome 33.0.1750.152 on GNU/Linux 64 bits | April 12, 2014 at 3:35 pm | Permalink

    It's not that features are bypassed. It's that you can no longer guarantee file integrity. Putting LUKS between the disk and ZFS opens up the possibility that data could get corrupt between LUKS and the disk, and ZFS would not know about it. the ZFS developers spent years closing all the bit rot holes. Putting LUKS underneath ZFS opens one up.

{ 2 } Trackbacks

  1. [...] http://pthree.org/2012/08/21/encrypted-zfs-filesystems-on-linux/ Leave a Comment TrackBack URI [...]

  2. [...] http://pthree.org/2012/08/21/encrypted-zfs-filesystems-on-linux/ [...]

Post a Comment

Your email is never published nor shared.

Switch to our mobile site