Create encrypted zones in HDFS

To create an HDFS encryption zone first you need to set up HDFS Data at Rest encryption service. For Cloudera distribution follow below:

Select Service.

For the Production cluster, it is recommended to use HSM KMS backed setup but for testing here we are going to select file-based Java Key-store which is not required Kerberos setup and easy to start. So select below and provide server cloudera-node1 where this service will be running.

Provide ACL admin user and group:

This will complete setup:

Verify KMS is setup to encrypt files.

[hasnain@cloudera-node1 ~]$ hadoop key create mykey1

mykey1 has been successfully created with options Options{cipher='AES/CTR/NoPadding', bitLength=128, description='null', attributes=null}.

org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider@3098cf3b has been updated.

[hasnain@cloudera-node1 ~]$ hadoop key list

Listing keys for KeyProvider: org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider@61322f9d

mykey1

[hasnain@cloudera-node1 ~]$ hadoop fs -mkdir /tmp/zone1

Create a zone and link to the key.

[hasnain@cloudera-node1 ~]$ sudo -su hdfs

[hdfs@ cloudera-node1 hasnain]$ hdfs crypto -createZone -keyName mykey1 -path /tmp/zone1

Added encryption zone /tmp/zone1

[hdfs@cloudera-node1 hasnain]$ hdfs crypto -listZones

/tmp/zone1  mykey1

Create a file, put it in your zone and ensure the file can be decrypted.

su <KEY_ADMIN_USER>

[hasnain@cloudera-node1 ~]$ echo "Hello World" > /tmp/helloWorld.txt

[hasnain@cloudera-node1 ~]$ hadoop fs -put /tmp/helloWorld.txt /tmp/zone1

[hasnain@cloudera-node1 ~]$ hadoop fs -cat /tmp/zone1/helloWorld.txt

Hello World

[hasnain@cloudera-node1 ~]$ rm /tmp/helloWorld.txt

Ensure the file is stored as encrypted.

[root@cloudera-node1 hasnain]# hadoop fs -cat /tmp/zone1/helloWorld.txt

cat: User [root] is not authorized to perform [DECRYPT_EEK] on key with ACL name [mykey1]!!

su hdfs

[hasnain@cloudera-node1 ~]$ sudo -su hdfs

[hdfs@cloudera-node1 hasnain]$ hadoop fs -cat /.reserved/raw/tmp/zone1/helloWorld.txt

B▒▒)▒-▒η▒

hadoop fs -rm -R /tmp/zone1

By default, non-admin users cannot access any encrypted data. You must create appropriate ACLs before users can access encrypted data. See the Cloudera documentation for more information on managing KMS ACLs.