My Amazon EMR cluster uses a security configuration that has in-transit data encryption enabled. When I try to access Amazon Simple Storage Service (Amazon S3) buckets on the cluster using Hue (Hadoop User Experience), I get an error.
Resolution
If you get one of the following errors, then see the related section for troubleshooting steps:
- "Cannot access: s3a://."
- "Cannot access: s3a://. The HDFS REST service is not available."
- "bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')])"
"Cannot access: s3a://." and "Cannot access: s3a://. The HDFS REST service is not available."
When you launch an encrypted cluster, Amazon EMR expects all connections to be secure. However, the default protocol is HTTP, not HTTPS. To resolve these errors:
1. Open the /etc/hue/conf/hue.ini file.
2. In the webhdfs_url property, be sure that the protocol is https and that the port is 50470. Example:
webhdfs_url = https://master-node-private-ip-address:50470/webhdfs/v1
3. Run the following commands to restart the Hue service:
$ sudo stop hue
$ sudo start hue
"bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')])"
If you're using a self-signed certificate, you might also get the error "bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')])." This happens when Hue tries to verify the certificate with a certificate authority (CA), which isn't possible when you use a self-signed certificate. To resolve this error:
1. Open the /etc/hue/conf/hue.ini file.
2. Change the ssl_cert_ca_verify property from true to false:
ssl_cert_ca_verify=false
3. Run the following commands to restart the Hue service:
$ sudo stop hue
$ sudo start hue
Related information
Create keys and certificates for data encryption
Hue