1 Answer
- Newest
- Most votes
- Most comments
0
Input/Output errors almost always indicate a network connection problem between the DataSync agent and HDFS NameNode or HDFS DataNodes. You can check this here:
You need to validate the namenode metadata service port (IPC port) set on the core-site.xml under the following property: fs.defaultFS or fs.default.name, depending upon your Hadoop distribution.
$ grep -A 2 -B 1 -i fs.default core-site.xml
<property>
<name>fs.defaultFS</name>
<value>hdfs://ip-xx-xx-xx-xx.ec2.internal:8020</value>
</property>
You must also validate the datanode port setting under dfs.datanode.address property.
$ grep -A 2 -B 1 -i dfs.datanode.address hdfs-site.xml
<property>
<name>dfs.datanode.address</name>
<value>xxx.xxx.xxx.xxx:1004</value>
</property>
Are you using Simple Authentication or Kerberos Authentication?
answered 2 years ago
Relevant content
- asked 8 months ago
- asked 4 days ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 7 months ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 3 years ago
I am using Simple authentication.