How to take a source file and output file in text format in HDFS
This recipe helps you take a source file and output the file in text format in HDFS. The Hadoop fs shell command text takes the source file and outputs the file in the text format. It detects the encoding of the file and decodes it to plain text.
Recipe Objective: How to take a source file and output the file in text format in HDFS?
Let us see how to take a source file and output the file in text format in the HDFS.
Access Snowflake Real-Time Project to Implement SCD's
Table of Contents
Prerequisites:
Before proceeding with the recipe, make sure Single node Hadoop is installed on your local EC2 instance. If not already installed, follow the below link to do the same.
Steps to set up an environment:
- In the AWS, create an EC2 instance and log in to Cloudera Manager with your public IP mentioned in the EC2 instance. Login to putty/terminal and check if HDFS is installed. If not installed, please find the links provided above for installations.
- Type “<your public IP>:7180” in the web browser and log in to Cloudera Manager, where you can check if Hadoop is installed.
- If they are not visible in the Cloudera cluster, you may add them by clicking on the “Add Services” in the cluster to add the required services in your local instance.
Taking a source file and output the file in text format in the HDFS:
The Hadoop fs shell command text takes the source file and outputs the file in the text format. It detects the encoding of the file and decodes it to plain text. The syntax for the command is:
hadoop fs -text <source file>
In this recipe, we read the zip file named “upgrad” present in the root user and are reading the output in the text format.
Command output:
What Users are saying..
Anand Kumpatla
ProjectPro is a unique platform and helps many people in the industry to solve real-life problems with a step-by-step walkthrough of projects. A platform with some fantastic resources to gain... Read More