Loading data using Pig
6 Answer(s)
Abhijit-Dezyre Support
Hi Sugandha,
Please change the user as hduser: su - hduser
You can save your file under - /user/hduser/
Hope this helps.
Thanks
Feb 10 2016 06:06 PM
Please change the user as hduser: su - hduser
You can save your file under - /user/hduser/
Hope this helps.
Thanks
Sugandha
Hi Abhijit
In WinScp unable to login as user : su - hduser. Access denied.
In AWS already logged in as user : su - hduser
Feb 10 2016 07:46 PM
In WinScp unable to login as user : su - hduser. Access denied.
In AWS already logged in as user : su - hduser
Sugandha
Please check if hostname also needs to be modified while logging in to Winscp.
Feb 10 2016 07:48 PM
Abhijit-Dezyre Support
Hi Sugandha,
In case of Winscp, you don't have to switch to hduser.
You can upload your file, anywhere you want.
Thanks
Feb 10 2016 10:57 PM
In case of Winscp, you don't have to switch to hduser.
You can upload your file, anywhere you want.
Thanks
Sugandha
Hi Abhijit
I loaded the data with below command:
A = load 'emp.txt' using PigStorage(',') as (name:chararray,age:int,salary:int,dept_id:int);
And then on Dump A below error comes:
Looks like backend error.
Output(s):
Failed to produce result in "hdfs://ip-10-0-0-28.ec2.internal:8020/tmp/temp-291835293/tmp915702883"
Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_1453801387162_0022
2016-02-11 03:28:22,802 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
2016-02-11 03:28:22,805 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias A. Backend error : java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING
Details at logfile: /home/hduser/pig_1455179139165.log
Feb 11 2016 02:05 PM
I loaded the data with below command:
A = load 'emp.txt' using PigStorage(',') as (name:chararray,age:int,salary:int,dept_id:int);
And then on Dump A below error comes:
Looks like backend error.
Output(s):
Failed to produce result in "hdfs://ip-10-0-0-28.ec2.internal:8020/tmp/temp-291835293/tmp915702883"
Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_1453801387162_0022
2016-02-11 03:28:22,802 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
2016-02-11 03:28:22,805 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias A. Backend error : java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING
Details at logfile: /home/hduser/pig_1455179139165.log
Abhijit-Dezyre Support
Hi Sugandha,
One thing I would like to mention that "emp.txt" should be in hdfs before loading in PigStorage().
If I understand correctly, emp.txt is in /home/ec2-user/
Please upload the file in hdfs. To do that-
- su - hduser //Change user because it has permission to access /user/hduser/
- hadoop fs -put emp.txt /user/hduser/sugandha/emp.txt //Upload the file
Hope this helps. Please correct me if I understand it wrong.
Thanks.
Feb 12 2016 04:41 AM
One thing I would like to mention that "emp.txt" should be in hdfs before loading in PigStorage().
If I understand correctly, emp.txt is in /home/ec2-user/
Please upload the file in hdfs. To do that-
- su - hduser //Change user because it has permission to access /user/hduser/
- hadoop fs -put emp.txt /user/hduser/sugandha/emp.txt //Upload the file
Hope this helps. Please correct me if I understand it wrong.
Thanks.