PIG - Unable to STORE in local filesystem



0
Running the wordcount example in PIG. Able to run all commands successfully. Even dump gives the correct output but when running STORE command getting an error. Below are the steps taken to run the wordcount example.

[cloudera@localhost /]$ pig -x local
14/11/07 21:08:26 WARN pig.Main: Cannot write to log file: //pig_1415423306392.log
2014-11-07 21:08:26,398 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.0-cdh4.7.0 (rexported) compiled May 28 2014, 11:05:48
2014-11-07 21:08:26,431 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/cloudera/.pigbootup not found
2014-11-07 21:08:26,744 [main] WARN org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-11-07 21:08:26,747 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
2014-11-07 21:08:27,437 [main] WARN org.apache.hadoop.conf.Configuration - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2014-11-07 21:08:27,446 [main] WARN org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
grunt> lines = LOAD '/home/cloudera/data/wordcount' using PigStorage('\n') AS (line:chararray);
2014-11-07 21:08:31,587 [main] WARN org.apache.hadoop.conf.Configuration - dfs.umaskmode is deprecated. Instead, use fs.permissions.umask-mode
2014-11-07 21:08:31,587 [main] WARN org.apache.hadoop.conf.Configuration - topology.node.switch.mapping.impl is deprecated. Instead, use net.topology.node.switch.mapping.impl
2014-11-07 21:08:31,588 [main] WARN org.apache.hadoop.conf.Configuration - dfs.df.interval is deprecated. Instead, use fs.df.interval
2014-11-07 21:08:31,593 [main] WARN org.apache.hadoop.conf.Configuration - topology.script.number.args is deprecated. Instead, use net.topology.script.number.args
2014-11-07 21:08:31,593 [main] WARN org.apache.hadoop.conf.Configuration - hadoop.native.lib is deprecated. Instead, use io.native.lib.available
grunt> tokens = FOREACH lines GENERATE flatten(TOKENIZE(line)) AS token:chararray;
grunt> tokenGroup = GROUP tokens BY token;
grunt> countToken = FOREACH tokenGroup GENERATE group, COUNT(tokens);
grunt> dump countToken
2014-11-07 21:08:51,026 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: GROUP_BY
2014-11-07 21:08:51,446 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2014-11-07 21:08:51,469 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer - Choosing to move algebraic foreach to combiner
2014-11-07 21:08:51,661 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2014-11-07 21:08:51,661 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2014-11-07 21:08:51,992 [main] WARN org.apache.hadoop.conf.Configuration - session.id is deprecated. Instead, use dfs.metrics.session-id
2014-11-07 21:08:51,996 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with processName=JobTracker, sessionId=
2014-11-07 21:08:52,105 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
2014-11-07 21:08:52,422 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2014-11-07 21:08:52,438 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
2014-11-07 21:08:52,446 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=204
2014-11-07 21:08:52,446 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
2014-11-07 21:08:52,564 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
2014-11-07 21:08:52,604 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
2014-11-07 21:08:52,604 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
2014-11-07 21:08:52,604 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Distributed cache not supported or needed in local mode. Setting key [pig.schematuple.local.dir] with code temp directory: /tmp/1415423332604-0
2014-11-07 21:08:53,085 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
2014-11-07 21:08:53,103 [JobControl] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-11-07 21:08:53,191 [JobControl] WARN org.apache.hadoop.mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
2014-11-07 21:08:53,197 [JobControl] WARN org.apache.hadoop.mapred.JobClient - No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
2014-11-07 21:08:53,269 [JobControl] WARN org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-11-07 21:08:53,327 [JobControl] WARN org.apache.hadoop.conf.Configuration - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2014-11-07 21:08:53,556 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2014-11-07 21:08:53,557 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
2014-11-07 21:08:53,595 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
2014-11-07 21:08:53,654 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2014-11-07 21:08:54,509 [Thread-4] INFO org.apache.hadoop.mapred.LocalJobRunner - OutputCommitter set in config null
2014-11-07 21:08:54,605 [Thread-4] WARN org.apache.hadoop.conf.Configuration - dfs.df.interval is deprecated. Instead, use fs.df.interval
2014-11-07 21:08:54,605 [Thread-4] WARN org.apache.hadoop.conf.Configuration - hadoop.native.lib is deprecated. Instead, use io.native.lib.available
2014-11-07 21:08:54,605 [Thread-4] WARN org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-11-07 21:08:54,605 [Thread-4] WARN org.apache.hadoop.conf.Configuration - topology.script.number.args is deprecated. Instead, use net.topology.script.number.args
2014-11-07 21:08:54,608 [Thread-4] WARN org.apache.hadoop.conf.Configuration - dfs.umaskmode is deprecated. Instead, use fs.permissions.umask-mode
2014-11-07 21:08:54,608 [Thread-4] WARN org.apache.hadoop.conf.Configuration - topology.node.switch.mapping.impl is deprecated. Instead, use net.topology.node.switch.mapping.impl
2014-11-07 21:08:54,608 [Thread-4] WARN org.apache.hadoop.conf.Configuration - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2014-11-07 21:08:54,614 [Thread-4] INFO org.apache.hadoop.mapred.LocalJobRunner - OutputCommitter is org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter
2014-11-07 21:08:54,693 [Thread-4] INFO org.apache.hadoop.mapred.LocalJobRunner - Waiting for map tasks
2014-11-07 21:08:54,694 [pool-1-thread-1] INFO org.apache.hadoop.mapred.LocalJobRunner - Starting task: attempt_local1737408954_0001_m_000000_0
2014-11-07 21:08:54,846 [pool-1-thread-1] WARN mapreduce.Counters - Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
2014-11-07 21:08:54,976 [pool-1-thread-1] INFO org.apache.hadoop.util.ProcessTree - setsid exited with exit code 0
2014-11-07 21:08:54,992 [pool-1-thread-1] INFO org.apache.hadoop.mapred.Task - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@67a5fb5a
2014-11-07 21:08:55,008 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local1737408954_0001
2014-11-07 21:08:55,008 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases countToken,lines,tokenGroup,tokens
2014-11-07 21:08:55,008 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: lines[1,8],tokens[-1,-1],countToken[4,13],tokenGroup[3,13] C: countToken[4,13],tokenGroup[3,13] R: countToken[4,13]
2014-11-07 21:08:55,011 [pool-1-thread-1] INFO org.apache.hadoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 204
Input split[0]:
Length = 204
Locations:

-----------------------

2014-11-07 21:08:55,022 [pool-1-thread-1] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader - Current split being processed file:/home/cloudera/data/wordcount:0+204
2014-11-07 21:08:55,035 [pool-1-thread-1] INFO org.apache.hadoop.mapred.MapTask - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-11-07 21:08:55,051 [pool-1-thread-1] INFO org.apache.hadoop.mapred.MapTask - io.sort.mb = 100
2014-11-07 21:08:55,094 [pool-1-thread-1] INFO org.apache.hadoop.mapred.MapTask - data buffer = 79691776/99614720
2014-11-07 21:08:55,094 [pool-1-thread-1] INFO org.apache.hadoop.mapred.MapTask - record buffer = 262144/327680
2014-11-07 21:08:55,120 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.max.objects is deprecated. Instead, use dfs.namenode.max.objects
2014-11-07 21:08:55,120 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.data.dir is deprecated. Instead, use dfs.datanode.data.dir
2014-11-07 21:08:55,120 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.name.dir is deprecated. Instead, use dfs.namenode.name.dir
2014-11-07 21:08:55,120 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - fs.checkpoint.dir is deprecated. Instead, use dfs.namenode.checkpoint.dir
2014-11-07 21:08:55,120 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.block.size is deprecated. Instead, use dfs.blocksize
2014-11-07 21:08:55,120 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.access.time.precision is deprecated. Instead, use dfs.namenode.accesstime.precision
2014-11-07 21:08:55,120 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.replication.min is deprecated. Instead, use dfs.namenode.replication.min
2014-11-07 21:08:55,120 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.name.edits.dir is deprecated. Instead, use dfs.namenode.edits.dir
2014-11-07 21:08:55,120 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.replication.considerLoad is deprecated. Instead, use dfs.namenode.replication.considerLoad
2014-11-07 21:08:55,120 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.balance.bandwidthPerSec is deprecated. Instead, use dfs.datanode.balance.bandwidthPerSec
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.safemode.threshold.pct is deprecated. Instead, use dfs.namenode.safemode.threshold-pct
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.http.address is deprecated. Instead, use dfs.namenode.http-address
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.name.dir.restore is deprecated. Instead, use dfs.namenode.name.dir.restore
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.https.client.keystore.resource is deprecated. Instead, use dfs.client.https.keystore.resource
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.backup.address is deprecated. Instead, use dfs.namenode.backup.address
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.backup.http.address is deprecated. Instead, use dfs.namenode.backup.http-address
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.permissions is deprecated. Instead, use dfs.permissions.enabled
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.safemode.extension is deprecated. Instead, use dfs.namenode.safemode.extension
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.datanode.max.xcievers is deprecated. Instead, use dfs.datanode.max.transfer.threads
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.https.need.client.auth is deprecated. Instead, use dfs.client.https.need-auth
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.https.address is deprecated. Instead, use dfs.namenode.https-address
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.replication.interval is deprecated. Instead, use dfs.namenode.replication.interval
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - fs.checkpoint.edits.dir is deprecated. Instead, use dfs.namenode.checkpoint.edits.dir
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.write.packet.size is deprecated. Instead, use dfs.client-write-packet-size
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.permissions.supergroup is deprecated. Instead, use dfs.permissions.superusergroup
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - dfs.secondary.http.address is deprecated. Instead, use dfs.namenode.secondary.http-address
2014-11-07 21:08:55,121 [pool-1-thread-1] WARN org.apache.hadoop.conf.Configuration - fs.checkpoint.period is deprecated. Instead, use dfs.namenode.checkpoint.period
2014-11-07 21:08:55,182 [pool-1-thread-1] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
2014-11-07 21:08:55,227 [pool-1-thread-1] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map - Aliases being processed per job phase (AliasName[line,offset]): M: lines[1,8],tokens[-1,-1],countToken[4,13],tokenGroup[3,13] C: countToken[4,13],tokenGroup[3,13] R: countToken[4,13]
2014-11-07 21:08:55,292 [pool-1-thread-1] INFO org.apache.hadoop.mapred.LocalJobRunner -
2014-11-07 21:08:55,293 [pool-1-thread-1] INFO org.apache.hadoop.mapred.MapTask - Starting flush of map output
2014-11-07 21:08:55,362 [pool-1-thread-1] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine - Aliases being processed per job phase (AliasName[line,offset]): M: lines[1,8],tokens[-1,-1],countToken[4,13],tokenGroup[3,13] C: countToken[4,13],tokenGroup[3,13] R: countToken[4,13]
2014-11-07 21:08:55,377 [pool-1-thread-1] INFO org.apache.hadoop.mapred.MapTask - Finished spill 0
2014-11-07 21:08:55,381 [pool-1-thread-1] INFO org.apache.hadoop.mapred.Task - Task:attempt_local1737408954_0001_m_000000_0 is done. And is in the process of commiting
2014-11-07 21:08:55,410 [pool-1-thread-1] INFO org.apache.hadoop.mapred.LocalJobRunner -
2014-11-07 21:08:55,410 [pool-1-thread-1] INFO org.apache.hadoop.mapred.Task - Task 'attempt_local1737408954_0001_m_000000_0' done.
2014-11-07 21:08:55,410 [pool-1-thread-1] INFO org.apache.hadoop.mapred.LocalJobRunner - Finishing task: attempt_local1737408954_0001_m_000000_0
2014-11-07 21:08:55,415 [Thread-4] INFO org.apache.hadoop.mapred.LocalJobRunner - Map task executor complete.
2014-11-07 21:08:55,434 [Thread-4] WARN mapreduce.Counters - Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
2014-11-07 21:08:55,488 [Thread-4] INFO org.apache.hadoop.mapred.Task - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7f0ab78a
2014-11-07 21:08:55,489 [Thread-4] INFO org.apache.hadoop.mapred.LocalJobRunner -
2014-11-07 21:08:55,498 [Thread-4] INFO org.apache.hadoop.mapred.Merger - Merging 1 sorted segments
2014-11-07 21:08:55,513 [Thread-4] INFO org.apache.hadoop.mapred.Merger - Down to the last merge-pass, with 1 segments left of total size: 460 bytes
2014-11-07 21:08:55,513 [Thread-4] INFO org.apache.hadoop.mapred.LocalJobRunner -
2014-11-07 21:08:55,544 [Thread-4] WARN org.apache.pig.data.SchemaTupleBackend - SchemaTupleBackend has already been initialized
2014-11-07 21:08:55,583 [Thread-4] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce - Aliases being processed per job phase (AliasName[line,offset]): M: lines[1,8],tokens[-1,-1],countToken[4,13],tokenGroup[3,13] C: countToken[4,13],tokenGroup[3,13] R: countToken[4,13]
2014-11-07 21:08:55,590 [Thread-4] INFO org.apache.hadoop.mapred.Task - Task:attempt_local1737408954_0001_r_000000_0 is done. And is in the process of commiting
2014-11-07 21:08:55,601 [Thread-4] INFO org.apache.hadoop.mapred.LocalJobRunner -
2014-11-07 21:08:55,601 [Thread-4] INFO org.apache.hadoop.mapred.Task - Task attempt_local1737408954_0001_r_000000_0 is allowed to commit now
2014-11-07 21:08:55,607 [Thread-4] INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - Saved output of task 'attempt_local1737408954_0001_r_000000_0' to file:/tmp/temp-949352641/tmp-1819209615
2014-11-07 21:08:55,616 [Thread-4] INFO org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce
2014-11-07 21:08:55,616 [Thread-4] INFO org.apache.hadoop.mapred.Task - Task 'attempt_local1737408954_0001_r_000000_0' done.
2014-11-07 21:08:59,528 [main] WARN org.apache.pig.tools.pigstats.PigStatsUtil - Failed to get RunningJob for job job_local1737408954_0001
2014-11-07 21:08:59,536 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2014-11-07 21:08:59,536 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode. Stats reported below may be incomplete
2014-11-07 21:08:59,545 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:

HadoopVersion PigVersion UserId StartedAt FinishedAt Features
2.0.0-cdh4.7.0 0.11.0-cdh4.7.0 cloudera 2014-11-07 21:08:52 2014-11-07 21:08:59 GROUP_BY

Success!

Job Stats (time in seconds):
JobId Alias Feature Outputs
job_local1737408954_0001 countToken,lines,tokenGroup,tokens GROUP_BY,COMBINER file:/tmp/temp-949352641/tmp-1819209615,

Input(s):
Successfully read records from: "/home/cloudera/data/wordcount"

Output(s):
Successfully stored records in: "file:/tmp/temp-949352641/tmp-1819209615"

Job DAG:
job_local1737408954_0001


2014-11-07 21:08:59,545 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
2014-11-07 21:08:59,549 [main] WARN org.apache.pig.data.SchemaTupleBackend - SchemaTupleBackend has already been initialized
2014-11-07 21:08:59,553 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2014-11-07 21:08:59,553 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
(I,1)
(a,3)
(do,1)
(in,1)
(is,2)
(of,1)
(to,5)
(PIG,1)
(Try,1)
(for,1)
(lot,2)
(run,1)
(the,3)
(try,1)
(Need,1)
(This,1)
(down,1)
(file,1)
(hope,1)
(note,1)
(test,1)
(this,1)
(with,1)
(Still,1)
(ideas,1)
(using,1)
(hadoop,1)
(example,1)
(examples,2)
(practice,1)
(wordcount,1)
(successful,1)
grunt> STORE countToken INTO '/home/cloudera/data/countToken_wc';
2014-11-07 21:09:08,144 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: GROUP_BY
2014-11-07 21:09:08,157 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2014-11-07 21:09:08,165 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer - Choosing to move algebraic foreach to combiner
2014-11-07 21:09:08,177 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2014-11-07 21:09:08,177 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2014-11-07 21:09:08,180 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-11-07 21:09:08,184 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
2014-11-07 21:09:08,220 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2014-11-07 21:09:08,220 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
2014-11-07 21:09:08,222 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=204
2014-11-07 21:09:08,222 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
2014-11-07 21:09:08,245 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
2014-11-07 21:09:08,247 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
2014-11-07 21:09:08,247 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
2014-11-07 21:09:08,247 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Distributed cache not supported or needed in local mode. Setting key [pig.schematuple.local.dir] with code temp directory: /tmp/1415423348246-0
2014-11-07 21:09:08,412 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
2014-11-07 21:09:08,413 [JobControl] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-11-07 21:09:08,416 [JobControl] WARN org.apache.hadoop.mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
2014-11-07 21:09:08,418 [JobControl] WARN org.apache.hadoop.mapred.JobClient - No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
2014-11-07 21:09:08,463 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2014-11-07 21:09:08,463 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
2014-11-07 21:09:08,474 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2014-11-07 21:09:08,724 [Thread-11] INFO org.apache.hadoop.mapred.LocalJobRunner - OutputCommitter set in config null
2014-11-07 21:09:08,746 [Thread-11] WARN org.apache.hadoop.conf.Configuration - dfs.df.interval is deprecated. Instead, use fs.df.interval
2014-11-07 21:09:08,746 [Thread-11] WARN org.apache.hadoop.conf.Configuration - hadoop.native.lib is deprecated. Instead, use io.native.lib.available
2014-11-07 21:09:08,746 [Thread-11] WARN org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-11-07 21:09:08,747 [Thread-11] WARN org.apache.hadoop.conf.Configuration - topology.script.number.args is deprecated. Instead, use net.topology.script.number.args
2014-11-07 21:09:08,747 [Thread-11] WARN org.apache.hadoop.conf.Configuration - dfs.umaskmode is deprecated. Instead, use fs.permissions.umask-mode
2014-11-07 21:09:08,747 [Thread-11] WARN org.apache.hadoop.conf.Configuration - topology.node.switch.mapping.impl is deprecated. Instead, use net.topology.node.switch.mapping.impl
2014-11-07 21:09:08,748 [Thread-11] WARN org.apache.hadoop.conf.Configuration - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2014-11-07 21:09:08,758 [Thread-11] INFO org.apache.hadoop.mapred.LocalJobRunner - OutputCommitter is org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter
2014-11-07 21:09:08,775 [Thread-11] INFO org.apache.hadoop.mapred.LocalJobRunner - Waiting for map tasks
2014-11-07 21:09:08,775 [pool-4-thread-1] INFO org.apache.hadoop.mapred.LocalJobRunner - Starting task: attempt_local1054838449_0002_m_000000_0
2014-11-07 21:09:08,775 [pool-4-thread-1] WARN mapreduce.Counters - Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
2014-11-07 21:09:08,816 [pool-4-thread-1] INFO org.apache.hadoop.mapred.Task - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4e94a28e
2014-11-07 21:09:08,826 [pool-4-thread-1] INFO org.apache.hadoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 204
Input split[0]:
Length = 204
Locations:

-----------------------

2014-11-07 21:09:08,838 [pool-4-thread-1] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader - Current split being processed file:/home/cloudera/data/wordcount:0+204
2014-11-07 21:09:08,842 [pool-4-thread-1] INFO org.apache.hadoop.mapred.MapTask - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-11-07 21:09:08,842 [pool-4-thread-1] INFO org.apache.hadoop.mapred.MapTask - io.sort.mb = 100
2014-11-07 21:09:09,232 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local1054838449_0002
2014-11-07 21:09:09,232 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases countToken,lines,tokenGroup,tokens
2014-11-07 21:09:09,232 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: lines[1,8],tokens[-1,-1],countToken[4,13],tokenGroup[3,13] C: countToken[4,13],tokenGroup[3,13] R: countToken[4,13]
2014-11-07 21:09:09,234 [Thread-11] INFO org.apache.hadoop.mapred.LocalJobRunner - Map task executor complete.
2014-11-07 21:09:09,243 [Thread-11] WARN org.apache.hadoop.mapred.LocalJobRunner - job_local1054838449_0002
java.lang.Exception: java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:406)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:826)
at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:376)
at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:85)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:584)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:656)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:268)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2014-11-07 21:09:09,276 [Low Memory Detector] INFO org.apache.pig.impl.util.SpillableMemoryManager - first memory handler call - Collection threshold init = 178978816(174784K) used = 109724616(107152K) committed = 178978816(174784K) max = 178978816(174784K)
2014-11-07 21:09:09,308 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
2014-11-07 21:09:13,818 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
2014-11-07 21:09:13,818 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_local1054838449_0002 has failed! Stop running all dependent jobs
2014-11-07 21:09:13,818 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2014-11-07 21:09:13,819 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
2014-11-07 21:09:13,819 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode. Stats reported below may be incomplete
2014-11-07 21:09:13,821 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:

HadoopVersion PigVersion UserId StartedAt FinishedAt Features
2.0.0-cdh4.7.0 0.11.0-cdh4.7.0 cloudera 2014-11-07 21:09:08 2014-11-07 21:09:13 GROUP_BY

Failed!

Failed Jobs:
JobId Alias Feature Message Outputs
job_local1054838449_0002 countToken,lines,tokenGroup,tokens GROUP_BY,COMBINER Message: Job failed! /home/cloudera/data/countToken_wc,

Input(s):
Failed to read data from "/home/cloudera/data/wordcount"

Output(s):
Failed to produce result in "/home/cloudera/data/countToken_wc"

Job DAG:
job_local1054838449_0002


2014-11-07 21:09:13,821 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
grunt>

1 Answer(s)


0

hi Keith,

Looking at the lines , its unclear what is wrong here. Request you to exit the PIG shell, re-execute and see if you are still getting the errors.

Thanks