Regarding destination file containing bytewritables


1 Answer(s)


Hi Venkatesh,
Please share the flume.conf with us to better understand the issue.
Here is the sample for flume.conf for local to hdfs:
-----------------------------------------------------------------
#Flume.conf
agent1.sources=src-1
agent1.channels=memory-channel
agent1.sinks=hdfs-sink

#describe and configure the sources
agent1.sources.src-1.type = spooldir
agent1.sources.src-1.spoolDir = /home/cloudera/flumeSpool
agent1.sources.src-1.fileHeader = true
agent1.sources.src-1.channels=memory-channel

#describe the channel
agent1.channels.memory-channel.type=memory
agent1.channels.memory-channel.capacity=10000
agent1.channels.memory-channel.transactionCapacity=100

#describe the sink properties
agent1.sinks.hdfs-sink.type=hdfs
agent1.sinks.hdfs-sink.channel=memory-channel
agent1.sinks.hdfs-sink.hdfs.path=/user/cloudera/flumeSpool
agent1.sinks.hdfs-sink.hdfs.fileType=DataStream
agent1.sinks.hdfs-sink.hdfs.writeFormat=Text
agent1.sinks.hdfs-sink.hdfs.rollCount=20
---------------------------------------------------------------------

Hope this helps
Thanks