Using Spark SQL in Hadoop.

If we use Spark SQL do we still need a Hbase or MongoDB databases? Can`t we connect directly to HDFS?

1 Answer(s)


hi Trinadh,

The use case for Hbase, MongoDB, Spark SQL are different as well as HDFS. HDFS is a distributed storage layer. Hbase is used for low latency queries specially suited for Customer facing tools like REST APIs, Mongo DB is used to store documents (in billions) and query them within a few seconds.