error "value words is not a member of org.apache.spark.SparkContext)

Eva

i am getting that error in spark-shell when i run val words = sc.words.txt("hdfs://192.168.75.133:9000/words/sh.txt")"

error "value words is not a member of org.apache.spark.SparkContext)?

How do I fix this? Please see attached core-site.xml file and spark-shell.

Thanks.

Spark

Oct 31 2016 02:05 AM

3 Answer(s)

Abhijit-Dezyre Support

Hi Eva,

Please use the correct line:

val words = sc.words("hdfs://192.168.75.133:9000/words/sh.txt");

Hope this helps.

Thanks.

Nov 01 2016 11:26 PM

Eva

Thanks but I am still getting the same error

Nov 06 2016 07:54 AM

Abhijit-Dezyre Support

Hi Eva,

Could you please share the screenshot, when you enter the above mention command.

Thanks

Nov 07 2016 02:03 AM

Relevant Projects

Perform Time series modelling using Facebook Prophet

In this project, we are going to talk about Time Series Forecasting to predict the electricity requirement for a particular house using Prophet.

PySpark Tutorial - Learn to use Apache Spark with Python

PySpark Project-Get a handle on using Python with Spark through this hands-on data processing spark python tutorial.

Explore features of Spark SQL in practice on Spark 2.0

The goal of this spark project for students is to explore the features of Spark SQL in practice on the latest version of Spark i.e. Spark 2.0.

Data Science Project - Instacart Market Basket Analysis

Data Science Project - Build a recommendation engine which will predict the products to be purchased by an Instacart consumer again.

Forecast Inventory demand using historical sales data in R

In this machine learning project, you will develop a machine learning model to accurately forecast inventory demand based on historical sales data.

Movielens dataset analysis for movie recommendations using Spark in Azure

In this Databricks Azure tutorial project, you will use Spark Sql to analyse the movielens dataset to provide movie recommendations. As part of this you will deploy Azure data factory, data pipelines and visualise the analysis.

Online Hadoop Projects -Solving small file problem in Hadoop

In this hadoop project, we are going to be continuing the series on data engineering by discussing and implementing various ways to solve the hadoop small file problem.

Tough engineering choices with large datasets in Hive Part - 2

This is in continuation of the previous Hive project "Tough engineering choices with large datasets in Hive Part - 1", where we will work on processing big data sets using Hive.