1-844-696-6465 (US)        +91 77600 44484        help@dezyre.com

Scala Interview Questions and Answers for Spark Developers

Last Update Made on March 21, 2018

When choosing a programming language for big data applications, Python and R are the most preferred programming languages among data scientists and Java is the go -to language for developing applications on Hadoop. With the enterprise adoption of Scala based big data frameworks like Apache Kafka and Apache Spark- Scala is becoming popular among big data professionals. Any developer in the big data world should be smart enough to learn a programming language that has some complexity. The statement “Scala is hard to master” is definitely true to some extent but the learning curve of Scala for Spark is well worth the time and money. Complex programming features like Macros, Tuples and Functions make it easy for spark developers to write better code and improve performance by programming in Scala. If you are being interviewed for any of the big data job openings that require Apache Spark skills, then it is quite likely that you will be asked questions around Scala programming language as Spark is written in Scala. What follows is a list of commonly asked Scala interview questions for Spark jobs.

Top Scala Interview Questions and Answers

The most interesting part of learning Scala for Spark is the big data job trends. Consider all the popular functional programming languages supported by Apache Spark big data framework like Java, Python, R and Scala and look at the job trends. Of all the four programming languages supported by Spark, most of the big data job openings list Scala as a “must-have” programming skill for Apache Spark developers instead of Java, Python or R. Typesafe CEO Mark Brewer made a statement supporting the increasing demand for Scala developers –“We’re hearing about startups choosing Scala specifically because that’s where the best developers are now. We’ve also heard that Scala developers are consistently fetching $110K+ salaries, because they are in such high demand.”

For the complete list of big data companies and their salaries- CLICK HERE

Scala Interview Questions and Answers

1) What is a Scala Map?

Scala Map is a collection of key value pairs wherein the value in a map can be retrieved using the key. Values in a Scala Map are not unique but the keys are unique. Scala supports two kinds of maps- mutable and immutable. By default, Scala supports immutable map and to make use of the mutable map, programmers have to import the scala.collection.mutable.Map class explicitly. When programmers want to use mutable and immutable map together in the same program then the mutable map can be accessed as mutable.map and the immutable map can just be accessed with the name of the map.

Apache Spark Certification and Training

If you would like more information about Big Data careers, please click the orange "Request Info" button on top of this page.

2) What is the advantage of using Scala over other functional programming languages? 

  • As the name itself indicates Scala meaning Scalable Language, its high scalable, maintainability, productivity and testability features make it advantageous to use Scala.
  • Singleton and Companion Objects in Scala provide a cleaner solution unlike static in other JVM languages like Java.
  • It eliminates the need for having a ternary operator as if blocks’, ‘for-yield loops’, and ‘code’ in braces return a value in Scala.

3) What is the advantage of companion objects in Scala?

Classes in Scala programming language do not have static methods or variables but rather they have what is known as a Singleton object or Companion object. The companion objects in turn are compiled to classes which have static methods.

A singleton object in Scala is declared using the keyword object as shown below –

object Main {

    def sayHello () {

        println ("Hello!");

    }

}

In the above code snippet, Main is a singleton object and the method sayHello can be invoked using the following line of code –

Main. SayHello ();

If a singleton object has the same name as that of the class then it is known as a Companion object and it should be defined in the same source file as that of the class.

class Main {

    def sayHelloWorld() {

        println("Hello World");

    }

}

 

object Main {

    def sayHello() {

        println("Hello!");

    }

}

Advantages of Companion Objects in Scala

  • Companion objects are beneficial for encapsulating things and they act as a bridge for writing functional and object oriented programming code.
  • Using companion objects, the Scala programming code can be kept more concise as the static keyword need not be added to each and every attribute.
  • Companion objects provide a clear separation between static and non-static methods in a class because everything that is located inside a companion object is not a part of the class’s runtime objects but is available from a static context and vice versa.

4) Which Scala library is used for functional programming?

Scalaz library has purely functional data structures that complement the standard Scala library. It has pre-defined set of foundational type classes like Monad, Functor, etc.

5) What do you understand by “Unit” and “()” in Scala?

Unit is a subtype of scala.anyval and is nothing but Scala equivalent of Java void that provides the Scala with an abstraction of the java platform. Empty tuple i.e. () in Scala is a term that represents unit value.

6) What is the difference between concurrency and parallelism?

People often confuse with the terms concurrency and parallelism. When several computations execute sequentially during overlapping time periods it is referred to as concurrency whereas when processes are executed simultaneously it is known as parallelism. Parallel collection, Futures and Async library are examples of achieving parallelism in Scala.

7) What is a Monad in Scala?

The simplest way to define a monad is to relate it to a wrapper. Any class object is taken wrapped with a monad in Scala. Just like you wrap any gift or present into a shiny wrapper with ribbons to make them look attractive, Monads in Scala are used to wrap objects and provide two important operations –

  • Identity through “unit” in Scala
  • Bind through “flatMap” in Scala

8) Differentiate between Val and var in Scala.

Val and var are the two keywords used to define variables in Scala. Var keyword is just similar to variable declaration in Java whereas Val is little different. Once a variable is declared using Val the reference cannot be changed to point to another reference. This functionality of Val keyword in Scala can be related to the functionality of java final keyword. To simplify it, Val refers to immutable declaration of a variable whereas var refers to mutable declaration of a variable in Scala.

9) What do you understand by a closure in Scala?

Closure is a function in Scala where the return value of the function depends on the value of one or more variables that have been declared outside the function.

10) What is Scala Future? How it differs from java.util.concurrent.Future?

Scala Future is a monadic collection, which starts a background task. It is an object which holds the potential value or future value, which would be available after the task is completed. It also provides various operations to further chain the operations or to extract the value. Future also provide various call-back functions like onComplete, OnFailure, onSuccess to name a few, which makes Future a complete concurrent task class. The main and foremost difference between Scala’s Future and Java’s Future class is that the later does not provide promises/callbacks operations. The only way to retrieve the result is Future.get () in Java.

11) What is Option in Scala? Why would you use it?

It is used for representing whether a value is present or absent. Option collections can be used for wrapping missing values. It can also be seen as replacement for returning null values, which can be very helpful for reducing the occurrence of NullPointerException. The Option type itself is unimplemented but depends on two sub types: Some and None. 

What is an option in Scala

One more example to describe functionality of Option type is to use it as a method return type, which tells the caller that the method can return a string or it can return none.

Option in Scala

12) What’s the difference ‘Nil’, ‘Null’, ‘None’ and ’Nothing’ in Scala?

  • Null – It’s a sub-type of AnyRef type in Scala Types hierarchy. As Scala runs on JVM, it uses NULL to provide the compatibility with Java null keyword, or in Scala terms, to provide type for null keyword, Null type exists. It represents the absence of type information for complex types that are inherited from AnyRef.
  • Nothing – It’s a sub-type of all the types exists in Scala Types hierarchy. It helps in providing the return type for the operations that can affect a normal program’s flow. It can only be used as a type, as instantiation of nothing cannot be done. It incorporates all types under AnyRef and AnyVal. Nothing is usually used as a return type for methods that have abnormal termination and result in an exception.
  • Nil – It’s a handy way of initializing an empty list since, Nil, is an object, which extends List [Nothing]. 
  • None – In programming, there are many circumstances, where we unexpectedly received null for the methods we call. In java these are handled using try/catch or left unattended causing errors in the program. Scala provides a very graceful way of handling those situations. In cases, where you don’t know, if you would be able to return a value as expected, we can use Option [T]. It is an abstract class, with just two sub-classes, Some [T] and none. With this, we can tell users that, the method might return a T of type Some [T] or it might return none.

13) What is a Scala Trait?

A trait is a special kind of Class that enables the use of multiple inheritance. Although a trait can extend only one class, but a class can have multiple traits. However, unlike classes, traits cannot be instantiated.

14) When do you use Scala Traits?

Traits are mostly used, when we require dependency injection. Unlike Java, through Spring framework, dependency injection is achieved through annotations. In Scala, there are no annotations or no special package to be imported. We just need to initialize the class with the trait and done, dependency is injected.

15) What are the considerations you need to have when using Scala streams?

Streams in Scala are a type of lazy collection, which are created using starting element and then recursively generated using those elements. Streams are like a List, except that, elements are added only when they are accessed, hence “lazy”. Since streams are lazy in terms of adding elements, they can be unbounded also, and once the elements are added, they are cached. Since Streams can be unbounded, and all the values are computed at the time of access, programmers need to be careful on using methods which are not transformers, as it may result in java.lang.OutOfMemoryErrors.

stream.max

stream.size

stream.sum

16) What do you understand by diamond problem and how does Scala resolve this?

Multiple inheritance problem is referred to as the Deadly diamond problem or diamond problem. The inability to decide on which implementation of the method to choose is referred to as the Diamond Problem in Scala. Suppose say classes B and C both inherit from class A, while class D inherits from both class B and C. Now while implementing multiple inheritance if B and C override some method from class A, there is a confusion and dilemma always on which implementation D should inherit. This is what is referred to as diamond problem. Scala resolves diamond problem through the concept of Traits and class linearization rules.

17) What is tail-recursion in Scala?

There are several situations where programmers have to write functions that are recursive in nature. The main problem with recursive functions is that, it may eat up all the allocated stack space. To overcome this situation, Scala compiler provides a mechanism “tail recursion” to optimize these recursive functions so that it does not create new stack space, instead uses the current function stack space. To qualify for this, annotation “@annotation.tailrec” has to be used before defining the function and recursive call has to be the last statement, then only the function will compile otherwise, it will give an error.

18) What do you understand by Implicit Parameter?

Wherever, we require that function could be invoked without passing all the parameters, we use implicit parameter. We provide the default values for all the parameters or parameters which we want to be used as implicit. When the function is invoked without passing the implicit parameters, local value of that parameter is used. We need to use implicit keyword to make a value, function parameter or variable as implicit.

19) How does yield work in Scala?

The yield keyword if specified before the expression, the value returned from every expression, will be returned as the collection. The yield keyword is very useful, when there is a need, you want to use the return value of expression.  The collection returned can be used the normal collection and iterate over in another loop.

How does Yield work in Scala

20) What do you understand by a case class in Scala?

Case classes are standard classes declared with a special modifier case. Case classes export their constructor parameters and provide a recursive decomposition mechanism through pattern matching. The constructor parameters of case classes are treated as public values and can be accessed directly. For a case class, companion objects and its associated method also get generated automatically. All the methods in the class, as well, methods in the companion objects are generated based on the parameter list. The only advantage of Case class is that it automatically generates the methods from the parameter list.

Features of Case Class in Scala

  • Case objects and Case class are serializable by default.
  • Case classes can be used for pattern matching.

21) What is the use of Auxiliary Constructors in Scala?

Auxiliary Constructor is the secondary constructor in Scala declared using the keywords “this” and “def”. The main purpose of using auxiliary constructors is to overload constructors. Just like in Java, we can provide implementation for different kinds of constructors so that the right one is invoked based on the requirements. Every auxiliary constructor in Scala should differ in the number of parameters or in data types.

22) Differentiate between Array and List in Scala.

  • List is an immutable recursive data structure whilst array is a sequential mutable data structure.
  • Lists are covariant whilst array are invariants.
  • The size of a list automatically increases or decreases based on the operations that are performed on it i.e. a list in Scala is a variable-sized data structure whilst an array is fixed size data structure.

23) What do you understand by apply and unapply methods in Scala?

apply and unapply methods in Scala are used for mapping and unmapping data between form and model data.

Apply method – Used to assemble an object from its components. For example, if we want to create an Employee object  then use the two components  firstName and lastName and compose the Employee object using the apply method.

Unapply method – Used to decompose an object from its components. It follows the reverse process of apply method. So if you have an employee object, it can be decomposed into two components- firstName and lastName.

24) Can a companion object in Scala access the private members of its companion class in Scala?

According to the private access specifier, private members can be accessed only within that class but Scala’s companion object and class provide special access to private members. A companion object can access all the private members of a companion class. Similarly, a companion class can access all the private members of companion objects.

25) What is the advantage of having immutability in design for Scala programming language?

Scala uses immutability by default in most of the cases as it helps resolve issues when dealing with concurrent programs and any other equality issues.

26) What is the use of Scala's App?

App is a trait defined in scala package as "scala.App" which defines the main method. If an object or class extends this trait then they will become Scala executable programs automatically as they inherit the main method from application. Developers need not write main method when using App but the only drawback of using App is that developers have to use same name args to refer command line arguments because scala.App's main() method uses this name.

Open Ended Scala Programming Interview Questions

1) How would you rate other JVM languages when compared to Scala?

2) What Scala features interest you about programming in Scala over Python, R and Java?

3) What is the most recent technical challenge that you have experienced when working with Scala and how did you solve it?

4) Is Scala programming language community mature enough according to you?

5) Which is you most preferred development environment IDE, OS, Editor, IDE or Tools?

6) Which testing framework have you used for Scala?

7) Have you ever worked with property based testing framework Scalacheck?

PREVIOUS

NEXT

Spark Training Online

comments powered by Disqus