Scala extension methods using implicits

What are extension methods?

Implicits are one of the fundamental language features in Scala. Some of its uses are in implicit method parameters, implicit object conversions and described in this article – implicit extension methods. This methodology is also called “pimp my library pattern”.

Extension methods allow you to add new capabilities to existing types / classes without the need of extending original type. This is especially useful when we want to enrich some class we do not have control over (e.g. external library).

A good example of this is in Apache Spark when adding support for new data sources for read write, e.g. Avro IO support. All we need to do is to add avro dependency to our project, add an import in the scope and magically our Dataframe/Dataset API has a new method available, e.g:

val df = spark.load.avro(...)

You can see implementation of this here: https://github.com/databricks/spark-avro/blob/master/src/main/scala/com/databricks/spark/avro/package.scala

Continue reading “Scala extension methods using implicits” »

Difference between functions and methods in Scala

Scala has a concept of functions and method definitions. These two concepts are often confused and it’s not always clear when to use which.

Consider this example.

Simple method:

def m1(x: Int) = x + x
m1(2)  // 4

Simple function:

val f1 = (x: Int) => x + x
f1(2)  // 4

Both m1 and f1 are called the same way and will produce the same result but when you look closer you will see that these are two different things.

f1  // Int => Int = <function1>
m1  // error: missing argument list for method m1...

Calling f1 without argument will give us the signature of anonymous function.
Our anonymous functions is actually an instance of Function1[Int, Int] that means a function with 1 parameter of type Int and return value of type Int.

f1.isInstanceOf[Function1[Int, Int]]  // Boolean = true  

When calling m1 we will get an error. This is because m1 is not a value, it is a method, something that will produce a value when you call it by providing all required arguments.

Converting method into a function

Method can be converted into a proper function (often referred to as lifting) by calling method with underscore “_” after method name.

val f2 = m1 _  // Int => Int = <function1>

Alternatively you can supply a type and compiler will know what to do.

val f2: Int => Int = m1  // Int => Int = <function1>

Above method to function conversions will always create a new instance of Function1[Int, Int] which is an important observation.

Let’s consider following example. We are creating a sequence of tuples with type (Int, Int => Int). For each tuple we decided to convert our previously defined method into a function. The following will create 10 instances of a function that does the same thing.

for (i <- 0 until 10) yield i -> m1 _

Better approach would be to pass the same instance of the function for each tuple and avoid allocating memory for each instance as in previous approach.

for (i <- 0 until 10) yield i -> f1

When to use methods and when functions

– use functions if you need to pass them around as parameters
– use functions if you want to act on their instances, e.g. f1.compose(f2)(2)
– use methods if you want to use default values for parameters e.g. def m1(age: Int = 2) = ...
– use methods if you just need to compute and return

How to create git like command line interface in Scala

Recently I’ve been building a small application that exposed only command line interface to the user. While I found some solutions that suggested to use plain Scala for this task (which was pretty straighforward) I wanted something that is more suitable. For example I wanted to make sure that –help will be auto-generated so I don’t have to define it manually.

After little research I found scopt which had all the features I needed. Syntax may be a bit confusing at first especially for people new to Scala (myself included) but after a while you will appreciate it’s simplicity.

As an example of what scopt can do I decided to implement a demo app resembling GIT CLI with branch and commit child commands. This will allow for executing commands like git commit [options] or git checkout <branch> [options].

Continue reading “How to create git like command line interface in Scala” »