Scala Complexity vs Book of Magic Spells
Today was the first day of the wonderful Devoxx conference. I got a lot of positive impressions, but what I want to share with you today is something different. Today I was reminded about recent buzz about Scala complexity. I hear a lot about this recently, but I still fail to understand some aspects of it.
In order to describe my point of view I would like to talk about Java complexity. Let me ask you a question: How complex Spring or Hibernate frameworks are? Some of you may say, that they are pretty simple (especially in comparison to EJB 2). But why is it so? Do you really believe that they are that simple… or may be they are just familiar? I think we can view these frameworks from 2 perspectives: from user perspective and from library developer perspective. I don’t think that you expect all developers in your team to actually be able understand, how Spring/Hibernate is internally implemented. You want them to be able to use these frameworks and understand their purpose, ideology. What does it actually take to understand their internals? Which knowledge about Java can help them to understand these frameworks? Actually users should edit XML files or add annotations in their code and with the help of the Spring/Hibernate magic interesting things will happen. Spring will magically instantiate object and wire them. Hibernate will be able to load/save objects from/to the database. Do you really expect new Java developer, which you want to hire, to understand how exactly this magic works?
I want to talk more about this “magic”. Java reflection lies behind most of the modern popular Java frameworks. What does it actually means? We are just compromising the safety, performance, common java semantics of our code for convenience. Today I watched interesting talk from Spring guys called “Spring Data JPA - Repositories done right”. To make long story short, one part of these new API will allow you to write interface like this:
interface UserRepositry {
User findByFirstNameAndLastName(Sting firstName, String lastName)
}
Library will “parse” method signature and generate JQL query and all plumbing code for you… When I see code like this, Spring or other similar library looks more like book of magic spells to me rather than library or API. And I already can predict the first question of junior Java developer (after hours of code search): Where can I find UserRepositryImpl? (I still have a hard time explaining young java developers that these XML tags out there are actually creating new instances of Java classes and injecting them into other objects)
And what can i answer? No, really, is there any standard way to find answer for this puzzle, except looking into the “Spring Data Reference Manual”? I’m pretty sure that this library will have good documentation. Spring always has excellent reference manuals. But what about Javadoc? As was noted today, the first place new user will search for the help is in the examples and Javadoc. But we can’t have it anymore. Actually we don’t have any semantics of the library in the code, we just replaced public API with reflection and externalized semantics in the reference manual. Sometimes I have situation in which I can’t find solution to my problem in the documentation. In such moments I just need to look in the source code of the library… but where should I start my journey? Definitely not from UserRepositry interface…
Seems that each library adds it’s own kind of magic in the project. Then problem Is that the only common concept they have is usage of reflection. For example, can I apply my knowledge about Spring in order to better understand how Hibernate works? Seems that Scala’s map method from the collections API is pretty popular example that frightening people. But the signature of the map method is simply uses concept called Type classes . This concept adds great flexibility of the code, but also requires you to understand it, if you want to understand why map method will return you new Set[String] when you are executing it on BitSet and converting each bit to String. Actually this concept of Type classes is very common and a lot of Scala libraries are using it. Even Scala itself is using it a lot. Naturally you can invest your time in order to understand how Scala collections work (if you want, but you don’t have to, you can just use map method according to the use case in the scaladoc for this method and treat it’s method signature as a magic). But when you actually invested this time and understood it, you have learned a bunch of concepts along the way that will make your understanding of other completely different library much much easier.
In Java world you can invest time in understanding Spring school of magic, but it won’t be of any help with Hibernate’s one. From the other hand in Scala you have set of concepts that are shared between all the libraries. I also believe that Scala brings us more safety and can free us from the reflection craziness that has got a lot of popularity in Java wold recently. Also notice, that magic has not proven itself to be very reliable. It can always fail you at Friday evening a the production environment. I don’t think that in most cases Scala actually adds more complexity, it just offers you to explore new world were magic is replaced with technology which solves the same problems but in more consistent and regular way.
So my point is: You don’t need to worry about Scala’s complexity. Actually I believe that this topic is relevant only to library developers who want to make it very robust and very general. With Scala they actually have much better opportunity to make your code more safe and faster. This philosophy goes even in language design: Have you ever tried to use Java generics with all these wildcards and definitions of co/in/contravariant types? Several weeks ago I tried to use it and ended up with something like:
Map<String, ? extends Type<Integer, ? extends Map<String, ? extends List<String>>>> result = // ...
Java forces you to actually know how exactly variance works. I believe that it’s too complicated to get it right. From the other hand in Scala, variance information is described in the type definition itself, so, as user of the library, you don’t have to understand it. According to Scaladoc type parameter of immutable list is covariant, for the mutable list it’s invariant and in function it’s arguments should be contravariant and result type covariant. Can you explain why? Last time I thought about it, it took me about 1 hour of mental work before I got more or less clear understanding. And after about 5 minutes I have lost the focus and landed at the same point where I started. The problem is that Java forces you to understand these concepts if you are going to use them. In Scala, library designer can save your time and do this for you.
Hope you enjoyed this post. As always, any feedback is highly appreciated!
PS: By the way, I have nothing against the magic. If it would really exist in our world I’d probably become Archimage rather than developer :)
