DPP's Blog

Yes, Virginia, Scala is hard September 1, 2011

Let me first say that I am a Scala lover and have been a Scala champion for almost 5 years.  I've written books and articles on Scala.  I've worked with dozens of companies that have launched Scala and Lift projects.  I've code reviewed many dozens of Scala projects.

I used to think that Scala was easy.  It was, and continues to be, a cure for some of the numerous problems with Java.  From the "stuff that's hard or impossible in Java is simple in Scala," Scala is a very easy language.  Dealing with collections is super easy in Scala.  Isolating business logic making programs much more maintainable is vastly easier in Scala than it is in Java.

So, why is Scala hard?  Here's the best list I can come up with:

  • Scala tries to be too many things.  This means that you can code it like Java which is a curse and a blessing, but I think over the long term, it's a curse.  It means there are too many debates about OO vs. FP.  These kind of decisions/debates are okay if you're a small team, but they really suck when you're trying to teach Java developers to write Scala when they don't really want to learn.  The overwhelming advantages of Scala really manifest themselves when you're coding mostly FP, but it's nearly impossible to get OO developers there unless they really want to get there.  In this case, less language features/choices (like Java or Ruby) is easier.  It makes for less choices.
  • IDE support is weak and always will be.  The Eclipse plugin for Scala (or whatever it's called this week) sucks.  It has sucked for the 5 years I've been doing Scala and it's always "just about to get better" but it sucks.  IntelliJ's Scala support is reasonable for me.  But the folks who need patterns in their IDE will not find the Scala support good.  For one, the Scala patterns as so diverse and disjunct (OO via mixins all the way to ScalaZ), there will be too many patterns/templates.  If you're happy coding in Emacs or Vi or TextMate, Scala is okay and moving to IntelliJ is nice.  If you're expecting the hand-holding that you get with Java IDEs, it isn't there and never will be because encompassing Scala's power with simple paradigms is difficult and requires tons of resources that even TypeSafe with $3M in the bank cannot underwrite.
  • The Scala type system is tremendously powerful, but it's in your face way too much.  In the ScalaDocs, the type signatures are just plain frightening.  Look at flatMap [B, That] (f: (A) ⇒ Traversable[B])(implicit bf: CanBuildFrom[List[A], B, That]) : That and tell me that doesn't make you want to run screaming.  This is a method that beginners use every day, 20 times a day and it's way too intimidating.  Scala documentation needs a library consumer more that hides the complexity that is going on which in practice makes flatMap simply amazingly powerful.  The type system and associated documentation needs a simpler mode that hides the features that trade power and complexity at the library level for simplicity at the end user level.
  • Junior developers who are asked to maintain code written by senior developers have to understand the idioms and patterns in the code.  While Scala makes putting business logic at the forefront of the code (rather than distributed through a bunch of for loops and complex if statements), depending on the idioms used, decoding that logic is non-obvious.  This is a variant of the lack of idioms issue, but at the end of the day, you need a team with a mind-meld to grok some Scala code.  This is true in Ruby and Rails as well where hashmaps replace virtually all other programming paradigms.  But in Rails-land, the paradigm is uniform (although not type-checked) and well understood because it's "the way."  In Java, the patterns are vomited out of the IDE, so developers grow up being able to spot the patterns.  That's not true of Scala where the idioms are diverse and team/framework specific.
There exists a number of team types for which Scala is a significantly better choice than Java or Ruby or any other language that I know of. Twitter is the paradigmatic example.  They had a need for concise, type-checked, high performance, etc. language and runtime.  Scala provided this for them.  Foursquare uses Scala's difficulty as a filtering mechanism.  You've gotta be good enough to be able to learn Scala to succeed at Foursquare.

But if you've got a team that's closer to the mean skill-wise, Scala may not be a win for your company (this depends on management... does it want to use the challenge of Scala as a way of filtering and improving the team?)  If you're a center-of-the-mean (COTM) company, then Scala costs you in terms of learning curve, existing developer rejection, lack of patterns.  You'll need a strong CTO or architect to enforce the patterns rather than getting them from books and the IDE and the number of COTM companies with strong CTOs or Architects is, well, "limited".

So, how can you figure out if Scala will be "easy" or "hard" for your organization:

  • Your company has speakers at JavaOne, OSCON, Strangle Loop, QCon: Scala will be easy
  • Lunch-time discussions involve the criteria for moving from a developer to a senior developer: Scala will be hard
  • Your developers can write code in NotePad if they have to: Easy
  • Your developers stare blankly or say 3 "Hail Marys" when they hear the name "Zed Shaw": Scala == Hard
  • Developers all follow Dean Wampler on Twitter: Scala Easy
  • Your developers come in at 9:15 and leave before 6 and don't check work email at night: Hard
Well... you get the idea.  But I totally agree with the assertion that for an average team, Scala is hard.  Not only is it hard, but it will not yield the short or long term benefits for the average team that it does for teams composed of members at the 95th percentile of skill.

A couple of notes:

  • Yes, Scala's type system is powerful and leads to some beautifully functioning code like Scala's collections.  See http://stackoverflow.com/questions/1722726/is-the-scala-2-8-collections-library-a-case-of-the-longest-suicide-note-in-histo and http://www.scala-lang.org/docu/files/collections-api/collections-impl.html  But there's a difference between what the language/library designers need and what the COTM developer needs.  Because there's no separation in Scala, it makes it hard for the COTM developer.  Personally, I don't think I could express the ideas in Lift as concisely or as powerfully in any other language, so as a library author, I love Scala.  I've also come to realize that Lift is scary for and may be too much for a COTM developer.  As a library user who understands type signatures I love Scala.  But I'm not at the mean and most of the people who don't see Scala as hard are not at the mean.  And the 11 year old kid who writes Scala is not at the mean.
  • Yes, improving ScalaDocs to allow the "simple" view and the "architecture" view would be a huge win.  But that's a start, not an endpoint.
  • I am explicitly rejecting the argument "well, then, find better developers." We could solve the "Scala is hard" problem by working to improve the overall quality of developers (ones who can read a type signature, ones who can express their programs mathematically, etc.), but that misses the point.  The point is that Scala is not better enough to force a revolution in training, education, and hiring such that Scala will be able to change the quality of the average developer enough to make Scala not hard for that developer.
  • I doubt that anyone who reads this or reads my Twitter stream is an average developer and Paul Snively, you are so not average you shouldn't even try!