Skip to content
Print

The Plague of Polyglotism

28
Apr
2008

For those of you who don’t know, polyglotism is not some weird religion but actually a growing trend in the programming industry.  In essence, it is the concept that one should not be confined to a single language for a given system or even a specific application.  With polyglot programming, a single project could use dozens of different languages, each for a different task to which they are uniquely well-suited.

As a basic example, we could write a Wicket component which makes use of Ruby’s RedCloth library for working with Textile.  Because of Scala’s flexible syntax, we can use it to perform the interop between Wicket and Ruby using an internal DSL:

class TextileLabel(id:String, model:IModel) extends WebComponent(id, model) with JRuby {
  require("textile_utils")
 
  override def onComponentTagBody(stream:MarkupStream, openTag:ComponentTag) {
    replaceComponentTagBody(markupStream, openTag, 
        'textilize(model.getObject().toString()))
  }
}
# textile_utils.rb
require 'redcloth'
 
def textilize(text)
  doc = RedCloth.new text
  doc.to_html
end

Warning: Untested code

We’re actually using three languages here, even though we only have source for two of them.  The Wicket library itself is written in Java, our component is written in Scala and we work with the RedCloth library in Ruby.    This is hardly the best example of polyglotism, but it suffices for a simple illustration.  The general idea is that you would apply this concept to a more serious project and perform more significant tasks in each of the various languages.

The Bad News

This is all well and good, but there’s a glaring problem with this philosophy of design: not everyone knows every language.  You may be a language aficionado, picking up everything from Scheme to Objective-C, but it’s only a very small percentage of developers who share that passion.  Many projects are composed of developers without extensive knowledge of diverse languages.  In fact, even with a really good sampling of talent, it’s doubtful you’ll have more than one or two people fluent in more than two languages.  And unfortunately, there’s this pesky concern we all love called “maintainability”.

Let’s pretend that Slava Pestov comes into your project as a consultant and decides that he’s going to apply the polyglot programming philosophy.  He writes a good portion of your application in Java, Lisp and some language called Factor, pockets his consultant’s fee and then moves on.  Now the code he wrote may have been phenomenally well-designed and really perfect for the task, but you’re going to have a very hard time finding a developer who can maintain it.  Let’s say that six months down the road, you decide that your widget really needs a red push button, rather than a green radio selector.  Either you need a developer who knows Factor (hint: there aren’t very many), or you need a developer who’s willing to learn it.  The thing is that most developers with the knowledge and motivation to learn a language have either already done so, or are familiar enough with the base concepts as to be capable of jumping right in.  These developers fall into that limited group of people fluent in many different languages, and as such are a rare find.

Now I’m not picking on Factor in any way, it’s a very interesting language, but it still isn’t very widespread in terms of developer expertise.  That’s really what this all comes down to: developer expertise.  Every time you make a language choice, you limit the pool of developers who are even capable of groking your code.  If I decide to build an application in Java, even assuming that’s the only language I use, I have still eliminated maybe 20% of all developers from ever touching the project.  If I make the decision to use Ruby for some parts of the application, while still using Java for the others, I’ve now factored that 80% down to maybe 35% (developers who know Java and Ruby).  Once I throw in Scala, that cuts it down still further (maybe at 15% now).  If I add a fourth language – for example, Haskell – I’ve now narrowed the field so far, that it’s doubtful I’ll find anyone capable of handling all aspects within a reasonable price range.  It’s the same problem as with framework choice, except that frameworks are much easier to learn than languages.

The polyglot ideal was really devised by a bunch of nerdy folks like me.  I love languages and would like nothing better than to get paid to learn half a dozen new ones (assuming I’m coming into a project with a strange combination I haven’t seen before).  However, as I understand the industry, that’s not a common sentiment.  So a very loud minority of developers (/me waves) has managed to forge a very hot methodology, one which excludes almost all of the hard-working developer community.  If I didn’t know better, I would be tempted to say that it was a self-serving industry ploy to foster exclusivity in the job market.

I want to work on multi-language projects as much as anyone, but I really don’t think it’s the best thing right now.  I’m working on a project now which has an aspect for which Scala would be absolutely perfect, but since I’m the only developer on hand who is remotely familiar with the language, I’m probably going to end up recommending against its adoption.  Consider carefully the ramifications of trying new languages on your own projects, you may not be doing future developers any favors by going down that path.

Comments

  1. I’ll just focus on one thing, and take it out of context a bit. I hope you don’t mind.

    “If I add a fourth language – for example, Haskell – I’ve now narrowed the field so far, that it’s doubtful I’ll find anyone capable of handling all aspects within a reasonable price range. ”

    I have heard that every public Haskell job posting attracts hunderds of applications, and perhaps several dozen good ones. The #haskell channel in Freenode is one of the largest (if not the largest) language channels.

    I know I personally would at this point accept a Haskell coding job at a relatively low price, just to get some real world experience with that beautiful platform.

    And I don’t think I’m the only one with similar thoughts. So the claim that Haskell = no workforce seems very much bogus.

    Vesa Kaihlavirta Monday, April 28, 2008 at 3:00 am
  2. Your post is exceptionally pragmatic, and I’d have to agree that for most cases, you’re right.

    To be idealistic for a moment, though: much ballyhoo is made about Paul Graham’s statement that using Lisp for Viaweb, his web store startup that became Yahoo! Stores, gave him a strategic advantage. He has extrapolated from his statement that using a niche language for a project – albeit one that is suited for the problem domain – is an advantage, in that you have to hire smart people for the project, who will help you succeed. He’s not incorrect. For many companies, that idea is not a great one, in that they couldn’t afford to hire 10 or 20 people who, frankly, are good enough that they not only know two or three languages, one of which is obscure, but also want to know more and are capable of quickly learning more.

    For a young company seeking an advantage, though, polyglotism is not a terrible idea: it forces you to find the best developers. This may be expensive – although I’d argue back that the best developers could give a hoot about money, given a decent wage and a chance to learn – but it’s an amazing advantage.

    Clinton R. Nixon Monday, April 28, 2008 at 3:11 am
  3. Your concerns are indeed valid considering the example that you give. The example uses 2 *general purpose* programming languages. Polyglot programming makes a lot more sense when 1 general purpose programming language is used in combination with several domain specific languages.
    In this respect, this is nothing new. Almost all web applications are developed with at least 3 to 4 languages; SQL,HTML, Regular expressions in combination with PHP/C#/whatever.

    You mention that projects will consist of many languages. For the same reason most software projects don’t use dozens of libraries, most projects will also not use that many laguages, there’s no added value. However, I’m sure that you would also not question to use libraries in the proper situation. Well, DSL’s can be interesting subsitutes for those libraries.

    Corno Monday, April 28, 2008 at 3:20 am
  4. Can’t deny it’s a powerful argument against polyglot programming in the real world. But there is perhaps a counterargument that a competent developer really should be willing and able to pick up a new language pretty rapidly if he’s working on it full time. And if you’re not using competent developers, you’re likely as not stuffed anyway. Is that a bit elitist? I think it might be a bit true, though.

    And some languages are “closer” than others. Knowing Java makes it easier to pick up a JVM language because you can make use of the platform. And Groovy in particular is extremely accessible for a Java dev; Scala perhaps a bit less so.

    Matt Monday, April 28, 2008 at 5:17 am
  5. You make a fair point but there is a counter argument worth considering.

    I have worked in a large corporate IT environment where Java, C++ and C# hold sway. In that environment a shift in management usually means the introduction of, what the proponents consider, a radical new technology strategy. What this amounts to is a functor to take the code base from one of the above languages to another. This is the world where polyglot programming is an anathema.

    But if one allows best tool for the job to drive technology selection and the functionality can be encapsulated and made accessible it should be relatively easy to either maintain or replace; it will likely be re-factored or replaced in any case and it is in all our interests to expose the one trick ponies of this world to a thoroughbred horse, once in a while.

    Justin Monday, April 28, 2008 at 7:58 am
  6. I think more likely you will see polyglot programming in the web space more than other places. The obvious necessary knowledge of HTML, XML, SQL, JavaScript and whatever server technologies requires a rather diverse set of skills. Throw RIA environments and the potential need to run things like Java/.NET for long running/intensive tasks with something like Rails in the foreground and I think the essence of the polyglot movement can be realized.

    You make a good point though that if folks start making the assumption that doing a project in Lisp, Factor and OCaml is OK, then they are dreaming. Web development uses different language because the problem space has required it.

    Eric Larson Monday, April 28, 2008 at 8:49 am
  7. These days, familiarizing yourself with a particular subset of libraries or a style of program architecture can take longer than learning the basics of a new language. Learning a new language, however, quickly weeds out the weaker programmers, and makes it easier to break people out of their preconceptions (which may conflict with your companies procedures) of how code should be structured. If they can’t adapt, they won’t be much use…

    dmaclay Monday, April 28, 2008 at 9:09 am
  8. “As a basic example, we could write a Wicket component which makes use of Ruby’s RedCloth library for working with Textile.”

    You will not rest until you’ve proved me wrong about this being easier to do in an external runtime over HTTP. ;) Someone should rewrite all of these in Scala and end the debate! (Several people have made nice regex wrappers; I don’t know if anything is in the stdlib tho.)

    n8 Monday, April 28, 2008 at 10:19 am
  9. A couple of people have commented that most development environments are already polyglot. For example, all developers in our neck of the woods are expected to be able to maintain Java, SQL, Javascript, HTML, JSP and XML. I would argue that each XML schema is its own language even. Using all these languages is well established and accepted as the norm.

    I like to think of polyglot programming the same as construction. I have a fairly standard set of tools for doing work around the house. While doing some remodeling recently, I needed to attach some 2×4s to the concrete floor. Now I could have accomplished this with the tools I had, but it would have taken quite a bit of time and the result would have been questionable. Instead, I rented a stud driver that uses .22 caliber shells to drive the nails through the wood into the concrete at one time, and it was fun to use to boot.

    Now this stud driver was a standard tool, although not one I own. There was an extra cost involved, both in time and money, to use this tool, but the results were better. The same analysis should be performed before adding a new language into the mix. Will the result be better and easier to maintain, even with the cost of training developers in the new language? Sometimes, the answer is yes and sometimes no. If the new language will cut out a large chunk of code, then it there might be a good argument. How much is a large chunk? 100s of lines of code eliminated. If you think about what using SQL versus writing raw database IO saves you, you will see my point.

    On the other hand, there is the “shiny” problem. Programmers will want to use the latest and greatest, not because of the great saving, but just because it is new and interesting.

    In summary, those responsible for a project are already open to the idea of multiple languages, but they should also be wise in evaluating the use of a new tool. If a language is not already well-established and of great benefit, it should not be used. If there is a great benefit, then it should be embraced.

    Likewise, when using additional languages, the costs need to be weighed.

    Matt Monday, April 28, 2008 at 3:20 pm
  10. Sadly I find very few programmers who can use even one language effectively. Programming is not in the first instance about language at all so from that viewpoint language shouldn’t matter. In the next step you need to use the constructs of a particular language to encode your algorithm, e.g. if you write C-code in Java you have in some sense failed. So to use several languages effectively you don’t just need to know them, you need to know them well, including the standard dirty workarounds of limitations. What is easier, to apply another language you barely know because parts of your program are more easily expressed with the constructs available or expand your ability to handle the situation in a language you know fairly well, deepening your skill in that language? I must confess I don’t know.

    Tobe Monday, May 5, 2008 at 8:22 am

Post a Comment

Comments are automatically formatted. Markup are either stripped or will cause large blocks of text to be eaten, depending on the phase of the moon. Code snippets should be wrapped in <pre>...</pre> tags. Indentation within pre tags will be preserved, and most instances of "<" and ">" will work without a problem.

Please note that first-time commenters are moderated, so don't panic if your comment doesn't appear immediately.

*
*