Closures for Java: The Q&A
2009/12/02 08:54:30 -08:00

At Devoxx last month I said that it’s time to add closures to Java.

Closures at Devoxx

That statement, and my follow-up blog entry, elicited reactions ranging from enthusiastic support to skeptical acceptance to cynical indifference. It certainly motivated lots of tweets and lots of questions. In classic Stephen O’Grady style I’ll attempt here to summarize and answer some of the key questions I’ve seen, so far.

Q Before we begin, do you have anything to disclose?

A Yes. At heart I’m an old Lisp hacker; more specifically, I’m a past teacher and implementor of the Scheme dialect. Closures are completely natural to me—along with tail recursion, continuations, and the definition of procedure call in terms of rename plus goto. (I accept that I’m unusual in this regard.)

Having said that, in all my years at Sun I’ve done my level best to respect and preserve the “feel of Java,” as James calls it. Java incorporates some of the great ideas of the Lisp tradition but it is not, and never will be, a Lisp. So I’ve always been a bit nervous at the prospect of adding closures to Java. If we do so then we absolutely must preserve the existing character of Java as a language for “working programmers.”

Q So, why now?

A Two reasons.

First, the JDK 7 schedule has been extended, which in turn means two things: We have more time, and JDK 8 will be even later. If we don’t do closures in 7 then they’ll have to wait for 8, whenever that turns out to be. I don’t think we can wait that long to gain good support for writing scalable parallel programs using bulk-data APIs such as parallel arrays.

Second, some of the key participants in the extremely lively closures debate of a couple of years ago have indicated their willingness to consider a simplified proposal.

Q By the way, why was the JDK 7 schedule extended?

A The schedule was pushed out, to at least September 2010, so that we can finish several critical features as well as have time to submit an official Java SE 7 Platform JSR, form an Expert Group, and allow all the appropriate deliberations to take place. JDK 7 will not ship until the Java SE 7 JSR goes final.

Q Why did you make the decision to include closures in Java SE 7 behind closed doors, without any community input?

A That’s not the decision that I made, contrary to various reports. Making such a unilateral decision is not within my power, nor is it within Sun’s.

Q Okay, so what did you decide?

A I decided to initiate an effort to draft a new proposal within the design space I’ve outlined and along the lines of the straw-man examples I’ve shown, and to prototype an implementation as part of JDK 7 so as to enable broad experimentation. If all goes well then we’ll later submit a language-change JSR which will, in turn, be proposed as a component of the eventual Java SE 7 JSR.

Q Are your design constraints and straw-man examples the effort of months of internal deliberations within Sun?

A No, the constraints and examples are mine alone. They’re a reflection of my sense—which could be wrong!—of what the overall community is most likely to support.

Q Why did you choose FCM as a starting point?

A I didn’t choose FCM; I just happened to choose a notation for the examples in my talk which looks a lot like FCM.

Q Why not start with one of the existing proposals?

A The existing proposals are, in my view, either too simple or too complex. CICE only goes part-way toward making the parallel-array API easy to use, while BGGA and FCM contain more features than are needed to address that key use case.

Q Do you, or does someone else at Sun, have a specific proposal ready for review?

A No.

Q Why did Sun delay the formation of a JSR for closures for so many years?

A There was never a clear consensus in favor of a specific proposal, and in fact key respected members of the Java community supported significantly different proposals. In our view a JSR Expert Group would not have been an effective forum for resolving that disagreement.

Q Why not start a closures JSR now and let its Expert Group do the work of formulating a new proposal?

A For the same reason that there’s not yet a JSR for Project Coin, namely that no new Java SE JSR is likely to be approved until the present dispute in the JCP Executive Committee is resolved. Once that happens, and assuming that consensus forms around the result of this new effort, we fully intend to submit and lead a language-change JSR for closures. In the meantime we’ll proceed with the initial design and prototyping work in the OpenJDK Community, just as we have done with Project Coin.

Q Where will this work be done? How can I participate?

A Anyone with an informed opinion will be welcome to participate in Project Lambda, which I’ve just proposed.

Q Where can I find a written version of the straw-man examples you showed at Devoxx?

A I’ll post an updated and expanded version to the Project Lambda pages, once the Project is approved.

Q Why didn’t you take part in the original closures debate?

A I considered that, but there was an awful lot of heat in those discussions. I thought it better to let the debate run its course, see what I could learn from the proposals and their prototypes, and then try to synthesize something new and simpler.

At that point in time, moreover, we were trying to ship JDK 7 sooner rather than later. Even if broad consensus had formed around a single proposal it likely would not have made it into JDK 7.

Q Why did you use the ‘#’ character in your straw-man syntax?

A The ‘#’ character is traditionally used in Lisp to introduce certain kinds of literals, and in some dialects even closure literals. That it’s also used in the FCM proposal is coincidental, though in retrospect perhaps I should’ve chosen a different character in order to avoid giving the impression that I was backing FCM.

Q Did you really invent your straw-man syntax while flying from San Francisco to Europe?

A Yes. I’d been thinking about a simplified closures proposal for several months, but without a specific syntax. It’s hard to do a presentation about language features without a concrete syntax, however, so while writing my slides for Devoxx, somewhere high above the North Atlantic, I played around with several different possibilities. The one I showed in my talk was the best of them. I mentioned this fact during the talk only to make it clear that this syntax is just a straw-man; apparently some people found that tweet-worthy.

Q Do you expect the final proposal to use your syntax?

A No. My straw-man syntax is meant only to be a starting point, not an actual design—that takes a lot more work! I’m sure there will be a long, and likely bitter, debate over the actual concrete syntax.

Q Isn’t adding closures pretty risky? Won’t it add a lot of risk to JDK 7?

A Every big change carries risk. The key features I’ve outlined, along with many others, have been implemented in the BGGA prototype thanks to Neal Gafter’s fine work. We therefore already know that it’s possible to implement them together in a reasonably harmonious fashion; this greatly reduces the overall risk to both the design and the implementation.

Q Do you have anything else to add?

A No, I think that about covers it. I’m sure there will be many questions, comments, and suggestions going forward. The best place to engage will be the Project Lambda mailing list, once it’s been created.