Java Control Abstractions

If you were puzzled by my last post, it was a poor attempt at humour.  Apparently, there is some politics around adding closures to Java.  Two current Google and former Sun heavyweights have competing proposals for adding closures to Java 7, so Google issued a statement of neutrality. 

Closures allow us to write succinct, clean code like this (taken from Ruby)

def highPaid(emps)
 threshold = 150
 return emps.select {|e| e.salary > threshold}
end

in which the free variable threshold is bound in its lexical context. 

The first of the competing closure proposal, what I call "closure light", is admittedly syntax sugar for anonymous inner classes.  For example, you could sort a list of Strings with

List<String> ls = ... ;
Collections.sort(ls,
 Comparator<String>(String s1, String s2){ return s1.length() - s2.length(); });

instead of the current

List ls = ... ;
Collections.sort(ls, new Comparator() {
 public int compare(String s1, String s2) {
  return s1.length() - s2.length(); 
 }
});

The goal is not to revamp the language to support closure but just to make things more concise.   Proponents argue that it does much of what is desired via anonymous inner classes (limited closures) but without the complexity of "full blown" closure.

So what does full blown closure buy us that makes me want it?  Two words: control abstraction.  In the following

List<Double> deposits = ... ;
synchronized (myMonitor) {
 for (Double d : deposits) {
  // update checking account
 }
}

synchronized is a control abstraction, as is the for-each loop.  They are both built-in control abstractions.  Full blown closure would allow us to implement our own control abstractions.  With full blown closure, synchronized  and for-each could have been implemented as a library API.  In fact, in Smalltalk, if statements, while loops, for loops, etc. are all implemented as regular methods in the language using block closures.   If you have been working in Java your entire career like I have, re-read these last two sentences and tell me it doesn't make you pause for a second.

With full blown closure you replace

FileReader fileReader = ... ;
try {
 // read file using fileReader
} finally {
 fileReader.close();   
}

long start = System.nanoTime();
try  {
 // do some stuff
} finally {
 log("stuff", System.nanoTime() - start );
}

with

FileReader fileReader = ...;
withReader (fileReader) {
 // read file using fileReader
}

recordTime ("stuff") {
 // do stuff 
}

The key thing to note is that within the block closure every statement and expression has the same semantics as it does in its enclosing scope.  Not only are free variables bound in its lexical context, so is "this" and "return," "continue," and "break".  You wouldn't be able to write

List<Double> deposits = ... ;
synchronized (myMonitor) {
 for (Double d : deposits) {
  this.deposit(d);
 System.out.println("deposited " + d + " in instance " + this.hashCode());
 }
}

if the semantics changed inside the synchronized block.

Obviously full blown closure is much more powerful than closure light, but at the expensive of adding complexity to the Java language.  For example, what happens with returns when the closure is executed in a different thread? And therein lies the debate, whether the added complexity is worth it. 

I've never really bought into the complexity argument.  When was the last time the UI guy at work complained JavaScript is too complex because of closure.  Generics and anonymous inner classes added complexity to the language, but if you and I (the average programmer) don't use them, they don't really get in our way.  And I imagine that's how closure will be.  Mainstream programmers will casually use closures where needed, but leave the finer points to library developers.

In fairness, closure light does have an additional proposal to add resource management to Java, which would alleviate the need for some of the control abstractions.  It then gets philosophical, whether you want to strategically change the Java language to do X and Y; or give the language the facilities so that anyone can implement X and Y (and Z).

Maybe I will get my employer, Rearden Commerce, to issue a statement on adding closures to Java before Google acquires us and we are force to remain neutral.  For now, at least you know my position.

References:

1. Neal Gafter's blog

2. javac.info (BGGA)

3. Smalltalk versus Java

4. Crazy Bob: The Java Closure Spectrum

Comments

[Casper Bang]

Nice overview of the situation!

[michael B]

The first, ruby, example is not really hitting the mark as in that it doesn't highlight an certain aspect. That is whether or not the 'final' restriction should be lifted. threshold (seems) constant. Tbh though optimization I see the the variable not even making it to bytecode/runtime depending on how it is translated.

Regardless, I'm starting to wonder do people really want closures or do people really want control abstraction. Are these really closure proposals or are they 'control abstraction'-proposals. Much of the added, if not all, complexity with regard to the proposals is only there to enable control abstraction. Do we perhaps need a shift in perspective, just because other languages enable control abstraction though closure does that necessary mean java should? _Are_ there other options? Have we given though to that? People seem to be stuck in 'the box'.

[Joshua Cranmer]

Simplifying anonymous inner classes, I can deal with. Probably (I haven't really played with a syntax long enough to make a final decision). Function pointers look like a mistake to me, but I'll probably accept them if they make it in, if the syntax isn't too ugly.

Now we come to these control abstractions. This I do not think would be good for Java. I know at one point the proposal called for making "return" different from "return;" (note that the only difference is in the presence or lack of the typically omnipresent end-of-statement indicator). The only commonly-requested abstraction I've seen is the "with"; people talk about forEach, but that already exists, so that's pointless. Until today, I've not seen a serious request for anything other than those two, in fact. And that is assuming that recordTime is actually useful beyond one or two developers.

The primary benefits, as I see them: decrease boilerplate, and, er, that's it (although I suspect that "XYZ language has it" is at least a hidden motive for many people). The only things that have been added to the Java language whose sole purpose was to decrease boilerplate that I can think of ATM are assertions and for each. And unlike the proposals indicated herein, those add no complexity: they are equivalent to an if statement and a regular for loop at the JVM level, unlike the closures.

What do you lose with control abstractions?
1. Mistakes are harder to catch.
2. What does "recordTime() { }" do? It's another case where I have to go figure out what the heck is going on.
3. Readability goes down the drain. Continuation of 2. If there's something that we should have learned from C/C++, it's that, for all the power and flexibility of the C preprocessor, if used in the wrong circumstances, boilerplate is reduced at the cost of readibility. CASES_CONVERT_INT_TO_STRING reduces boilerplate, but it's annoying when I'm trying to find out what actually happens if I pass in an int.
4. The precedent in Java is to decrease boilerplate while increasing safety. Closures would reduce safety, AFAICT, while decreasing boilerplate. I like Java because it is INCREDIBLY type-safe. I have had to rewrite code written in JS, and I would like to tell you that type safety is a very underrated feature in a language.

Some more things about closures: if you want them, you essentially have most of salient features already, in a handier format. Anonymous inner classes implicitly document what you're doing, something which is hard to do with function pointers (Comparator/compare is sufficient to tell you what goes on, but T, T => int is mystifying).

And, finally, there's so much controversy surrounding even the idea of closures, while there are major areas which uncontroversially need work. I am referring to, of course, generics reification. Instead of trying to add new features, why don't we try to fix up something already in the code?

[Howard Lovatt]

Your examples could be written with inner classes therefore they do not provide an argument for the added complexity of BGGA style closures. Here are some control examples in various inner class/closure proposals:

http://www.artima.com/weblogs/viewpost.jsp?thread=202004

Very similar to your synchronised example is the standard withLock example used in many posts about inner classes/closures. In current Java would be:

withLock(lock, new Block0() {
public Void call() {
for (Double d : deposits) {
deposit(d);
out.println("deposited " + d + " in instance " + Outer.this.hashCode());
}
}
});

Therefore inner classes can do you example currently by accessing the outer this pointer, in this case the enclosing class is called Outer, and by extension so can all the proposals like C3S and CICE that are inner class based. Note you currently only have to qualify if there is ambiguity, normally just stuff from Object, therefore no need to qualify deposit.

The key is that an inner class has two this pointers a BGGA closure has only one. Therefore anything you can do with a BGGA closure you can do with an inner class, the converse isn't true. BGGA for example can't inherit a partial implementation, have fields, or recursively call itself, it doesn't have the this pointer to access them!

I am largely reiterating other comments to your blog - why do we need the complexity of new semantics that are less powerful than the semantics we already have - give a compelling use case.

[tinou]

thanks for the comments. there's definitely both sides to this.

regarding "recordTime" -- in almost every project/company i've worked on/for there has been a need to log stuff for performance analysis, error handling, etc. i think this is a very real, concrete use case.

my examples are not the best, but if I changed it to so the block returned the balance after the deposit then the inner class solution wouldn't work.

[Caligula]

@Joshua: 2. What does "recordTime() { }" do?

What does foo.recordTime() do?

I fail to see how readability has been actively decreased. Is Lisp or Smalltalk readability decreased? No. If anything the reduced boilerplate *increases* readability because it reduces cognitive overhead: parsing boilerplate costs brain cycles.

You have to understand the APIs you're using regardless of how they're expressed.

[Patrik Beno]

I love nested/inner classes.

And I hate two things about them:

(1) they are too verbose for a usually single method classes
(2) they keep a reference to outer class (this$0 field) even when they don't need it, thus preventing garbage collector from doing its job

This is why I want closures. It is not question of why, just HOW?

[fahrrad]

Dies ist ein großer Ort. Ich möchte hier noch einmal.