D doesn’t have real closures
Note: This article was written before D got full closure support.
Delegates are something that gives me mixed feelings. While I used to embrace them in Delphi and Object Pascal, I now find them difficult to get used to in C#. I guess that is because they introduce a little of the functional programming paradigm into an otherwise object oriented environment.
Don’t get me wrong, I do find delegates powerful and I use them whenever I see fit. It’s just that they bug me, like a badly-pitched note in an otherwise perfect chord. In that respect I am much more fond of the inner classes of Java.
The D Programming Language, my favorite pet at the moment, has support for both delegates and inner classes, thus providing the best of both worlds. And since D is a multi paradigm language with support for functional programming, its delegates are justified.
The semantics of D delegates are somewhat different from that of C#. For instance, C# delegates may bind to both static and non-static methods, while D may bind to non-static methods only. But on the other hand, D delegates may also bind to inner functions, which makes them look a lot like closures.
bool retry(int times, bool delegate() dg)
{
int tries = 0;
while (tries++ < times) {
if(dg())
return true;
}
return false;
}
// Example usage
retry(3, {
writefln("Password: ");
char[] buf = readln();
return (buf == "secretn");
});
In order to be a real closure a delegate is required to keep variables that are bound from the surrounding environment alive, even if the program has left their scope. For example, consider the following program.
int delegate() countdown(int from)
{
int current = from;
int countdown_helper() {
return current\-\-;
}
return &countdown_helper;
}
int delegate() dg = countdown(10);
dg(); // should return 10;
dg(); // should return 9;
dg(); // should return 8;
The countdown function returns a delegate that refers to the inner function countdown_helper. Counter_helper binds the variable current from it’s parent by returning its value and decreasing it by one.
So, if D’s delegates were real closures the first invocation of the delegate in the above example should return 10, the second 9, followed by 8, 7, and so on. But in reality they produce bogus values like 4202601 or 4202588. The reason is this: D doesn’t implement real closures. The variable current is invalid because the program is no longer executing the countdown function when countdown_helper is invoked via the delegate.
So D doesn’t have real closures, is that a big deal? No! In my opinion real closures are not important for languages that are not pure functional. The creators of D shouldn’t waste their resources on this feature since it brings little value but lots of implementation complexity. The energy is better spent on features like multicast delegates. Those would bring value in the form of convenience, and I hope future versions of D will incorporate them.
Complete versions of the code samples in this post can be found here and here.
Holy. Crap.
I don’t know what to be more astonished by: the fact that a *new* programming language doesn’t have a basic, incredibly useful feature like closures (and it’s *incredibly* useful for GUI work — it lets me easily construct anonymous functions to add as hooks for things like button presses etc.) or the fact that in this day and age there’s someone who actually thinks that closures aren’t a big deal except for functional languages.
The languages where I use closures the *most* are:
– Python
– Java (using the array trick)
– JavaScript
None of these languages is functional. In all of them, closures are incredibly useful. I can forgive ObjC for not having closures — it’s a language from the ’80s — but D? No way in hell.
Foo: Oh but I do find closures *incredibly* useful. I use them all the time in Ruby and JavaScript (and I’d love to see the array trick you mention for Java). But I have yet to see a useful example which rely on variables that have lost their scope. (This is what I meant by “real closure” in the article). The examples I’ve seen, and the ones I have come up with myself are artificial and can often be solved with a nice object oriented approach. If you have a good example please show me. No one would be happier than me if you managed to prove me wrong.
Well Foo as the author has shown, D has anonymous function so you can use them for GUIs, are closure really necessary for GUIs?
Bookmarked, ’cause I want to know how you use array to get closures in Java (come back, Foo, come back!). Not that I actually write in Java hardly ever, but still.
Hans-Eric, I don’t think it’s even really a closure (or it wouldn’t matter if it were not) if it doesn’t use variables that have gone out of scope. And closure is incredibly useful for GUI code. E.g., (in browser JavaScript) half the time you use setTimeout. How many times have you seen someone assemble a string to pass as the first argument of setTimeout, potentially with escaping required and always making the later function query the DOM again to get back to the right element(s)? Passing a string invokes eval, which is asking for trouble, and the whole thing is a mess maintenance- and clarity-wise. Trivially solved with a real closure (as JavaScript can do, but many people don’t realize).
Um
Point the first: You can emulate closures in D with just a little bit of platform dependent code by writing a function that copies the stack. This is the easiest way – like “return stacklift(&countdown_helper); ”
Point the second: You can also emulate closures in D by binding a function with the variables it will use. Example:
int countdown_helper(int c) { return c–; }
return bind(&countdown_helper, current);
Point the third: As you said yourself, if you’re in a hurry, why not just use inner classes/structs?
struct countdown_helper { int c; int call() { return c–; } }
return &(new countdown_helper).call;
Point being, there’s lots of ways to get close to real closures in D 🙂
This blog post was discussed in the D.announce newsgroup
http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D.announce&article_id=9874
Here the code from Jarrett Billingsley for the multi-cast delegates:
>>
struct MCD(Args…)
{
void function(Args)[] funcs;
void opCatAssign(void function(Args) f)
{
funcs ~= f;
}
void opCall(Args args)
{
foreach(f; funcs)
f(args);
}
}
void fa(int x)
{
Stdout.formatln(“fa({})”, x);
}
void fb(int x)
{
Stdout.formatln(“fb({})”, x);
}
void main()
{
MCD!(int) a;
a ~= &fa;
a ~= &fb;
a(3);
}
Coincidentally, I posted details of the stack-copying technique FeepingCreature talked about a few weeks ago over at: http://while-nan.blogspot.com/2007/08/fakin-closures.html
To be honest, it’s fairly rare that I come across a case where I miss being able to just directly return an inner function; and in cases where I want to do so, D has a few ways of working around that (both the hack in my post above, and also ones involving constructs like bind).
What does it mean that “For instance, C# delegates may bind to both static and non-static methods, while D may bind to non-static methods only. ” ? That doesn’t seem to make sense.
Bruno: In C# a delegate can be assigned to a static method and a non-static method interchangeably. You can even mix static and non-static methods within the same multi cast delegate.
D only allows non-static (instance) methods.
FeepingCreature: Three interesting ways to fake closures, thanks for the tip.
Daniel: Your stack-copying writeup is really cool, I had no idea it could be that simple.
Joshua: You have several good points in your comments (and no bad ones 🙂 ).
the array trick is to use a single element final array to store a non final value. this way you can still modify it from an inner class. i’ve used it for test classes, but i hope to never have to use it for anything else… hopeful gafter will have his way and get closures into java 7.
Alex
I’m back, Joshua!
Joshua wrote:
> Bookmarked, ’cause I want to know how you use array to get closures in Java (come back, Foo, come back!). Not that I actually write in Java hardly ever, but still.
Answer: Java’s anonymous functions are *almost* closures. Specifically, when you build an anonymous function within a block, it’s closed to all final local variables declared in that block or in an outer block.
The problem is that those variables *must* be final. Why? Who knows? There’s no performance hit unless you actually use a non-final variable, and even that’s not a very big hit. But Sun sought to screw us over I guess.
Well, almost. The array trick is as follows. Let’s say I wanted to make an accumulator generator (Paul Graham style, albeit with methods). I can do this:
interface NextInt { public int next(); }
public void makeNextInt(int init)
{
final int[] count = { init };
return new NextInt() { public int next() { return ++count[0]; }
}
Note that the NextInt class uses an outer lexical variable (count), and thus that variable must be final. But it doesn’t mean you can’t change it — it just has to be composite. Instead of having it be an int, make it be an int[1]. That’s the array trick.
Of course, you can make a MutableInteger or whatever if you want, and that’s a little faster as it doesn’t require an array bounds check, but it’s more work to write. It’s not part of the syntax of the language like the array is.
It turns out that there was a strong counterproposal to the crap being put out right now for excessively complex closure semantics. The counterproposal just does a simple semantic wrapper around exactly this idea. Let’s hope the Java folks get a clue and adopt it.
RenoX asked:
> Well Foo as the author has shown, D has anonymous function so you can use them for GUIs, are closure really necessary for GUIs?
Closures are hardly a necessary feature. But they are an *enormously* valuable feature for GUIs, threading, event management, and exception handling. Because they make doing callbacks elegant. Having anonymous functions without having closures based on lack of “need” is like saying you can do everything in recursion, so why bother with including for-loops…
D 2.007 adds “full closure support” (see link for changelog).