www.hans-eric.com

Sorry for the off-topic.

I saw Al Gore on Swedish television last week. He had received the Gothenburg Sustainable Development Award and was holding a speech at the ceremony. As usual Al Gore was able to capture his audience with a mix of scaring facts and a call for action. This time he conveyed the new prediction that the arctic ice could be gone in as little as five years. While that could very well be a grave exaggeration, we can’t deny that something is terribly wrong with our climate.

Where I live, a coast town in the middle of Sweden, the rising temperature has been particularly noticeable. The first sign came in the late 1980’s when I experienced my first non-white Christmas. Now, having no snow in December has become normal.

This year, the year of 2008 – a year I’ll never forget – we have witnessed something I thought was impossible. This year we had rain in January. Where I live January is the winter month. It’s supposed to be cold and the only thing coming down from the skies should be snow. In my life, and I’ve been on this planet for 37 years, I have never experienced rain in January before. But this year it rained for ten whole days.

So what can we do? Well, Al Gore’s mantra is that “we can do something about it,” and I agree. There are definitely things I can do to reduce my share of released greenhouse gases. I can use my car much less (we have great public transportations here,) and I can, with little effort, save energy. Simple things would be a good place to start; like really turning things off when they’re not used (computers, the TV, lamps), turning heat down (human beings sleep best in 18 degree Celsius) or replacing old energy devouring machines (like our freezer from 1980).

My promise to you, my dear reader, and to the rest of the world, is that 2008 will be the year when I start to implement changes to become a more optimized human being. I will make sacrifices, my life will be less convenient, but I’ll do it happily for the sake of my children and for future generations.

My intentions are not entirely unselfish though, I expect the changes to have a significant and positive effect on my economy as well.

Cheers!

P.S. I can’t help wondering what the world would be like if the outcome of the election had been different.

In a Reddit discussion following my last post on object lifetime management in D, bonzinip wondered why the D closures aren’t used with the open-closure-close idiom that is common, for instance, in Ruby.

bonzinip: languages with closures (Ruby, Smalltalk) use them to ensure that the file is closed when you get out of scope, and that does not require new keywords and does not require to bypass GC and possibly get dangling pointers.

NovaProspekt: D actually has full closures

bonzinip: so why don’t they use them with a “try { open; run closure; } finally { close; }” idiom instead?

Well, let’s take a look at how that could be accomplished in D.

To get an image of the open-closure-close idiom, we can look at Ruby’s File class, and particularly the open method.

IO.open(fd, mode_string=”r” ) => io
IO.open(fd, mode_string=”r” ) {|io| block } => obj

With no associated block, open is a synonym for IO::new. If the optional code block is given, it will be passed io as an argument, and the IO object will automatically be closed when the block terminates. In this instance, IO::open returns the value of the block.

What this means is that a File can be opened, either in the normal way with explicit closing (begin-ensure corresponds to try-finally in D)

f = File.open("c:\\log.txt", "w")
begin
  f.puts "Log this"
ensure
  f.close
end

or with a code block (closure)

File.open("c:\\log.txt", "w") do |f|
  f.puts "Log this"
end

In this last example the file is both opened and closed by the open method, in between the method invokes the associated code block with the file object reference as a parameter (f). In essence, that is the open-closure-close idiom.

How could we do this in D? Well, as I’ve written before, D’s delegates in combination with function literals can be used to mimic the code blocks of Ruby. So, a D version of the File::open method could look something like this:

import std.stdio;
import std.stream;

void open_run_close(char[] fn, FileMode fm = FileMode.In,
  void delegate(Stream) closure)
{
  Stream f = new File(fn, fm);
  try {
    closure(f);
  } finally {
    f.close();
  }
}

and be invoked like this:

open_run_close("c:\\log.txt", FileMode.Out, (Stream f) {
  OutputStream os = f;
  os.writefln("Log this");
});

To make it a little more like the original Ruby version, let’s make it support the non-closure variant as well.

Stream open_run_close(char[] fn, FileMode fm = FileMode.In,
void delegate(Stream) closure = null)
{
  Stream f = new File(fn, fm);
  if (!closure) {
    return f;
  } else {
    try {
      closure(f);
      return null;
    } finally {
      f.close();
    }
  }
}

// Example usage
open(”c:\\log.txt”, FileMode.Out, (Stream f) {
  OutputStream os = f;
  os.writefln(”Log this”);
});

// Example traditional usage
OutputStream f = open(”c:\\log2.txt”, FileMode.Out);
try {
  f.writefln(”Do log this”);
} finally {
  f.close;
}

The open-closure-close idiom is definitely doable in D. It’s not as pretty and natural as in Ruby, but not so far from it either. Whether this particular idiom will be embraced by the D community is yet to be seen.

Cheers!

The D Programming Language is a modern version of C. It adds productivity features to the performance power of C, features like object oriented programming and garbage collection.

It may seem strange that a language with focus on performance utilizes automatic memory management. A GC equals overhead, right? Well, actually that is a common misconception. These days implicitly memory managed code is generally faster than code where the programmer handles deallocations. One reason for this counter-intuitive fact is that a number of optimizations can be done by the GC, especially on short-lived objects.

But garbage collection isn’t a problem free feature. For one thing, it is indeterministic. Normally, an object that uses external resources acquires them in a constructor and releases them in the destructor.

import std.stream;

class LogFile {
  File f;

  this() {
    f = new File("c:\\log.txt", FileMode.Out);
  }

  ~this() {
    f.close;
  }

  // Logging methods goes here
}

But with a GC there is no way to know when or even if the destructor is run. This may be a problem if you’re dealing with scarce resources like window handles, or file locks. So how can we control object destructions in a GC capable language? One possibility is to trigger a GC sweep explicitly in your code. In D this is done with the fullCollect function.

import std.gc;

fullCollect();

Explicit trigging of garbage collection is usually a bad idea, for several reasons. For one thing, it’s like killing an ant with a bazooka, and still you risk missing the target. Therefore, the normal approach to deal with the problem of indeterminism is to use a dispose method.

class LogFile {
  File f;

  this() {
    f = new File("c:\\log.txt", FileMode.Out);
  }

  ~this() {
    dispose();
  }

  void dispose() {
    if (f !is null) {
      f.close;
      f = null;
    }
  }

  // Logging methods here
}

LogFile log = new LogFile();
try {
  // Do some logging
} finally {
  log.dispose;
}

We don’t have to use the dispose pattern though because in D we have the luxury of choosing between implicit and explicit memory management. We can either free the object ourselves with the delete operator, or leave it for the GC to dispose later.

LogFile log = new LogFile();
try {
  // Do some logging
} finally {
  delete log; // Explicit memory management
}

D has another construct which is very convenient when you need to control the lifetime of an object. With the scope attribute, an object is automatically destroyed when the program leaves the scope in which the object was created.

void function some_func() {
  scope LogFile log = new LogFile();
  :
  // log will be freed on exit
}

The ability to mix explicit and implicit memory management is a simple, yet great feature. It is one of many examples where D provides us with the best of two worlds. Convenience and control, as well as productivity and performance, is blended in a way that has no equivalence in any other language I know.

Cheers!

In my previous post on Loop Abstractions in D I showed you how we could make loop constructs abstract, in a similar way which is common in Ruby. The example I used as a model was the retryable method from Cheah Chu Yeow. His version is customizable in a way that let you define the type of exception that triggers a retry.

retryable(:tries => 5, :on => OpenURI::HTTPError) do
  # retryable code goes here
end

To mimic that in D we had to use templates, which are invoked with a special syntax.

retryable!(HTTPError)({
  // Retryable code goes here
}, 5);

To be honest, I don’t like the template syntax. I don’t know why, it just doesn’t feel right. If possible, I’d much prefer a more native looking code. Maybe something like this:

retryable({
  // Retryable code goes here
}, 5, HTTPError);

Christopher Wright points out an implementation that would be the closest one could get to a signature like that. He uses the somewhat primitive RTTI in D.

void retryable(ClassInfo info, int times,
  void delegate() totry)
{
  for (int i = 0; i < times; i++) {
    try {
      totry();
      break;
    } catch (Exception e) {
      if (e.classinfo is info) continue; else throw e;
    }
  }
}

Which could be invoked with the following code.

retryable(HTTPError.classinfo, 5, {
  // Retryable code goes here
});

The problem with this approach, which was pointed out by Jarret Billingsley, is that this implementation wouldn’t catch and retry on exceptions from derived classes (descendants to HTTPError in the above example). Fortunately, Jarret provides us with a solution.

What you have to do then is perform a dynamic cast. There’s no syntax for this, but you can
hack around with some of the runtime functions to do the same thing. Place:

extern(C) Object _d_dynamic_cast(Object o, ClassInfo c);

somewhere in your module, then in the catch clause of retryable:

catch(Exception e)
{
  if(_d_dynamic_cast(e, info))
    continue;
  else
    throw e;
}

That _should_ work. It’s doing the same thing as the cast operator would
but without the pretty syntax.

Not pretty, but it works, at least if you use one of the standard libraries: Tango or Phobos. I’m not sure it’s better than the template version though. The .classinfo property brings nearly as much noice as the template version does. Also, the template version has the advantage that it is resolved at compile-time.

I think I’ll go with templates after all. Who knows, I might even get used to them one day.

Cheers!

Paul W. Homer has a post up discussing essential development problems. While this is a huge subject (somewhat reflected by the size Paul’s article,) I’d like to emphasize one of the things he touches: the lack of understanding the users’ problem domain.

The biggest problem in most systems today is the total failure of the programmers to have any empathy for their users.

I too have a similar perception, and I have many times been guilty as charged, although I’ve come to realize that I’d better take the Customer View if I want to succeed. Still, I fail from time to time in the analysis of the users’ situation. I don’t know why this keeps happening, but I think impatience and a strong wish to “get going” is partly to blame.

One part of the problem could also be how many of us look upon system development. We usually decide for technology at an early stage, often on grounds that matters little to the end users. Things like personal preference and hype usually have a big impact on the choices we make. We then build the system pretty much bottom-up, adapting the business process to fit the technology. I’ve seen this unspoken philosophy many times in many places, and the results are usually poor.

The cure is a change of paradigm. We need to start with the users, identify and help develop their processes before we build systems that support them. We need to realize that the system is not the solution; it’s just a part of it.

Another part of the problem is a question of attitude. We need to accept that we’re actually in the support business. Our job is to make things easier for others, not to take the easy way out on the users’ expense, as Paul also points out.

“Use this because I think it works, and I can do it”, produces an arrogant array of badly behaving, but pragmatically convenient code. That of course is backwards, if the code needs to be ‘hard’ to make the user experience ’simple’ then that is the actual work that needs to be done. Making the code ’simple’, and dismissing the complaints from the users, is a cop-out to put it mildly.

Of course, there is a trade-off between what users want and what we can provide, but at least we need to start at the right end.

Cheers!

I haven’t been happy with the way code snippets look on my blog. Yesterday I decided to do something about it, but, which is typical me, the number of changes got a lot more than I set out for, including a Wordpress upgrade and a new theme.

After a couple of minor layout and style sheet changes, I was satisfied with the new look - in Firefox that is; When I tested it in Internet Explorer 6.0, I noticed that the content of the sidebar got displaced when there were code snippets in a post.

With a little investigation, I discovered that the triggering factor of this annoying bug was the following css rule, or more precisely, the padding property.

code{
  font:1.2em 'Courier New',Courier,Fixed;
  display:block;
  overflow:auto;
  text-align:left;
  margin: 10px 0 10px 0;
  background: #FBF5DF;
  border-top: solid 1px #EDE0B3;
  border-bottom: solid 1px #EDE0B3;
  padding: 5px 10px 5px 10px;
}

When I changed it and removed the right hand side padding, the problem went away.

code{
  font:1.2em 'Courier New',Courier,Fixed;
  display:block;
  overflow:auto;
  text-align:left;
  margin: 10px 0 10px 0;
  background: #FBF5DF;
  border-top: solid 1px #EDE0B3;
  border-bottom: solid 1px #EDE0B3;
  padding-left: 5px;
  padding-top: 10px;
  padding-bottom: 10px;
}

One may wonder how an internal space property like padding could have this effect on the positioning of other objects, but then again, I’m not surprised. I can see why Internet Explorer is not the favorite browser among web designers, or among you for that matter.

Anyway, I hope you like the new looks of my blog; If you don’t, please let me know.

Cheers!

One of the great things with Ruby is the natural way in which you can hide looping constructs behind descriptive names. Like the retryable example that Cheah Chu Yeow gives on his blog.

retryable(:tries => 5, :on => OpenURI::HTTPError) do
  open('http://example.com/flaky_api')
end

Notice how elegantly the loop logic is abstracted; There’s no need to look at the implementation of retryable to figure out what it does. The question is, can we do something similar with D as well? It turns out that with features like delegates and function literals we can actually get pretty close.

bool retryable(int tries, void delegate() dg)
{
  for(int i = tries; i > 0; i--)
  {
    try
    {
      dg();
      return true;
    }
    catch
    {
      // Retry
    }
  }
  return false;
}

Which can be used like this:

retryable(5, {
  open("http://example.com/flaky_api");
}) ;

Not as nice as with Ruby, but almost.

The custom exception of the Ruby version is a tricky one to implement in D. Templates to our rescue.

bool retryable(E)(int tries, void delegate() dg)
{
  for(int i = tries; i > 0; i--)
  {
    try
    {
      dg();
      return true;
    }
    catch (E)
    {
      // Retry
    }
  }
  return false;
}

With the (little bit odd) template syntax, we can then make retryable retry only when, for example, StdioExceptions are thrown.

retryable!(StdioException)(5, {
  open("http://example.com/flaky_api");
}) ;

To clean it up a bit, we can add some defaults (which requires us to switch places between the parameters).

bool retryable(E = Exception)(void delegate() dg, int tries = 5)
{
  for(int i = tries; i > 0; i--)
  {
    try
    {
      dg();
      return true;
    }
    catch (E)
    {
      // Retry
    }
  }
  return false;
}

That gives us a little more freedom when utilizing retryable.

retryable({
  // Retry up to 5 times
});

retryable({
  // Retry up to 10 times
}, 10);

retryable!(StdioException)({
  // Retry up to three times
  // on StdioException failures
}, 3);

I totally agree with Cheah Chu that Ruby is nice, but I think D is pretty cool too.

Cheers!

I love Rails Migrations. Not only do they help making database development a part of an agile development process, they also make my life easier as a developer with shallow knowledge in the field of database programming. But, even with frameworks like these, I think we’re still dealing with databases on a very low level.

Conceptually, databases are very simple objects. You can store data and you can retrieve data. The complexity should go no further than to declare and organize the data into relational units. Making queries should be a simple matter of getting the right data.

Reality is different. We need to consider performance, create and drop indexes, write stored procedures, configure the database for optimal performance on our specific data and specific usage; We need to write queries that not only gets us the right data, but also gets us the right data efficiently.
To get the most out of our databases we therefore need deep knowledge of the database engine, and how it’s being used by our system. For the last part, all we can do is make a good guess based on tests and supervision. In all the projects I’ve been in so far, tuning databases has always been a hefty task, hard to get right.

If we allow ourselves to think outside the box, does the tuning task really have to be this difficult? We go through great effort to collect and predict usage data although there is an object that has access to the most accurate data at all times: The database engine should be able to figure out the best tuning actions, and do them at run-time.

Java’s Virtual Machine HotSpot is a great example of a self-tuning technique, called Adaptive optimization. HotSpot starts out as a simple interpreter, but performs dynamic recompilation on portions of the bytecode that is heavily used. An adaptive optimizer can theoretically perform better than a pre-compiled program since it can make optimizations based on actual usage and local conditions.

Now, wouldn’t it be possible to create a self-tuning database engine as well?
As I said, I’m not a database expert, and I appreciate the complexity involved, but on a conceptual level there are no real obstacles - that I can see. I see no reason why it couldn’t be done. Can you?

Cheers!

Isn’t life funny? Two weeks ago I stated my opinion that unit-testing graphical user interfaces isn’t worth the trouble. Now I find myself doing it, writing unit-tests for GUI components.

What happened, did I come to my senses or go crazy (pick the expression that fits your point of view) during Christmas holidays? No, I still think that unit-testing user interfaces is most likely a waste of time, but for some special occasions, like this one, they will pay off.

My task is to make changes to a tool box control in a legacy application. The control is full of application logic and it has a strong and complicated relationship to an edit area control. There are two things I need to do:
First I have to pull out the application logic of the tool box and break the tight relationship to the edit area. Then I need to push that application logic deeper into the application core, so that the tool box could be used via the plug-in framework.

I figured I had two options. I could either refactor the tool box and make the changes in small steps, or I could rebuild the complete tool box logic and then change the controls to use the new programming interface.
I found the rebuild alternative too risky. The code base is full of booby traps and I have to be very careful not to introduce bugs. Therefore I decided to go with refactoring.

But refactoring requires a unit-testing harness, which of course this application doesn’t have. Trust me; you don’t want to refactor without extensive unit-testing, so here I am setting up the tests around the involved GUI controls. It’s a lot of work, especially since I don’t have a decent framework for creating mock objects, but it’ll be worth the effort once I start messing around with the code.

As a conclusion I’d like to refine the “Don’t unit-test GUI” statement to read “Don’t unit-test GUI unless you’re refactoring it, and there’s no easier way to make sure your changes doesn’t break anything.”

Cheers!

Christmas for me is celebration, gormandizing and the joy of happy children. It’s also a time for rest and reflection, and when I think back on this year there is one thing that disturbs me: for me this was a year of us and them.

The company I work for is divided; we have three divisions divided into several so called “functions”, which in turn is divided into teams, where people mostly work – well - alone… And on top of that we have several temporary project organizations fighting for the scarce resources.

Although we’re trying to change things for the better, there is still a lot of distrust and arrogance that stems from lack of understanding between the various formations. We are an organisation that needs more empathy.

My guess is that this is a common problem in our business. We tend to mentally separate us from the others, thinking in terms of us and them. There’s a lot of “those stupid users”, and “management has no clue” kind of talk out there.

We even turn on ourselves. What’s the first thing you do when you look at code that is new to you? I know my reflex is to try and spot design flaws, so that I can point them out. If I can’t understand the code fast enough, the normal response is “who created this piece of crap?”

These responses only show a lack of empathy in the person that utters them. Nowadays I try not to be so judgemental when I look at code that others have created. I’ve come to realize that while there is always a reason for crappy code, it’s never because the programmer wanted to do a bad job. He or she has most likely tried their best, with their current knowledge, or made a deliberate choice, given the circumstances. At least they deserve respect for that.

When we start talking negatively about each other, in terms of us and them, it might be because we lost track of the bigger picture. Here’s an exercise that helps me when I need to put things into perspective. Look at this image from Big Sky Astronomy Club.

The image was taken by Voyager 1 in 1990 on its way out of the solar system. The little blue dot is planet earth. Fellows, that pixel is our home!

In this humbling perspective there is no us and them; There’s only us.

Cheers!