www.hans-eric.com

When it comes to the question of whether methods should be virtual by default or not, there are two schools of thought. Anders Hejlsberg, lead architect of C#, describes them in an interview from 2003.

The academic school of thought says, “Everything should be virtual, because I might want to override it someday.” The pragmatic school of thought, which comes from building real applications that run in the real world, says, “We’ve got to be real careful about what we make virtual.”

As I told you about in my post from last week, I have left the “pragmatic school of thought” to join the “academic” camp. The main reason was unit-testing, that – in my opinion – calls for a more flexible object model than the one of C#. When unit-testing, I often want to use components in unusual ways, all in the name of dependency breaking, and therefore I like their methods to be virtual.

But, it wasn’t – and still isn’t – an easy pick; Virtual by default does bring some serious problems to the table. Again from the interview with Anders:

Every time you say virtual in an API, you are creating a call back hook. As an OS or API framework designer, you’ve got to be real careful about that. You don’t want users overriding and hooking at any arbitrary point in an API, because you cannot necessarily make those promises.

Whenever they ship a new version of the Java class libraries, breakage occurs. Whenever they introduce a new method in a base class, if someone in a derived class had a method of that same name, that method is now an override—except if it has a different return type, it no longer compiles. The problem is that Java, and also C++, does not capture the intent of the programmer with respect to virtual.

C# captures the intent better and avoids versioning problems, and Java offers the flexibility needed for unit-testing. Which is better? The answer seems to be: it depends. But, do we really have to choose? Why can’t we have both? Well, I think we can, and Java has shown the way to achieve it.

Java annotations is a powerful language feature. (The concept was rightfully stolen from C# where it is called custom attributes.) With it one can attach metadata to parts of a program, to give additional information to the compiler or other external agents. In other words, annotations can be used to extend the programming language without the need for changing the language itself.

A good example is the @override annotation.

class SomeClass extends SomeBaseClass {

  <strong>@override</strong>
  void someMethod() { … }

}

From the Java documentation:

[@override indicates] that a method declaration is intended to override a method declaration in a superclass. If a method is annotated with this annotation type but does not override a superclass method, compilers are required to generate an error message.

The use of the @override annotation takes care of the problem when name changes to virtual methods silently breaks behavior of descending classes; Or, which is more common, when misspelled names of intended overrides aren’t captured at compile-time. Now, with the introduction of the @override annotation, you can help the compiler help you to fail fast. It is now possible to show your intention better than what was possible in the pre annotation days.

Unfortunately Java doesn’t take this concept all the way. They could have introduced an @virtual annotation as a complement to @override, and thus reach the same level of expressiveness as C#, but without forgoing the flexibility of the Java object model. It would be the perfect middle-way, and provide the best of both worlds.

Class SomeBaseClass {

  <strong>@virtual</strong>
  void someMethod() { … }

}

The benefit of an annotation (or custom attribute) based solution is that it’s configurable. It would be possible to alter the compiler’s behavior based on context or environment. For instance one could enforce the use of @virtual and @override in production code. Additionally, one could relax the controls when necessary, like in test projects or legacy code, to mere warnings or complete silence.

Wouldn’t that be better than the all or nothing solutions of today?

Cheers!

I’ve taken quite a few paradigm shifting journeys in my life. Some have been quick insights, instant moments of clarity. Others have been slow processes, spanning over several years. One such slow journey for me has been how I look upon virtual methods of object oriented languages.

Java methods are virtual by default. Unless explicitly marked as final, a method may be overridden in a deriving class.

class A {

  a() { … }

  final b() { … }

}

Class B extends A {

  // B::a overrides A::a
  a() { … }

  // Compile error, A::b is non-virtual
  b() { … }

}

C# on the other hand, takes the opposite approach. There you have to explicitly mark a method as virtual, and an overriding method must be marked with the override keyword.

class A {

  virtual a() { … }

  b() { … }

}

class B {

  override a() { … }

  // This won’t compile since A::b is non-virtual.
  // Without the override keyword though,
  // B::b only hides A::b which produces a
  // warning instead of an error.

  override b() { … }

}

In that sense C# is more expressive, and intention is better communicated with the explicit marks. On the other hand, Java is more flexible and less restrictive.

I used to favor the non-virtual by default approach. Partly because I was kind of born with it – my first OOP language was Object Pascal and Delphi – but also because I appreciate the better control as to how my components are being used. The ability to restrict polymorphism is good because usage in a way that wasn’t intended can create a lot of mess.
To give one non-obvious example, consider a library class A with a method a.

class A {

  a { … }

}

A library user decides he needs to add functionality to A, so he creates a descender B with a method b.

class B: A {

  b { … }

}

Everything works just fine, but two year later a new version of the library is released. In this new version a new method b was added to class A. Suddenly B::b overrides A::b although this was not the original intention. In a language like C# compiling class B will result in an error (since b is not virtual), but in Java and most other object-oriented languages the upgrade may have introduced a subtle bug.

My standpoint regarding virtual methods used to be that the need for control outweighs the need for flexibility, but some five years ago I started to re-evaluate that opinion. The reason was automated testing. I have slowly come to realize that the C# object model is a pain, at least when it comes to putting a piece of legacy code under a testing harness. While there are plenty of situations where sub classing is a bad idea, unit-testing is not one of them. Sub classing can be a powerful dependency breaking technique, and it allows for testing of code that isn’t otherwise testable without refactoring.

One (contrived) example:

class SpecialConnection {

  //…

  send_string(String s) {
    //…
  };

  //…
}

class SomeLegacyClass {

  SpecialConnection _conn;

  SomeLegacyClass(SpecialConnection connection) {
    //…
  }

  //…

  create_and_send_string() {
    String s;

    // some code that builds
    // the string to send

    _connection.send_string(s);
  }

  //…
}

How can I test the create_and_send_string method? Well, if SpecialConnection::send_string is virtual the answer is easy, just subclass SpecialConnection and stub out the method.

Class FakeSpecialConnection: SpecialConnection {
  Send_string(String s) {
    if (s != “expected value”)
    throw new Exception(…);
  }
}

And the test code could be like this:

SomeLegacyClass c = new SomeLegacyClass(new FakeSpecialConnection());
c.create_and_send_string();

On the other hand, if send_string is not a virtual method we must do some refactoring (like extracting an interface from SpecialConnection) before we can do testing without a real connection. One could argue that the code in the example is poorly designed, and that we should refactor it anyway. That’s true, but a prerequisite for safe refactoring is thorough unit-testing, something we don’t have at this point, which is why we’re trying to get this piece of code into a testing harness in the first place; so that we could make safe refactorings.

When it comes to testing, virtual methods means more options, better options, safer options. That is the reason why I have converted, and now value that flexibility more than I fear possible regression.

Cheers!

One of the great things with Ruby is the natural way in which you can hide looping constructs behind descriptive names. Like the retryable example that Cheah Chu Yeow gives on his blog.

retryable(:tries => 5, :on => OpenURI::HTTPError) do
  open('http://example.com/flaky_api')
end

Notice how elegantly the loop logic is abstracted; There’s no need to look at the implementation of retryable to figure out what it does. The question is, can we do something similar with D as well? It turns out that with features like delegates and function literals we can actually get pretty close.

bool retryable(int tries, void delegate() dg)
{
  for(int i = tries; i > 0; i--)
  {
    try
    {
      dg();
      return true;
    }
    catch
    {
      // Retry
    }
  }
  return false;
}

Which can be used like this:

retryable(5, {
  open("http://example.com/flaky_api");
}) ;

Not as nice as with Ruby, but almost.

The custom exception of the Ruby version is a tricky one to implement in D. Templates to our rescue.

bool retryable(E)(int tries, void delegate() dg)
{
  for(int i = tries; i > 0; i--)
  {
    try
    {
      dg();
      return true;
    }
    catch (E)
    {
      // Retry
    }
  }
  return false;
}

With the (little bit odd) template syntax, we can then make retryable retry only when, for example, StdioExceptions are thrown.

retryable!(StdioException)(5, {
  open("http://example.com/flaky_api");
}) ;

To clean it up a bit, we can add some defaults (which requires us to switch places between the parameters).

bool retryable(E = Exception)(void delegate() dg, int tries = 5)
{
  for(int i = tries; i > 0; i--)
  {
    try
    {
      dg();
      return true;
    }
    catch (E)
    {
      // Retry
    }
  }
  return false;
}

That gives us a little more freedom when utilizing retryable.

retryable({
  // Retry up to 5 times
});

retryable({
  // Retry up to 10 times
}, 10);

retryable!(StdioException)({
  // Retry up to three times
  // on StdioException failures
}, 3);

I totally agree with Cheah Chu that Ruby is nice, but I think D is pretty cool too.

Cheers!

I love Rails Migrations. Not only do they help making database development a part of an agile development process, they also make my life easier as a developer with shallow knowledge in the field of database programming. But, even with frameworks like these, I think we’re still dealing with databases on a very low level.

Conceptually, databases are very simple objects. You can store data and you can retrieve data. The complexity should go no further than to declare and organize the data into relational units. Making queries should be a simple matter of getting the right data.

Reality is different. We need to consider performance, create and drop indexes, write stored procedures, configure the database for optimal performance on our specific data and specific usage; We need to write queries that not only gets us the right data, but also gets us the right data efficiently.
To get the most out of our databases we therefore need deep knowledge of the database engine, and how it’s being used by our system. For the last part, all we can do is make a good guess based on tests and supervision. In all the projects I’ve been in so far, tuning databases has always been a hefty task, hard to get right.

If we allow ourselves to think outside the box, does the tuning task really have to be this difficult? We go through great effort to collect and predict usage data although there is an object that has access to the most accurate data at all times: The database engine should be able to figure out the best tuning actions, and do them at run-time.

Java’s Virtual Machine HotSpot is a great example of a self-tuning technique, called Adaptive optimization. HotSpot starts out as a simple interpreter, but performs dynamic recompilation on portions of the bytecode that is heavily used. An adaptive optimizer can theoretically perform better than a pre-compiled program since it can make optimizations based on actual usage and local conditions.

Now, wouldn’t it be possible to create a self-tuning database engine as well?
As I said, I’m not a database expert, and I appreciate the complexity involved, but on a conceptual level there are no real obstacles – that I can see. I see no reason why it couldn’t be done. Can you?

Cheers!

I’m currently rereading parts of the book Test-Driven Development: A Practical Guide, by David Astels. It’s a great book in many ways, well worth reading, but I have objections to one particular section in the book.
The author tries to convince me that developing my user interfaces using a test-driven approach is a good thing to do. I disagree.

I love TDD, it’s one of the most powerful development techniques I know, but it’s not without limitations. For one thing, code with unit-tests attached is more difficult to change. This is often an acceptable price to pay since the benefit of producing and having the unit-tests is usually greater.

But the return of the investment isn’t always bigger than the price, and sometimes the cost of change exceeds the benefit of protection. That’s why most developers won’t use TDD for experimental code, and that’s why I’m not using it to test my user interfaces.

I prefer to develop GUIs in a RAD environment, visually dragging and dropping components, moving them around, exchanging them for others if better ones are to be found. In that process unit-testing just gets in my way. After all, the GUI is supposed to be a thin layer, without business logic, so there is only so much to test.

One could theoretically test that the form contains certain key components, that they are visible, have a certain position or layout, stuff like that – but I find that kind of testing too specific for my taste.

In my opinion, unit-testing should test functionality, not usability. It shouldn’t dictate whether I decide to show a set of items in a plain list or in a combo-box. What it should do, is test that the business logic of my code produce the correct set of items, and leave the graphical worries to the testers.

This brings us to something that is sometimes misunderstood: Unit-testing can never replace conventional testing. Some things are just better left to humans. Like testing user interfaces.

Cheers!

We’ve done a lot with testing frameworks over the years, but does the testing concern deserve its own standalone DSL?

This intriguing question was asked by Michael Feathers in his Mock Objects: Leaping out of the Language post. My spontaneous answer is: Absolutely!

I’m a big fan of xUnit frameworks, but when I imagine an alternative unit-testing specific language one special property comes to mind, a feature that would really make a difference. I’d call it unconditional mocking. With a DSL based unit-testing framework one could test really complex objects, even legacy code, since mocking internal objects would require no change to the original programming interface.

For example, this (nonsense) code

class A {
  private B _b;

  // constructor
  this A() {
    _b = new B()
  }
}

unittest {
  // B can not be mocked
  A a = new A();
}

would require refactoring in order for _b to be mockable.

class A {
  private B _b;

  // constructor
  this A(B b) {
    _b = b;
  }
}

unittest {
  // B could be mocked
  B b = new BMock(...);
  A a = new A(b);
}

But in a unit-testing DSL, one should be able to mock any object, in this case B, without changing the source code first. This is handy for dealing with the unit-testing paradox: Refactoring requires a unit-testing harness to make sure no functionality gets broken, but unit-testing requires testable code; So what to do when the code isn’t testable? A unit-testing DSL would make it easier to put up the initial testing harness.

Also, as Michael points out, a unit-testing DSL could be used to mock any kind of construction, not just objects: Functions and methods for instance. Oh man, could I have use for such a feature?

To give us an image of a DSL for unit-testing in a non-object-oriented language like C, Michael provides this example:

function send_port_command with 90, “CMD: 12”
            calls io_mode which returns M_READ
            calls set_mode with M_WRITE
            calls write_byte with 90
            calls write_bytes with “12”
            returns E_OKAY

That would be testing a function like this:

status send_port_command(byte port, const char *message)
{
  if(io_mode() == M_READ)
    set_mode(M_WRITE);
  write_byte(port)
  write_bytes(translate_command(message));
  return E_OKAY;
}

I have a problem with his example though. In my opinion the test-code resembles the target code a little too much, like a bad manager performing low-level supervision. Too detailed testing beats the purpose since it makes changes more difficult. My philosophy is that test-code should test WHAT the code does, and not bother too much on the HOW.

So, my not so thought through proposal, using Michaels example, would be something like this:

TEST send_port_command

MOCK write_byte(port)
EXPECT port == 90

MOCK write_bytes(bytes)
EXPECT bytes == "12"

CALL send_port_command with 90, "CMD: 12"
EXPECT E_OKAY

Of course there should be support for more advanced mock features like call counting:

MOCK  write_bytes(bytes)
EXPECT   "12", "13"

CALL send_port_command with 90, "CMD: 12"
EXPECT E_OKAY
CALL send_port_command with 90, "CMD: 13"
EXPECT E_OKAY

or

MOCK  write_bytes(bytes)
EXPECT  2 CALLS

or sequential values

MOCK  io_mode
RETURN  M_READ, M_WRITE

Implementing the DSL would be a hefty task though. But, the problems aside, how would your unit-testing DSL be like? I’d be very interested to hear your opinions.

Cheers!

I just read a post by Uncle Bob that discusses the optimal length of a function. He quite correctly claims that the old functions-should-fit-on-a-screen rule lost its validity. He further states that “A well written function might be 4, 5, perhaps 8 lines long; but no longer.”

I’m not saying Uncle Bob is wrong, most well written functions are indeed short, but specific guidelines like that always makes me hesitant. The problem I see is that these rules tend to become an end in themselves, while the real purpose – the reason that the rule was once formulated – end up in the background.

I mean, you shouldn’t make a function short for the sake of making it short. A short function is not necessarily well-written, and a well-written function is not necessarily short. So, what makes a function well-written? Well, here’s my definition:

A piece of code is well-written if you with little effort can figure out what it does.

Keeping a function short certainly helps, but what really matters is how well it reflects intention and how well it hides details. So, don’t think lines of code, think abstraction.

Cheers!

I’m back from my three week vacation!

I had a great time, but as suspected I wasn’t able to stay away from computers. In the warm evenings, just for fun, I started to implement a ray tracer in the D Programming Language.

I have been looking for a suitable project that would give me a chance to get deep into D, and a ray tracer seems to be the perfect fit. D is supposed to be great at floating point programming and now I have the chance to find out on my own.

To make it a little more interesting I have used a more top-down breath-first kind of approach than I normally do. I want to see how that affects a test-driven development technique. As a part of the experiment I keep a detailed development log which I plan to share with you when I reach a certain point. It could be within a week or take several months depending on work load and inspiration level.

So stay tuned. I’ll be back with ray tracing, or other topics that comes across my sphere of interest.

Cheers!

It’s a well known fact that we regularly introduce errors with the code we write. Chances are slim to get it right on the first try. If we do, the risk is great that changing requirements and murdering deadlines will mess things up later on.

It’s also well known that the cost of failure increases with time. The sooner you discover the flaw, the easier it is to fix. In other words, if we are going to fail, there are good reasons to do it fast.

When developers talk about failing fast they usually refer to the defensive coding technique that is based on assertions and exception handling. It’s true that assertions are the very foundation of failing fast, they should be your first line of defense against bugs. But it doesn’t stop there. Failing fast should pervade your whole process of making software. You should fail fast on all levels.

The most effective fail fast-technique is automated testing, the fastest way to get feedback. Be sure to write the tests first. And don’t just automate unit-testing; integration and acceptance testing are often easier to automate than you might think. The key is to isolate your code using mock objects.

The fail-fast property doesn’t apply to code and systems alone. It should be used on the project level too. By using agile practices like short iterations, small releases, and on-site customers you create an environment of continuous feedback. It will help you steer the project to success, or – by failing fast – avoid a disaster. Kate Gregory puts it this way in a nice post from 2005:

“Failure can be a good thing. If it saves you from following a doomed path for a year, you’re glad to have failed early rather than late.”

Failing takes courage, failing increases your knowledge, failing calls upon action. That’s why I like the habit of Failing Fast.

This was the fifth post in this series. Here are the other Tools of The Effective Developer posts:

1. Tools of The Effective Developer: Personal Logs
2. Tools of The Effective Developer: Personal Planning
3. Tools of The Effective Developer: Programming By Intention
4. Tools of The Effective Developer: Customer View

This is the third post in my Tools of The Effective Developer series. The other two discussed the habit of keeping personal logs, and the habit of daily planning. Now the time has come to the habit of programming by intention.

Please note that I’m not speaking of intentional programming, which by the way introduces some interesting concepts. Instead I refer to the simple kind that only requires your brain and the programming language of your choice.

So what is programming by intention? Well, I like to describe it this way: if you are to implement a new set of functionality, and you don’t start with the implementation, nor the programming interface, but you start writing the code that is going to use it – then you are programming by intention.

Why is this way better? The short answer is: Because we are lazy. The long answer is: We humans usually seek the simplest solution. Start with the utilizing code and you should find yourself pushing complex code and decisions right where it should be: behind the programming interface, nicely encapsulated. We also focus more on the task at hand, which decreases the risk of committing the sin of premature generalization.

Programming by Intention is probably the easiest effective developer habit to acquire. It can be done at all levels of abstraction and there is no overhead cost. Therefor it’s my favorite habit.