What happened, did I come to my senses or go crazy (pick the expression that fits your point of view) during Christmas holidays? No, I still think that unit-testing user interfaces is most likely a waste of time, but for some special occasions, like this one, they will pay off.

My task is to make changes to a tool box control in a legacy application. The control is full of application logic and it has a strong and complicated relationship to an edit area control. There are two things I need to do:
First I have to pull out the application logic of the tool box and break the tight relationship to the edit area. Then I need to push that application logic deeper into the application core, so that the tool box could be used via the plug-in framework.

I figured I had two options. I could either refactor the tool box and make the changes in small steps, or I could rebuild the complete tool box logic and then change the controls to use the new programming interface.
I found the rebuild alternative too risky. The code base is full of booby traps and I have to be very careful not to introduce bugs. Therefore I decided to go with refactoring.

But refactoring requires a unit-testing harness, which of course this application doesn’t have. Trust me; you don’t want to refactor without extensive unit-testing, so here I am setting up the tests around the involved GUI controls. It’s a lot of work, especially since I don’t have a decent framework for creating mock objects, but it’ll be worth the effort once I start messing around with the code.

As a conclusion I’d like to refine the “Don’t unit-test GUI” statement to read “Don’t unit-test GUI unless you’re refactoring it, and there’s no easier way to make sure your changes doesn’t break anything.”

Cheers!

I love TDD, it’s one of the most powerful development techniques I know, but it’s not without limitations. For one thing, code with unit-tests attached is more difficult to change. This is often an acceptable price to pay since the benefit of producing and having the unit-tests is usually greater.

But the return of the investment isn’t always bigger than the price, and sometimes the cost of change exceeds the benefit of protection. That’s why most developers won’t use TDD for experimental code, and that’s why I’m not using it to test my user interfaces.

I prefer to develop GUIs in a RAD environment, visually dragging and dropping components, moving them around, exchanging them for others if better ones are to be found. In that process unit-testing just gets in my way. After all, the GUI is supposed to be a thin layer, without business logic, so there is only so much to test.

One could theoretically test that the form contains certain key components, that they are visible, have a certain position or layout, stuff like that – but I find that kind of testing too specific for my taste.

In my opinion, unit-testing should test functionality, not usability. It shouldn’t dictate whether I decide to show a set of items in a plain list or in a combo-box. What it should do, is test that the business logic of my code produce the correct set of items, and leave the graphical worries to the testers.

This brings us to something that is sometimes misunderstood: Unit-testing can never replace conventional testing. Some things are just better left to humans. Like testing user interfaces.

Cheers!

Developers from the first group constantly create mess and agony among fellow developers. The second group contain developers that never produce anything of value since they care more for the code than they do for the result. The optimal balance is somewhere in between, but regardless of what type of developer you are: you should always start by making it work, meaning implement the simplest solution first.

Why spend time on an implementation that isn’t likely to be the final one, you might ask. Here’s why:

1. The simple solution helps evolving the unit-testing safety net.
2. The simple solution provide rapid feedback, and may prevent extensive coding of the wrong feature. It is like a prototype on the code level.
3. The simple solution is often good enough, and – with a working solution ready – you are less inclined to proceed and implement a more complex solution unless you really have to. Thus avoiding premature optimization and premature design, that makes you add features that might be needed in the future.
4. With the simple solution in place, most integration and programming interfacing is done. That makes it easier to implement a more complex solution.
5. While implementing the simple solution, your knowledge of the system increases. This helps you make good decisions later on.

This may all sound simple enough to you. After all, the habit of Making It Work First comes naturally to many developers. Unfortunately, for me, I’m not one of those developers. I still let more or less insignificant design issues consume an unnecessary amount of time. The thing is, it is hard to find the perfect design on the first try. The perfect design may not even exist, or cost too much to be worth the effort.

That is why I struggle to attain the habit of Making It Work First.

Previous posts in the Tools of The Effective Developer series:

1. Tools of The Effective Developer: Personal Logs
2. Tools of The Effective Developer: Personal Planning
3. Tools of The Effective Developer: Programming By Intention
4. Tools of The Effective Developer: Customer View
5. Tools of The Effective Developer: Fail Fast!

I had a great time, but as suspected I wasn’t able to stay away from computers. In the warm evenings, just for fun, I started to implement a ray tracer in the D Programming Language.

I have been looking for a suitable project that would give me a chance to get deep into D, and a ray tracer seems to be the perfect fit. D is supposed to be great at floating point programming and now I have the chance to find out on my own.

To make it a little more interesting I have used a more top-down breath-first kind of approach than I normally do. I want to see how that affects a test-driven development technique. As a part of the experiment I keep a detailed development log which I plan to share with you when I reach a certain point. It could be within a week or take several months depending on work load and inspiration level.

So stay tuned. I’ll be back with ray tracing, or other topics that comes across my sphere of interest.

Cheers!

Acceptance testing is done by the customer to make sure she got what she ordered, to make sure you delivered the right system. This process cannot and should not be automated. What you can do – and this was what I meant to say – is to automate acceptance tests, and use them during construction.

The customer defines the acceptance tests. These are valuable to the developer since they can be used to validate the system as it develops. The sooner you get a hold on these test cases the better, so make sure you press the customer to produce them early. Better yet, help the customer in the process. That way you can help making the test cases automatable.

So, how do you design a test so that it can be automated? First of all you need to stop thinking in terms of user interface. Instead you should describe the test, in plain text, and in terms of action, function and result. Look for possible test data, edge cases, exceptional uses and describe those as well.

The second thing you need to do is to make the system in such a way that it can be automated. Separating the GUI from the business logic is often all you need to do to achieve that. You then automate the acceptance tests in the same way as you automate integration tests. In fact, they could even become a part of your integration testing harness. The only difference being the fact that they are defined by the customer.

I’m not saying this is easy. (Here’s a nice post which discusses when to automate.) I’m not saying it can be done for all of your acceptance tests. What I say is: it can be done for far more tests than you might think. For example, we are customizing a GIS system for an internal customer’s needs. The application is user controlled and involve plenty of complex actions, modifying graphical data and properties. Still we were able to automate most of the acceptance tests.

We spent a lot of time writing code to set up fixtures, initiate actions and check the results. It was worth every second though. You see, manually running one of our acceptance test cases usually takes several minutes. By having the computer do all the work, time is reduced significantly and free up developer time. We use the automated test cases individually, almost like an extended compiler, to verify features as we implement them. And at night we run all of our acceptance tests to get feedback on how far we’ve come and to spot unexpected problems.

But the real acceptance testing is still going to be done manually, by the customer, at the end of the project. Just like AC pointed out.

Cheers!

It’s also well known that the cost of failure increases with time. The sooner you discover the flaw, the easier it is to fix. In other words, if we are going to fail, there are good reasons to do it fast.

When developers talk about failing fast they usually refer to the defensive coding technique that is based on assertions and exception handling. It’s true that assertions are the very foundation of failing fast, they should be your first line of defense against bugs. But it doesn’t stop there. Failing fast should pervade your whole process of making software. You should fail fast on all levels.

The most effective fail fast-technique is automated testing, the fastest way to get feedback. Be sure to write the tests first. And don’t just automate unit-testing; integration and acceptance testing are often easier to automate than you might think. The key is to isolate your code using mock objects.

The fail-fast property doesn’t apply to code and systems alone. It should be used on the project level too. By using agile practices like short iterations, small releases, and on-site customers you create an environment of continuous feedback. It will help you steer the project to success, or – by failing fast – avoid a disaster. Kate Gregory puts it this way in a nice post from 2005:

“Failure can be a good thing. If it saves you from following a doomed path for a year, you’re glad to have failed early rather than late.”

Failing takes courage, failing increases your knowledge, failing calls upon action. That’s why I like the habit of Failing Fast.

This was the fifth post in this series. Here are the other Tools of The Effective Developer posts:

1. Tools of The Effective Developer: Personal Logs
2. Tools of The Effective Developer: Personal Planning
3. Tools of The Effective Developer: Programming By Intention
4. Tools of The Effective Developer: Customer View

Fortunately I’m one of the lucky few who have been able to recover from this particularly addictive behavior. I’ve been clean – thank you jUnit – for almost 5 years now. And you can do it too, you can let go of the safety zone that these integrated debuggers provide, and break free just like I did.

The first thing to do is to realize that there is a better alternative: test-driven development. To get rid of a bug, the right thing to do is not to fire up your debugger, but to write a unit-test to reveal it. If necessary, keep writing tests and go deeper and deeper into your code. Eventually the tests will tell you what is wrong, and they’ll even point out a solution for you.

I know that using a debugger may seem like a faster way to find and extinguish a bug, but that is just an illusion. Here are the reasons:

1. TDD improves the design. Being forced to think testability tends to divide your code into small manageable pieces. This will make your code a bad breeding ground for bugs.
2. Tests remain useful for a long time. They become an addition to your testing harness, which helps protecting your code against future infestations. The work spent in a debugging session can never be reused.
3. Unit-testing saves time, a lot! While this isn’t immediately obvious, the long term effects are huge. Think of it: all those debugging sessions can be run automatically at your command, whenever you want, how often you want, and in just a matter of seconds. All you have to do to achieve this is to let go of the debugger, and write relevant tests.
4. Unit-testing gives you courage. There’s nothing like a good harness to make you feel invincible. I still remember the first time I felt the real power of unit-testing. I was working on a huge legacy application and had developed a new set of functionalities, using TDD for the very first time. Several months later I realized I had to do a major rewrite. The rewrite was risky business and took me a couple of days to complete. When finished, I ran the unit-tests which all came out green! I could be confident the program worked just as before the rewrite. And the best part: I drew the conclusion out of just five seconds of testing. Boy, I still get the goose bumps.

[PREACHING OFF]

Of course debuggers are useful tools. In certain occasions they are even invaluable. For someone who’s new to the software they provide a great way of getting to know it. The problem is that a debugger tends to make you lazy. So be sure to get rid of it as soon as you identify a testing strategy.

Cheers!

Now an alternative exists: The D Programming Language is designed to combine the low level programming abilities and performance of languages like C and C++, with the productivity features of modern languages.

You could describe D as C with features like object oriented programming, exception handling, garbage collection and design by contract. Cool agile features indeed, but D has another one that I instantly fell in love with: built in support for unit testing.

In D you can declare unittest blocks.

unittest {
  assert(sometest, "sometest failed");
}

The unittest construct makes it very easy to practice test-driven development. For instance, lets say we want to create a new type for three dimensional vectors. In order to be test-driven we need to start with the test:

unittest {
  Vector3 v;
}

We compile the program using the -unittest switch.

dmd -unittest test.d

Of course we get a compiler error, Vector3 is still undefined. Lets define it.

struct Vector3 {
}

This time the program compiles. Back to the unittest. Now let’s try a simple assignment.

unittest {
  Vector3 v = Vector3(1.0, 2.0, 3.0);
}

This, again, produces a compile time error. Vector3 doesn’t have the x, y and z fields, so we implement them.

struct Vector3 {
  real x, y, z;
}

The code passes compilation. Next step: implement vector addition. We start with the test.

unittest {
  Vector3 v1 = Vector3(1, 2, 3);
  Vector3 v2 = Vector3(3, 2, 1);
  Vector3 vr = v1 + v2;
}

As we expect, the code doesn’t compile. We need to overload the + operator.

struct Vector3 {
  real x, y, z;

  // Overload + operator
  Vector3 opAdd(Vector3 a)
  {
    return Vector3(0, 0, 0);
  }
}

Now the program compiles, but we don’t know if the add operator produces the right result (which it doesn’t with the current code). To check the result we can use assert.

unittest {
  Vector3 v1 = Vector3(1, 2, 3);
  Vector3 v2 = Vector3(3, 2, 1);
  Vector3 vr = v1 + v2;

  assert(vr.x == 4);
  assert(vr.y == 4);
  assert(vr.z == 4);
}

We compile, and it compiles without error. To run the unittest code we simply run the program. Unittest code is executed after program initialization, but before the main function. If a unittest fails the program terminates prematurely. Our program terminates (as expected) with an AssertError. Lets correct the add operator.

struct Vector3 {
  real x, y, z;

  Vector3 opAdd(Vector3 a)
  {
    return Vector3(x + a.x, y + a.y, z + a.z);
  }
}

It compiles and runs without errors. Great!

As you can see, test-driven development is a very smooth and simple process in D. There is no need for a separate framework, just compile and run your application. Also, the test code dwells close to the production code, which makes it easier to maintain and keep up-to-date. In fact, you can put unittest blocks anywhere in your code, even within the piece of code you are testing.

struct Vector3 {
  real x, y, z;

  // unittest blocks are allowed
  // within structs.
  unittest { ... }
}

Any type of code is allowed within a unittest block. This means that you can declare functions to do repetitive tasks.

unittest {
  function assertEqual(Vector3 a, Vector3 b)
  {
    assert(a.x == b.x);
    assert(a.y == b.y);
    assert(a.z == b.z);
  }

  Vector3 v1 = Vector3(1, 2, 3);
  Vector3 v2 = Vector3(3, 2, 1);

  assertEqual(v1 + v2, Vector3(4, 4, 4));
  assertEqual(v1 - v2, Vector3(2, 0, -2));
}

The test code is only included in the executable if it is compiled with the unittest flag, so there’s no need for a separate test project or conditional compilation. This is a very clean solution, highly suitable for a traditional language that compiles directly to machine code. Although I’m a big fan of testing framworks such as JUnit, I find it much easier to work with the built in unit testing features of D. Of course you don’t have advanced features like mock object support, but I guess that will be offered soon with some kind of unittest-based framework add-on.

If you have doubts about the foundation of the D Programming Language, you should be relieved to hear that it’s been designed by Walter Bright, who have spent almost a lifetime constructing C and C++ compilers.

You’ll find the complete code within my code sample pages.