www.hans-eric.com

I’m reading Jim Highsmith’s book Agile Project Management. It’s a good book, as the frequent use of my marker pen shows, except for maybe this one sentence:

“APM reliably delivers innovative results to customers within cost and schedule constraints.”

Let me take that sentence out of its context and make a point I believe needs to be stressed. Software development methodologies (Agile Project Management in this case) delivers nothing! People, on the other hand, do.

Some years ago I developed a special interest in software development methodologies. Since then I’ve spent much time reading literature on Scrum, eXtreme Programming, etc, aiming to optimize the processes we use in our projects. During that time I’ve learned a lot, but also come to realize that I’ve been focusing too much on the wrong things.

In the past I thought the problems we were experiencing were problems with the methodology, an easy conclusion to draw from the propaganda-like information out there. However, I’ve found that the methodology matters little in comparison to the quality of my project team. Today I focus more on the team, the product, and the customer, and less on the latest within agile.

Thus, my formula for a successful project is this:

• Spend less energy on the methodology; i.e. pick a simple one and adapt it when (and only when) needed, or stick to the one you’re currently using
• Do more to get the right people
• Make sure they are motivated and connected
• If you think you’ll fail, do it fast

Summary

If we focus too much on the methodology, and give it too much importance, we risk loosing sight on the real goal, namely producing the right system. And for that you need the right people.

Cheers!

Exception handling has truly been a blessing to us software developers. Without it, dealing with special conditions and writing robust programs was a lot more painful. But, like any powerful tool, badly used it could cause more harm than good. This article name the top three on my Exception handling bad practices list, all of which I’ve been practicing in the past but now stay away from.

Swallowing Exceptions

Have you ever come across code like this?

try
{
  DoSomeNonCriticalStuff();
}
catch (Exception e)
{
  // Ignore errors
}

DoStuffThatMustBeDoneDispiteAnyErrorsAbove();

Of all the bad exception handling practices, this is the worst since its effect is the complete opposite of the programmer’s intention. The reasoning goes something like this: Catching exceptions where they don’t hurt makes my program more robust since it’ll continue working even when conditions aren’t perfect.

The reasoning could have been valid if it wasn’t for Fatal exceptions; Here described by Eric Lippert.

Fatal exceptions are not your fault, you cannot prevent them, and you cannot sensibly clean up from them. They almost always happen because the process is deeply diseased and is about to be put out of its misery. Out of memory, thread aborted, and so on. There is absolutely no point in catching these because nothing your puny user code can do will fix the problem. Just let your “finally” blocks run and hope for the best.

Catching and ignoring these fatal exceptions makes your program less robust since it will try to carry on as if nothing happened in the worst of conditions, most likely making things worse. Not very Fail fastish.

So, am I saying that ignoring exceptions is bad and should always be avoided? No, the bad practice is catching and ignoring general exceptions. Specific exceptions on the other hand is quite OK.

try
{
  DoSomeNonCriticalStuff();
}
catch (FileNotFoundException e)
{
  // So we couldn't find the settings file, big deal!
}

Bad example, I know, but you get the point.

Throwing Exception

Here’s another bad practice I come across every now and then.

throw new Exception("No connection!");

The problem is that in order to handle Exception we have to catch Exception, and if we catch Exception we have to be prepared to handle every other type of Exception, including the Fatal exceptions that we discussed in the previous section.

So, if you feel the need to throw an exception, make sure it’s specific.

throw new NoConnectionException();

If the idea of defining lots of specific exceptions puts you off, then the very least thing you should do is to define your own application exception to use instead of the basic Exception. This is nothing I recommend though, since general exceptions, like ApplicationException, violate the Be specific rule of the Three rules for Effective Exception Handling.  It’ll make you depend heavily on the message property to separate different errors, so don’t go there.

Overusing exceptions

Exceptions is a language construct to handle exceptional circumstances, situations where a piece of code discovers an error but don’t have the context necessary to handle it. It should not be used to handle valid program states. The reasons are:

1. Performance. Throwing an exception with all that’s involved, like building a stack trace, will cost you a second or so.
2. Annoyance. Debugging code where exceptions are a part of the normal execution flow can be frustrating.

Eric Lippert calls those exceptions Vexing exceptions, which I concider a great name given the second argument. Make sure you’ll check out his article (link at the beginning of this article).

…

Those were the three misuses of exception handling I concider worst. What’s on your list?

Cheers

I often come across code with little or no infrastructure for error handling. This is a big mistake, one that’ll make you pay increasingly as the code base grows. Why? Because your code’ll end up with loads of this:

try
{
  ParseText(SomeText);
}
catch (Exception e)
{
  MessageBox.Show("Error while parsing.", "My Application",
         MessageBoxButtons.OK, MessageBoxIcon.Error);

  // Skipping the rest due to the error
  return;
}

It may look OK but there are several problems with the error handling code above, some more severe than others.

• Code duplication
  A message box for displaying error messages to the user should almost always have an error icon, the application name as the title, and an OK button. Repeating this all over the code base is not very DRY and should be avoided.
• Inconsistency
  A related problem is that when you leave programmers with only the general error handling routines of the platform chances are they’ll end up doing things differently, resulting in an inconsistent user experience; For example showing some error dialogs with an error icon and some without.
• Information loss
  A more serious problem is the fact that the code in the example, by ignoring the exception object, drops information that could be important to track down a bug. We could of course mititgate that problem somewhat by showing the error message of the exception to the user (by concatinating e.Message), but we’d still miss an important piece of information: the stack trace.
  Since dumping a pile of function calls in the face of the users is a no no, what we’d really need is a way to put the details in a non-intrusive place, for example a log file. If the Error Handling Infrastructure doesn’t make this easy, we’re likely to leave that kind of information out completely. For shame.
• Not automation-friendly
  But the most severe problem with a spread out usage of UI displaying methods like MessageBox is that it makes your code impossible to automate. If someone has to monitor a scheduled run of for example unit tests, and every now and then klick an OK button, it kind of beats the purpose of automation.

So here’s my advice: Implement a strategy for handling errors at the earliest possible time. That is, right after setting up continuous integration for the Hello world application.

Properties of an Error Handling Infrastructure
So how should a error handling infrastructure be like? There’s only one rule. It has to be easy!
If it isn’t simple to use, it won’t be used, and programmers end up using the general purpose message showing methods again, or worse, not doing any error handling at all.

I usually build my error handling interfaces around two use cases.

1. Displaying error messages
2. Logging error messages

These are often used in combination, for example

try
{
  ParseText(SomeText);
}
catch (Exception e)
{
  ApplicationEnvironment.ShowErrorMessage("Error while parsing: " + e.Message);
  ApplicationEnvironment.LogError(e);
  return;
}

Or, to avoid code duplication, combined into a single convenient method.

try
{
  ParseText(SomeText);
}
catch (Exception e)
{
  ApplicationEnvironment.HandleException("Error while parsing", e);
  return;
}

When you design your error handling interfaces don’t be afraid to add lots of convenient methods. Remember, it has to be easy to use and that’s what convenient methods are all about, as opposed to utility methods who generally need to be given more arguments. In this case I prefer the Humane interface design style instead of a minimal interface approach.

Needs to be configurable
Furthermore, the Error Handling Infrastructure needs to be configurable or support some other kind of dependency breaking technique like dependency injection. For instance, you should be able to mute all error messages intended for the user, and instead write them to the error log. This way your code will be able to run in a scripted environment, like during unit testing.

There are many ways to design an Error Handling Infrastructure. You could create your own message dialogs allowing the user to send a report right away, you could use the built in application log of the operating system or just plain text files, etc etc.

Whatever you choose to do, don’t forget to do it early and make it easy.

Cheers

Previous posts in the Tools of The Effective Developer series:

1. Tools of The Effective Developer: Personal Logs
2. Tools of The Effective Developer: Personal Planning
3. Tools of The Effective Developer: Programming By Intention
4. Tools of The Effective Developer: Customer View
5. Tools of The Effective Developer: Fail Fast!
6. Tools of The Effective Developer: Make It Work – First!
7. Tools of The Effective Developer: Whetstones
8. Tools of The Effective Developer: Rule of Three
9. Tools of The Effective Developer: Touch Typing

As I noted in my previous post, anonymous methods is a big new feature in Delphi 2009 for the Win32 platform. While “closures” is natural and much appreciated in other languages, most notably Ruby, the Delphi community is still a bit reluctant and hesitant. Segunda Feira put words on it in his post on the subject.

I am still not convinced that this convenience is worth getting all that excited about – It has not enabled anything that was previously impossible, nor even especially difficult either to implement or to understand

[...]

anonymous methods should be used only when you absolutely have to, and so far I have not yet seen an example where anonymous methods are the only way to achieve anything

I agree that more often than not a problem can be solved with equal elegance using a pure object orientated approach, but there are situations where anonymous methods may actually be the better alternative. One situation that comes to mind is the setting up of test fixtures.
Say, for instance, that we want to test the event firing mechanism of a component. An easy way to set this up could be like the following.

procedure TestSomeComponent.TestOnChange;
var
  SUT: TSomeComponent;
  OnChangeFired: Boolean;
begin
  OnChangeFired := False;
  // Set up the fixture
  SUT := CreateTheComponentToTest;
  SUT.OnChange :=
    procedure(ASender: TObject)
    begin
      OnChangeFired := True;
      CheckSame(SUT, ASender, 'The Sender event argument should be set');
    end;
  // Perform operation
  SUT.Text := 'Some text';
  // Check the result
  CheckTrue(OnChangeFired, 'Setting the Text property should fire the OnChange event');
end;

The above code checks that the component’s OnChange event is fired when the Text property is set, and that the component is set as the Sender reference to the event handler. Except for being more compact, the biggest advantage to using anonymous methods in this scenario is that we avoid the obscure test smell and that we don’t have to clutter our test class with an instance variable (like FOnChangeFired) to identify if the event was really fired or not.

The only problem is: it doesn’t work. Why? Well, because the OnChange property is most likely of an instance method reference type (i.e. TNotifyEvent), meaning it doesn’t accept a references to an anonymous method even if it has the same signature.

type TNotifyEvent = procedure(ASender: TObject) of object;

For our code to compile we need to redeclare TNotifyEvent and remove the “of object” keywords and instead use the method reference type.

type TNotifyEvent = reference to procedure(ASender: TObject);

But of course, that’s not an option. It would mean that the traditional way of setting up an event handler (by using an instance method) would not work.

I see a definite problem with how the Delphi language explicitly forces you to distinguish between instance methods (and class methods for that matter) and anonymous method references, even though they share the same signature.
This is most unfortunate since I feel that the situations where we’d like to support both kinds of event handlers are quite common. And with the current semantics we have to use code duplication in order to achieve that. Like the two Synchronize methods of the built in TThread class.

In Delphi 2009 an overloaded variant of the TThread.Synchronize method was introduced, one that make use of anonymous methods. Here are snippets from that code:

type
  TThreadMethod = procedure of object;
  TThreadProcedure = reference to procedure;
...
  class TThread = class
    …
    procedure Synchronize(AMethod: TThreadMethod); overload;
    procedure Synchronize(AThreadProc: TThreadProcedure); overload;
    ...
  end;

The two methods have almost identical implementations. The only real difference is the type of the argument.

procedure TThread.Synchronize(AMethod: TThreadMethod);
begin
  FSynchronize.FThread := Self;
  FSynchronize.FSynchronizeException := nil;
  FSynchronize.FMethod := AMethod;
  FSynchronize.FProcedure := nil;
  Synchronize(@FSynchronize);
end;

procedure TThread.Synchronize(AThreadProc: TThreadProcedure);
begin
  FSynchronize.FThread := Self;
  FSynchronize.FSynchronizeException := nil;
  FSynchronize.FMethod := nil;
  FSynchronize.FProcedure := AThreadProc;
  Synchronize(@FSynchronize);
end;

I may be overly sensitive, but code like that really disturbs me. Unfortunately it gets worse. If we follow the execution chain down into the Synchronize class method that is invoked by the two, we find this.

class procedure TThread.Synchronize(ASyncRec: PSynchronizeRecord; QueueEvent: Boolean = False);
var
  SyncProc: TSyncProc;
  SyncProcPtr: PSyncProc;
begin
  if GetCurrentThreadID = MainThreadID then
  begin
    if Assigned(ASyncRec.FMethod) then
      ASyncRec.FMethod()
    else if Assigned(ASyncRec.FProcedure) then
      ASyncRec.FProcedure();
    end else
      …
end;

It would be a lot nicer if the two reference types were joined under a common reference type. And I can’t see why it couldn’t be done. When I look at the “of object” keywords I get a feeling that the language is leaking compiler implementation through the language interface; information that is indifferent to the developer. What matters from a callers’ perspective is the method signature, not whether the method has a self pointer or not.

I hope CodeGear recognizes this problem and find a way to clean this assymetry from the language. Anonymous methods would be so much more useful if they do.

Cheers!

Jay Fields has a post up today where he makes a distinction between Developers and Hackers.

Time after time I see requirements gathered and presented in a way that is totally disconnected from the business problem that’s being addressed. There are two ways to handle the situation.

1. Write the best code you can.
2. Talk to the business.

Hackers generally go for the first choice, which doesn’t guarantee failure. In fact, a good hacker can still deliver code quickly enough that the business will be happy. Often, he gets it right on the 2nd or 3rd try.

But, a developer can deliver what the business is looking for the first time. A(n often) quick conversation with the business ensures that the developer knows what to work on and how it will benefit the business. This dialog often leads to a superior solution implementation where the business gets what it wants and the developer is able to do it in the most efficient way (e.g. I don’t need a beautiful website, I need a command line application).

It’s essentially the same distinction I tried to make a while ago, although I used the term Programmer instead of Hacker. Maybe if I’d used the term Hacker as well, I might have avoided some of the heat I took for those definitions

Interestingly Jay too suggests — although he doesn’t say it outright — a definition where human interaction and taking the Customer’s view is what distinguishes a developer, rather than coding skills.

Cheers!

I’m back from the ESRI Developer Summit. While suffering from severe jet lag I’ve spent the last couple of days in slow reflection. The biggest impact the conference had on me was a keynote held by Alan Cooper.

Early in his talk he put me in a defensive mode by stating that agile processes are bad for developing quality software. Alan means that the idea of little or no upfront design is ridiculous and will result in either expensive development costs or crappy software.
Instead he believes in having all outstanding uncertainties removed with a thorough and detailed design, thus developing a “blueprint” for a production team to follow. Additionally, in opposition to the agile manifesto, change is nothing he seems to embrace.

Most business executives believe that writing production code is a good thing. They assume that getting to production coding early is better than getting to it later. This is not correct. Writing production code is extremely expensive and extremely permanent. Once you’ve written it, it tends to stay written. Any changes you might make to production code are 1) harmful to the conceptual integrity of the code, and 2) distracting and annoying to the programmers. Annoying your programmers is more self-destructive to a company than is annoying the Board of Directors. The annoyed programmers hold your company’s operations, products, and morale in the palms of their hands.

So he wants us to go back to Waterfall. Doesn’t that give me the right to discard his thoughts without further reflection? No, I don’t think so.

It’s easy to forget that the “traditional” development processes were not created to make our lives as developers miserable. They emerged from common knowledge of that time, and they were formulated to address real problems. We would be foolish to disregard that experience from the past.

Let me be clear with one thing. I don’t agree with Alan. I do believe we can produce high quality software with agile methods, where design evolves with the production code. But I did, after my initial defensive reflex, find his perspective refreshing.

Alan’s talk is not published anywhere, but the general ideas are documented on his company’s website.

Software construction is slow, costly, and unpredictable.
[…]
Unpredictable is by far the nastiest of these three horsemen of the software apocalypse. Unpredictable means 1) you don’t know what you are going to get; and 2) you won’t get what you want. In badness, slow and costly pale in comparison to unpredictable.
[…]
The key, it seems, is vanquishing unpredictability, which means determining in advance what the right product design is, determining the resources necessary to build it, and doing so. As the airline pilots say, “Plan your flight, and fly your plan.”

Alan’s solution to the development problems of today is to divide work into three separate fields of responsibilities, something he calls “The Triad”.

Interaction design is design for humans, design engineering is design for computers, and production engineering is implementation. Recognizing these three separate divisions and organizing the work accordingly is something I call “The Triad.” While it cannot exist without interaction designers, it depends utterly on teasing apart the two kinds of engineering which today, in most organizations, are almost inextricably linked. It will take some heroic efforts to segregate them.

Collaboration with the customer (or users), as the agile methodologies suggest, is out of the question according to Alan. Why let the least qualified make the most important decisions, he reasons. Instead, Alan Cooper advocates the use of interaction designers (HCI experts). Thus, he identifies three key roles: design engineers, production engineers and interaction designers.

Production engineers are good programmers who are personally most highly motivated by seeing their work completed and used for practical purposes by other people. Design engineers are good programmers who are personally most highly motivated by assuring that their solutions are the most elegant and efficient possible.
Interaction designers’ motivations are very similar to those of design engineers, but interaction designers are not programmers. Although most programmers imagine that they are also excellent interaction designers, all you have to do to dissuade them of this mistaken belief is to explain that interaction designers spend much of their time interviewing users.

Alan doesn’t rule out agile methods completely. He thinks they have a place, but only as a part of the design process.

Currently there is a pitched battle raging in the programmer world between conventional engineering methods and Agile methods. What neither side sees is a path of reconciliation; where Agile and conventional methods can effectively coexist. Providentially, the Triad reconciles them very well. The lean, iterative, problem-solving work of the software design engineer is the archetype of Agile programming. The purposeful, methodical construction work of the production engineer is the quintessence of conventional software engineering, particularly the type espoused by disciples of Grady Booch’s Rational Unified Process, or RUP. Both methods are correct, but only when used at the correct time and with the correct medium.

Despite Alan’s thought provoking keynote, I’m still a believer of agile methods for the whole development process. I think it’s possible to build rigid software with little upfront design, a readiness for change, rapid feedback and customer collaboration. The problem I see is that it demands a lot more from us developers. Knowing the language and how to program the platform is no longer enough. We need system and interface design skills, as well as social skills. We also need to master important but difficult techniques like unit testing and code refactoring.

Maybe agile is only for teams of elites?

Paul W. Homer has a post up discussing essential development problems. While this is a huge subject (somewhat reflected by the size Paul’s article,) I’d like to emphasize one of the things he touches: the lack of understanding the users’ problem domain.

The biggest problem in most systems today is the total failure of the programmers to have any empathy for their users.

I too have a similar perception, and I have many times been guilty as charged, although I’ve come to realize that I’d better take the Customer View if I want to succeed. Still, I fail from time to time in the analysis of the users’ situation. I don’t know why this keeps happening, but I think impatience and a strong wish to “get going” is partly to blame.

One part of the problem could also be how many of us look upon system development. We usually decide for technology at an early stage, often on grounds that matters little to the end users. Things like personal preference and hype usually have a big impact on the choices we make. We then build the system pretty much bottom-up, adapting the business process to fit the technology. I’ve seen this unspoken philosophy many times in many places, and the results are usually poor.

The cure is a change of paradigm. We need to start with the users, identify and help develop their processes before we build systems that support them. We need to realize that the system is not the solution; it’s just a part of it.

Another part of the problem is a question of attitude. We need to accept that we’re actually in the support business. Our job is to make things easier for others, not to take the easy way out on the users’ expense, as Paul also points out.

“Use this because I think it works, and I can do it”, produces an arrogant array of badly behaving, but pragmatically convenient code. That of course is backwards, if the code needs to be ‘hard’ to make the user experience ’simple’ then that is the actual work that needs to be done. Making the code ’simple’, and dismissing the complaints from the users, is a cop-out to put it mildly.

Of course, there is a trade-off between what users want and what we can provide, but at least we need to start at the right end.

Cheers!

Isn’t life funny? Two weeks ago I stated my opinion that unit-testing graphical user interfaces isn’t worth the trouble. Now I find myself doing it, writing unit-tests for GUI components.

What happened, did I come to my senses or go crazy (pick the expression that fits your point of view) during Christmas holidays? No, I still think that unit-testing user interfaces is most likely a waste of time, but for some special occasions, like this one, they will pay off.

My task is to make changes to a tool box control in a legacy application. The control is full of application logic and it has a strong and complicated relationship to an edit area control. There are two things I need to do:
First I have to pull out the application logic of the tool box and break the tight relationship to the edit area. Then I need to push that application logic deeper into the application core, so that the tool box could be used via the plug-in framework.

I figured I had two options. I could either refactor the tool box and make the changes in small steps, or I could rebuild the complete tool box logic and then change the controls to use the new programming interface.
I found the rebuild alternative too risky. The code base is full of booby traps and I have to be very careful not to introduce bugs. Therefore I decided to go with refactoring.

But refactoring requires a unit-testing harness, which of course this application doesn’t have. Trust me; you don’t want to refactor without extensive unit-testing, so here I am setting up the tests around the involved GUI controls. It’s a lot of work, especially since I don’t have a decent framework for creating mock objects, but it’ll be worth the effort once I start messing around with the code.

As a conclusion I’d like to refine the “Don’t unit-test GUI” statement to read “Don’t unit-test GUI unless you’re refactoring it, and there’s no easier way to make sure your changes doesn’t break anything.”

Cheers!

Christmas for me is celebration, gormandizing and the joy of happy children. It’s also a time for rest and reflection, and when I think back on this year there is one thing that disturbs me: for me this was a year of us and them.

The company I work for is divided; we have three divisions divided into several so called “functions”, which in turn is divided into teams, where people mostly work – well – alone… And on top of that we have several temporary project organizations fighting for the scarce resources.

Although we’re trying to change things for the better, there is still a lot of distrust and arrogance that stems from lack of understanding between the various formations. We are an organisation that needs more empathy.

My guess is that this is a common problem in our business. We tend to mentally separate us from the others, thinking in terms of us and them. There’s a lot of “those stupid users”, and “management has no clue” kind of talk out there.

We even turn on ourselves. What’s the first thing you do when you look at code that is new to you? I know my reflex is to try and spot design flaws, so that I can point them out. If I can’t understand the code fast enough, the normal response is “who created this piece of crap?”

These responses only show a lack of empathy in the person that utters them. Nowadays I try not to be so judgemental when I look at code that others have created. I’ve come to realize that while there is always a reason for crappy code, it’s never because the programmer wanted to do a bad job. He or she has most likely tried their best, with their current knowledge, or made a deliberate choice, given the circumstances. At least they deserve respect for that.

When we start talking negatively about each other, in terms of us and them, it might be because we lost track of the bigger picture. Here’s an exercise that helps me when I need to put things into perspective. Look at this image from Big Sky Astronomy Club.

The image was taken by Voyager 1 in 1990 on its way out of the solar system. The little blue dot is planet earth. Fellows, that pixel is our home!

In this humbling perspective there is no us and them; There’s only us.

Cheers!

I’m currently rereading parts of the book Test-Driven Development: A Practical Guide, by David Astels. It’s a great book in many ways, well worth reading, but I have objections to one particular section in the book.
The author tries to convince me that developing my user interfaces using a test-driven approach is a good thing to do. I disagree.

I love TDD, it’s one of the most powerful development techniques I know, but it’s not without limitations. For one thing, code with unit-tests attached is more difficult to change. This is often an acceptable price to pay since the benefit of producing and having the unit-tests is usually greater.

But the return of the investment isn’t always bigger than the price, and sometimes the cost of change exceeds the benefit of protection. That’s why most developers won’t use TDD for experimental code, and that’s why I’m not using it to test my user interfaces.

I prefer to develop GUIs in a RAD environment, visually dragging and dropping components, moving them around, exchanging them for others if better ones are to be found. In that process unit-testing just gets in my way. After all, the GUI is supposed to be a thin layer, without business logic, so there is only so much to test.

One could theoretically test that the form contains certain key components, that they are visible, have a certain position or layout, stuff like that – but I find that kind of testing too specific for my taste.

In my opinion, unit-testing should test functionality, not usability. It shouldn’t dictate whether I decide to show a set of items in a plain list or in a combo-box. What it should do, is test that the business logic of my code produce the correct set of items, and leave the graphical worries to the testers.

This brings us to something that is sometimes misunderstood: Unit-testing can never replace conventional testing. Some things are just better left to humans. Like testing user interfaces.

Cheers!