Home > DSLs, programming, testing > Does unit-testing deserve its own DSL?

Does unit-testing deserve its own DSL?

We’ve done a lot with testing frameworks over the years, but does the testing concern deserve its own standalone DSL?

This intriguing question was asked by Michael Feathers in his Mock Objects: Leaping out of the Language post. My spontaneous answer is: Absolutely!

I’m a big fan of xUnit frameworks, but when I imagine an alternative unit-testing specific language one special property comes to mind, a feature that would really make a difference. I’d call it unconditional mocking. With a DSL based unit-testing framework one could test really complex objects, even legacy code, since mocking internal objects would require no change to the original programming interface.

For example, this (nonsense) code

class A {
  private B _b;

  // constructor
  this A() {
    _b = new B()
  }
}

unittest {
  // B can not be mocked
  A a = new A();
}

would require refactoring in order for _b to be mockable.

class A {
  private B _b;

  // constructor
  this A(B b) {
    _b = b;
  }
}

unittest {
  // B could be mocked
  B b = new BMock(...);
  A a = new A(b);
}

But in a unit-testing DSL, one should be able to mock any object, in this case B, without changing the source code first. This is handy for dealing with the unit-testing paradox: Refactoring requires a unit-testing harness to make sure no functionality gets broken, but unit-testing requires testable code; So what to do when the code isn’t testable? A unit-testing DSL would make it easier to put up the initial testing harness.

Also, as Michael points out, a unit-testing DSL could be used to mock any kind of construction, not just objects: Functions and methods for instance. Oh man, could I have use for such a feature?

To give us an image of a DSL for unit-testing in a non-object-oriented language like C, Michael provides this example:

function send_port_command with 90, “CMD: 12”
            calls io_mode which returns M_READ
            calls set_mode with M_WRITE
            calls write_byte with 90
            calls write_bytes with “12”
            returns E_OKAY

That would be testing a function like this:

status send_port_command(byte port, const char *message)
{
  if(io_mode() == M_READ)
    set_mode(M_WRITE);
  write_byte(port)
  write_bytes(translate_command(message));
  return E_OKAY;
}

I have a problem with his example though. In my opinion the test-code resembles the target code a little too much, like a bad manager performing low-level supervision. Too detailed testing beats the purpose since it makes changes more difficult. My philosophy is that test-code should test WHAT the code does, and not bother too much on the HOW.

So, my not so thought through proposal, using Michaels example, would be something like this:

TEST send_port_command

MOCK write_byte(port)
EXPECT port == 90

MOCK write_bytes(bytes)
EXPECT bytes == "12"

CALL send_port_command with 90, "CMD: 12"
EXPECT E_OKAY

Of course there should be support for more advanced mock features like call counting:

MOCK  write_bytes(bytes)
EXPECT   "12", "13"

CALL send_port_command with 90, "CMD: 12"
EXPECT E_OKAY
CALL send_port_command with 90, "CMD: 13"
EXPECT E_OKAY

or

MOCK  write_bytes(bytes)
EXPECT  2 CALLS

or sequential values

MOCK  io_mode
RETURN  M_READ, M_WRITE

Implementing the DSL would be a hefty task though. But, the problems aside, how would your unit-testing DSL be like? I’d be very interested to hear your opinions.

Cheers!

Be Sociable, Share!
Categories: DSLs, programming, testing Tags:
  1. December 10th, 2007 at 22:36 | #1

    I’d personally say it does from my experience in programming!

  2. Martijn Faassen
    December 11th, 2007 at 18:34 | #2

    Unit testing should be in the way as little as possible. Learning or reading a new programming language seems to be “in the way”.

    The best way I’ve seen unit testing disappear into the background is with doctests:

    http://en.wikipedia.org/wiki/Doctest

    http://blog.ianbicking.org/2007/11/27/java-bdd/

    That doesn’t take care of any mocking concerns, of course, but another language just to write tests seems overkill.

  3. December 11th, 2007 at 19:06 | #3

    Good point! Probably that’s the reason we don’t see any unit-testing DSLs out there.
    A thought: Shouldn’t a reflective programming language like Python be able to implement a unit-testing framework with “unconditional mocking”, without the need to create a new language? The framework should be able to generate the mocking code and inject it in the source code during execution. Maybe this has already been done?

  4. Martijn Faassen
    December 11th, 2007 at 19:21 | #4

    I’m not really much into mock objects, but indeed there are quite a few mock object libraries in Python that can do things like that.

    http://labix.org/mocker

    is a recent one that comes from a very good Python programmer. I should try it out with doctests one day. The examples in the mocker documentation are probably doctests already. :)

  5. schlenk
    December 11th, 2007 at 20:07 | #5

    In very dynamic languages such DSLs can be done with not too much overhead.

    tcltest for example is a little test DSL, but for the stuff you wanted one could use things like variable and execution traces and maybe coupled with the unknown handler. Just delete all commands from a slave interpreter other than [unknown] and inspect what gets called.

    For some intro to Tcltest http://www.tclscripting.com/articles/apr06/article1.html

  6. September 2nd, 2009 at 16:23 | #6

    If you take a look into mocha, you will see find what you were looking for in python. Mocha can mock anything as ruby is absolutely a dynamic language. So, this does not really require any DSL.

    However, I gave a feeling that C#/other compiled languages still need a better way to fit in the easiest possible unit test writing.

  7. September 2nd, 2009 at 21:50 | #7

    @Sohan

    Thanks for the tip, I’ll look into Mocha. Ruby is it? My favorite dynamic language.

  1. No trackbacks yet.