Home > DSLs, programming, testing > Does unit-testing deserve its own DSL?

Does unit-testing deserve its own DSL?

We’ve done a lot with testing frameworks over the years, but does the testing concern deserve its own standalone DSL?

This intriguing question was asked by Michael Feathers in his Mock Objects: Leaping out of the Language post. My spontaneous answer is: Absolutely!

I’m a big fan of xUnit frameworks, but when I imagine an alternative unit-testing specific language one special property comes to mind, a feature that would really make a difference. I’d call it unconditional mocking. With a DSL based unit-testing framework one could test really complex objects, even legacy code, since mocking internal objects would require no change to the original programming interface.

For example, this (nonsense) code

class A {
  private B _b;

  // constructor
  this A() {
    _b = new B()
  }
}

unittest {
  // B can not be mocked
  A a = new A();
}

would require refactoring in order for _b to be mockable.

class A {
  private B _b;

  // constructor
  this A(B b) {
    _b = b;
  }
}

unittest {
  // B could be mocked
  B b = new BMock(...);
  A a = new A(b);
}

But in a unit-testing DSL, one should be able to mock any object, in this case B, without changing the source code first. This is handy for dealing with the unit-testing paradox: Refactoring requires a unit-testing harness to make sure no functionality gets broken, but unit-testing requires testable code; So what to do when the code isn’t testable? A unit-testing DSL would make it easier to put up the initial testing harness.

Also, as Michael points out, a unit-testing DSL could be used to mock any kind of construction, not just objects: Functions and methods for instance. Oh man, could I have use for such a feature?

To give us an image of a DSL for unit-testing in a non-object-oriented language like C, Michael provides this example:

function send_port_command with 90, “CMD: 12”
            calls io_mode which returns M_READ
            calls set_mode with M_WRITE
            calls write_byte with 90
            calls write_bytes with “12”
            returns E_OKAY

That would be testing a function like this:

status send_port_command(byte port, const char *message)
{
  if(io_mode() == M_READ)
    set_mode(M_WRITE);
  write_byte(port)
  write_bytes(translate_command(message));
  return E_OKAY;
}

I have a problem with his example though. In my opinion the test-code resembles the target code a little too much, like a bad manager performing low-level supervision. Too detailed testing beats the purpose since it makes changes more difficult. My philosophy is that test-code should test WHAT the code does, and not bother too much on the HOW.

So, my not so thought through proposal, using Michaels example, would be something like this:

TEST send_port_command

MOCK write_byte(port)
EXPECT port == 90

MOCK write_bytes(bytes)
EXPECT bytes == "12"

CALL send_port_command with 90, "CMD: 12"
EXPECT E_OKAY

Of course there should be support for more advanced mock features like call counting:

MOCK  write_bytes(bytes)
EXPECT   "12", "13"

CALL send_port_command with 90, "CMD: 12"
EXPECT E_OKAY
CALL send_port_command with 90, "CMD: 13"
EXPECT E_OKAY

or

MOCK  write_bytes(bytes)
EXPECT  2 CALLS

or sequential values

MOCK  io_mode
RETURN  M_READ, M_WRITE

Implementing the DSL would be a hefty task though. But, the problems aside, how would your unit-testing DSL be like? I’d be very interested to hear your opinions.

Cheers!

Categories: DSLs, programming, testing Tags:
  1. December 10th, 2007 at 22:36 | #1

    I’d personally say it does from my experience in programming!

  2. Martijn Faassen
    December 11th, 2007 at 18:34 | #2

    Unit testing should be in the way as little as possible. Learning or reading a new programming language seems to be “in the way”.

    The best way I’ve seen unit testing disappear into the background is with doctests:

    http://en.wikipedia.org/wiki/Doctest

    http://blog.ianbicking.org/2007/11/27/java-bdd/

    That doesn’t take care of any mocking concerns, of course, but another language just to write tests seems overkill.

    • December 11th, 2007 at 19:06 | #3

      Good point! Probably that’s the reason we don’t see any unit-testing DSLs out there.
      A thought: Shouldn’t a reflective programming language like Python be able to implement a unit-testing framework with “unconditional mocking”, without the need to create a new language? The framework should be able to generate the mocking code and inject it in the source code during execution. Maybe this has already been done?

      • Martijn Faassen
        December 11th, 2007 at 19:21 | #4

        I’m not really much into mock objects, but indeed there are quite a few mock object libraries in Python that can do things like that.

        http://labix.org/mocker

        is a recent one that comes from a very good Python programmer. I should try it out with doctests one day. The examples in the mocker documentation are probably doctests already. 🙂

  3. schlenk
    December 11th, 2007 at 20:07 | #5

    In very dynamic languages such DSLs can be done with not too much overhead.

    tcltest for example is a little test DSL, but for the stuff you wanted one could use things like variable and execution traces and maybe coupled with the unknown handler. Just delete all commands from a slave interpreter other than [unknown] and inspect what gets called.

    For some intro to Tcltest http://www.tclscripting.com/articles/apr06/article1.html

  4. September 2nd, 2009 at 16:23 | #6

    If you take a look into mocha, you will see find what you were looking for in python. Mocha can mock anything as ruby is absolutely a dynamic language. So, this does not really require any DSL.

    However, I gave a feeling that C#/other compiled languages still need a better way to fit in the easiest possible unit test writing.

  5. September 2nd, 2009 at 21:50 | #7

    @Sohan

    Thanks for the tip, I’ll look into Mocha. Ruby is it? My favorite dynamic language.

  1. No trackbacks yet.