Does unit-testing deserve its own DSL?
We’ve done a lot with testing frameworks over the years, but does the testing concern deserve its own standalone DSL?
This intriguing question was asked by Michael Feathers in his Mock Objects: Leaping out of the Language post. My spontaneous answer is: Absolutely!
I’m a big fan of xUnit frameworks, but when I imagine an alternative unit-testing specific language one special property comes to mind, a feature that would really make a difference. I’d call it unconditional mocking. With a DSL based unit-testing framework one could test really complex objects, even legacy code, since mocking internal objects would require no change to the original programming interface.
For example, this (nonsense) code
class A {
private B _b;
// constructor
this A() {
_b = new B()
}
}
unittest {
// B can not be mocked
A a = new A();
}
would require refactoring in order for _b to be mockable.
class A {
private B _b;
// constructor
this A(B b) {
_b = b;
}
}
unittest {
// B could be mocked
B b = new BMock(...);
A a = new A(b);
}
But in a unit-testing DSL, one should be able to mock any object, in this case B, without changing the source code first. This is handy for dealing with the unit-testing paradox: Refactoring requires a unit-testing harness to make sure no functionality gets broken, but unit-testing requires testable code; So what to do when the code isn’t testable? A unit-testing DSL would make it easier to put up the initial testing harness.
Also, as Michael points out, a unit-testing DSL could be used to mock any kind of construction, not just objects: Functions and methods for instance. Oh man, could I have use for such a feature?
To give us an image of a DSL for unit-testing in a non-object-oriented language like C, Michael provides this example:
function send_port_command with 90, “CMD: 12”
calls io_mode which returns M_READ
calls set_mode with M_WRITE
calls write_byte with 90
calls write_bytes with “12”
returns E_OKAY
That would be testing a function like this:
status send_port_command(byte port, const char *message)
{
if(io_mode() == M_READ)
set_mode(M_WRITE);
write_byte(port)
write_bytes(translate_command(message));
return E_OKAY;
}
I have a problem with his example though. In my opinion the test-code resembles the target code a little too much, like a bad manager performing low-level supervision. Too detailed testing beats the purpose since it makes changes more difficult. My philosophy is that test-code should test WHAT the code does, and not bother too much on the HOW.
So, my not so thought through proposal, using Michaels example, would be something like this:
TEST send_port_command
MOCK write_byte(port)
EXPECT port == 90
MOCK write_bytes(bytes)
EXPECT bytes == "12"
CALL send_port_command with 90, "CMD: 12"
EXPECT E_OKAY
Of course there should be support for more advanced mock features like call counting:
MOCK write_bytes(bytes)
EXPECT "12", "13"
CALL send_port_command with 90, "CMD: 12"
EXPECT E_OKAY
CALL send_port_command with 90, "CMD: 13"
EXPECT E_OKAY
or
MOCK write_bytes(bytes)
EXPECT 2 CALLS
or sequential values
MOCK io_mode
RETURN M_READ, M_WRITE
Implementing the DSL would be a hefty task though. But, the problems aside, how would your unit-testing DSL be like? I’d be very interested to hear your opinions.
Cheers!