The argument against Test Driven Development

It seems that every time I listen to a software development podcast (which is often), somebody is talking up the benefits of Test Driven Development (TDD).  TDD does indeed have many advantages.  It focuses our efforts by ensuring that the code we write is written for a reason, i.e. to make a test, which has been derived from the requirements specification, pass.  This is in keeping with the YAGNI (You Ain’t Gonna Need It) principle.

TDD also allows a safety net for refactoring, to ensure that, after a refactoring, the code is no more broken than it was before the refactoring.  TDD is also likely to lead to ‘better’ code – more modular, cleaner, with less unwanted dependencies between classes.  And all of these are worthy goals, and perhaps are reasons in themselves for adopting (at least partially) the TDD process.

Where I take exception, however, is that it is somehow implied (and even stated) that TDD cannot help but ensure that the code we write will be ‘correct’.  Wikipedia states that, “Test-driven development offers more that just simple validation of correctness”, which would indicate that TDD does indeed offer validation of correctness.  But it doesn’t.  Let me illustrate the point.

I need a method that will return the square of a given integer.  That seems simple, so I write a unit test.

[Test]
public void TestThatTwoSquaredIsFour()
{
    Assert.IsTrue(Square(2) == 4);
}

Note that this is psuedo-code, but you get the idea.  Okay, let’s have a crack at writing the method to satisfy this test.  Again, Wikipedia gives us some guidance.

The next step is to write some code that will cause the test to pass… It is important that the code written is only designed to pass the test…

So, with this in mind, here is my code.

public int Square(int n)
{
    return 4;
}

This is the simplest code that I can write that will pass the test.  And the test will pass.  Since this is the case, we obviously have not specified the requirements clearly enough (through tests), so we’ll add another test.

[Test]
public void TestThatThreeSquaredIsNine()
{
    Assert.IsTrue(Square(3) == 9);
}

Our code now fails the second test, so let’s ‘fix’ the code in the simplest way possible.

public int Square(int n)
{
    if (n == 2)
        return 4;
    else
        return 9;
}

Run the tests again, and we have a nice green lawn.  But incorrect code.  We can go on adding more and more specific test cases, modify our method under test to pass those tests by using if statements or switch statements, and still have incorrect code.

How about adding a randomization aspect to the test, such as the following?

[Test]
public void TestThatSquareOfRandomNumberIsCorrect()
{
    int n = Random(1000);
    int result = n * n;

    Assert.IsTrue(Square(n) == result);
}

We now have a test that will (probably) force us to write a method which calculates a result rather than selecting from a limited range of options, but there are two problems: first, the test is overly complicated, and second, we have already written the required method inside the test!

So here’s the takeaway: Test Driven Development is a useful methodology which encourages the developer to develop only what is needed, in a testable and maintainable way.  However, it is important to realise that TDD does NOT mean that the produced code is correct in all cases – it is only correct in those cases for which specific tests exist.  As far as TDD is concerned, the code can fail in every other imaginable case.  So keep using your brain as you develop.  Allow TDD to do those things at which it excels, but don’t expect it to have the Midas Touch – or you may very well fail.

About these ads

4 responses to “The argument against Test Driven Development

  1. I like to think back to some of the pure mathematics subjects I did in uni (which I actually quite enjoyed – much more so than the applied mathematics!), the concept of Mathematical Induction comes to mind when testing code.

    From Wikipedia: “Mathematical induction is a method of mathematical proof typically used to establish that a given statement is true of all natural numbers. It is done by proving that the first statement in the infinite sequence of statements is true, and then proving that if any one statement in the infinite sequence of statements is true, then so is the next one.”

    … then applying this to testing code … if a function received a set of inputs and then has an expected output, you could (theoretically) derive a formula representing the (mathematical) function of the (programming) function – and then test that the function gives the correct result for all possible cases of input values.

    Testing end-cases, or extremities is also critical, particularly when dealing with very large numbers. I recently had a case where I wrote a script to extract certain posts (status updates) from Twitter and store them in a database – what I didn’t take into account was the extreme case of the twitter statusid (a unique integer assigned to every status update) incrementing to the point where it exceeded the bounds of a 32-bit unsigned integer. Oops. Because I was using a loosely typed language (PHP), my code didn’t break, but my database table was set to expect a 32-bit unsigned integer, and rather than throwing up an error when it received something larger, it instead reset the integer to the largest possible value … so the error I received was actually a “duplicate key” error, rather than what I might have expected – something like “number too large” (which in retrospect was a bit silly to expect, given how computers tend to represent large numbers).

    Another point I think is worth making is that use-cases and user stories are all well and good, but in many cases, they only represent what the “user” (often a business user with little technical ability) wants or needs, with little regard to the future requirements. This can lead to code being developed which is non-portable and not very extensible.

    Taking your example – if the use-case is that the integers will only ever be n=1 or n=2 (because this is what the company currently does), then your simple switch/if-then-else methodology would easily be the most elegant solution which meets all specified use-cases. However, if the company ever changes policy or through mergers and acquisitions faces a previously inconceivable value for n, then the code needs to be revisited to make it work.

    If the code had been written to cater for more general cases, had been built with extensibility and flexibility in mind, then many of these issues would go away.

    Of course, I guess this is the point of TDD to a degree – why waste time building code to cater for something which might not happen? Re-usability and elegant design is nice in theory, but you waste a lot of effort and introduce a lot of complexity coding for something that isn’t actually needed, and may never be.

  2. Finally, someone who isn’t a blind evangelist. Excellent point of view, and might I add: excellent first comment @SIM.

  3. The only thing I want to point out about the article is that the general process for TDD is think-test-code-refactor. In the example you gave, after the second iteration, with the if/else, you already have code duplication that, as part of the TDD process should be refactored into a more general form.

    The code in this case happens to be constants, but replacing constants by variables is a form of refactoring, and in this case is the appropriate form of refactoring after the second test passes.

    Also, as Kent Beck writes in Test Driven Development, you don’t have to write microtests if you are capable of writing more. Microtests are just a benchmark that you are proceeding in a way that is sound because if you don’t understand or are having trouble with macrotests, you can indeed break it down into microtests. On the other hand if you only have macrotests, it takes much more time to pinpoint the issues.

  4. I was quite excited to find a post in which someone actually dared to write an argument against TDD. But I was disappointed that the argument against was not stronger.
    Here are two more arguments against it.
    Writing tests for every piece of code encourages programmers to test and not think.
    Its too easy to code tests, and there are now many tools that make it even easier. So, hey, why think? Lets just write a test. I hope everyone agrees, that it is always a bad idea not to think.
    Writing no tests is bad because it makes verification and refactoring slow. Writing a test for every function is bad because it makes developing new features slow.
    Like with many principles in life, I think testing should be used in moderation.
    And how much is the right amount?
    This leads me back to my first comment – think about how much is required – and decide for yourself!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s