The Stone Age

Here are some recent reflections on so-called “agile” practices, triggered by two things: First, a discussion with a colleague about the book, The Pragmatic Programmer by and . Second, the debacle around test-driven, agile, extreme development “guru” failure to produce a working Sudoku solver using Test Driven Development, and the ensuing comparisons to solution (for background, blog post is a good write-up, and interesting in its own right).

For many years now, I have felt there was something missing from the whole “agile” movement, but I could not quite put my finger on what, exactly, that missing something is. I found the books, studies, classes and workshops shallow, and that they left me a hollow feeling. But I could not quite articulate why until now: I think I have finally figured it out.

When I was a young programmer, I read a lot of books that had a profound impact on the way that I thought about computers and programming A short list of relevant titles includes:

Combined with a few, more “industry”-oriented books and papers:

These books inspired me; they made me excited about computing and the possibilities waiting in the machine. Rounding out the mix were Design Patterns by et al, and Refactoring by , though these two had far less of an impact: the sole contribution of Design Patterns is providing a common nomenclature for recurring patterns; it is basically a glossary. Refactoring was notable because it introduced a disciplined approach to maintenance. I found them interesting, but not eye-opening in the way the others had been.

With a very few notable exceptions, most of the books I have read since have had much less of an effect on me, including The Pragmatic Programmer and other staples of the agile bookshelf.

Why? All of the books I listed above were written by amazing, accomplished practitioners and academics. Names like Knuth, Kernighan, Sedgewick, Goldberg and Stevens are regarded as giants who have defined or chronicled the field; their work is canon. Further, all of these people have built large systems that are in use daily; they have designed the languages, operating systems and tools that have set the direction of the industry for the last four decades. They designed the algorithms and data structures that make the giant, seemingly intractable problems we work on computationally feasible. They have been published in the most respected journals, and their books set the standard or defined a subfield.

The others, however, have not; in particular, the “agile”/XP/Design Patterns proponents. What body of established work can the or Ron Jeffries' or of the world point to that has truly been definitional on a global scale? What enduring artifacts have they discovered or published? What fields have they defined? I cannot honestly think of any. Oh sure, they have popularized certain useful practices (e.g., unit testing as a matter of course), but aside from incremental improvements to existing processes, they have not created anything. In short, to the extent that they have pushed things forward, it has been by promoting common sense and pragmatic approaches to things that are already well-defined: incremental improvements, not break-throughs. While perhaps helpful, this is not revolutionary or evidence of deep insight.

In explaining my opinion about it, I likened reading The Pragmatic Programmer to eating a bag of chocolate bars: I was full after consuming it, but aware that I had just ingested a lot of stuff without a lot of substance. The advice seemed useful, but mostly superficial. Derivative works since have largely been variations on this theme.

And I now understand why: the authors just were not accomplished enough to have the insights that would equip them to give deep advice. That so many programmers look at these new authors as the drivers of the field is, I think, rather sad. A provocative quote I read on some blog discussing the Sudoku debacle: Can you see Linus Torvalds being a Scrum master? The idea is laughable, though I question whether Torvalds himself is accomplished enough to really be the one to learn from. He wrote a kernel that was a clone of a (then) 22 year old operating system. To my knowledge, he has not made any original discoveries.

The title of this post is taken from a comment (editorial note: this link used to point to a blog site hosted by Object Mentor, a consulting company Martin is associated with, but that site seems to have disappeared, along with most of the comments on the post) that Robert C. Martin made in response to the Sudoku solver discussion. He writes, anyone who continues to think that TDD slows you down is living in the stone age. Unfortunately, he has no data to back this up, and so I cannot understand how he can credibly make that claim. Indeed, a paper from Microsoft Research shows the opposite (see this article about research for more).

Ron Jeffries received a lot of support after he failed to produce a Sudoku solver. Some went so far as to accuse Peter Norvig of “cheating” because he “already knew” about constraint propagation and search, the techniques he employed in his solution; as if the two were somehow competing and it was not fair that Norvig won (note: I have seen little evidence that Norvig was even aware of Jeffries's efforts at the time he produced his solution, and I suspect he was not). But that is part of the point: Norvig knew about these things because he had studied and worked hard learning basic techniques, not randomly applying a process to everything.

And let us be honest, despite the intimidating names, the search and constraint propagation techniques Norvig used were not that complicated. These are things that someone like Ron Jeffries should know and be able to apply. Indeed, they are things that every programmer should know and be able to employ. But by focusing our attention on “agile”, we take time away from learning these techniques.

I will give Ron Jeffries credit for having the courage to fail publicly. That takes guts. But in the same breath, I am greatly concerned that so few people have taken this as an opportunity for lessons learned: agile techniques may be useful, but they have their limits. They should never be regarded as more than one (or several) of many tools in the programmer's kit.