I’ve worked on big projects, small projects, in huge teams and by myself, in fossilized federal agencies and cool Silicon Valley companies. I have learned and used at least twenty programming languages. I’ve lived through waterfall/BDUF (big design up front), structured programming, top-down, bottom-up, modular design, components, agile, Scrum, extreme, TDD, OOP, rapid prototyping, RAD, and probably others I’ve forgotten about. I’m not convinced any of these things work.
[Edit: Let me explain what I mean by writing methodologies "don't work." I mean they don't deliver a predictable or repeatable software development process in and of themselves. I don't mean that using a methodology dooms the project. Most software development methodologies try to make programming a more engineering-like process, and in that regard they fall short.]
How can you tell if a methodology works?
Whether a methodology works or not depends on the criteria: team productivity, happiness, retention, conformity, predictability, accountability, communication, lines per day, man-months, code quality, artifacts produced, etc. Every methodology works if you measure the right thing. But in terms of the only measurement that really matters—satisfying requirement on time and within budget—I haven’t seen any methodology deliver consistent results.
My own experiences are anecdotal, but they are shared by almost every programmer I know. It turns out that anecdotes are all that anyone has: rigorous studies of software development methodologies haven’t been done because it’s impossible to control for all of the variables.
Try this thought experiment: Imagine two teams of programmers, working with identical requirements, schedules, and budgets, in the same environment, with the same language and development tools. One team uses waterfall/BDUF, the other uses agile techniques. It’s obvious this isn’t a good experiment: The individual skills and personalities of the team members, and how they communicate with each other, will have a much bigger effect than the methodology.
In his 2003 thesis People and methodologies in software development Alistair Cockburn concludes “People’s characteristics, which vary from person to person and even from moment to moment, form a first-order driver of the team’s behavior and results. Such issues as how well they get along with each other and the fit (or misfit) of their personal characteristics with their job roles create significant, project-specific constraints on the methodology. This result indicates that people’s personal characteristics place a limit on the effect of methodologies in general.”
Our own worst enemy
When I started programming in the 1970s software development was tightly controlled by management through a hierarchy of project managers, business analysts, and senior programmers. We did not get to pick the language or tools. I worked on some big, complex projects in companies that worked this way. Some succeeded, some didn’t. Now it’s common for the programmers to choose the methodology and style of working, along with their languages and tools. Analysts are not part of most programmer’s experience anymore, and project managers have been reduced to “team leads” and “Scrum masters,” neutered of any managerial authority and not really in control of anything other than rituals dictated by team consensus.
Strict management, in love with Gantt charts and schedules and documentation, was reduced to “stakeholders” and “users,” and then abstracted away into “user stories.” It’s common now for me to get involved in a project that seems to have no adult supervision. Surprisingly, left to themselves programmers don’t revert to cowboy coding—they adopt or create methodologies stricter and more filled with ritual than anything I experienced in 1980. In fact programmers today can be much more rigid and religious about their methodologies than they imagine a 1970s-era COBOL shop was. I now routinely get involved with projects developed by one or two people burdened with so much process and “best practices” that almost nothing of real value is produced.
Once a programming team has adopted a methodology it’s almost inevitable that a few members of the team, or maybe just one bully, will demand strict adherence and turn it into a religion. The resulting passive-aggression kills productivity faster than any methodology or technology decision.
Does anything work?
My own experience, validated by Cockburn’s thesis and Frederick Brooks in No Silver Bullet, is that software development projects succeed when the key people on the team share a common vision, what Brooks calls “conceptual integrity.” This doesn’t arise from any particular methodology, and can happen in the absence of anything resembling a process. I know the feeling working on a team where everyone clicks and things just get done. What I don’t understand is why I had that feeling a lot more in the bad old days of BDUF and business analysts than I do now.
I think programmers should pay much more attention to listening to and working with their peers than to rituals and tools, and that we should be skeptical of too much process or methodologies that promise to magically make everyone more productive. Maybe social skills come harder to programmers than to other people (I’m not convinced that’s true), but developing those skills will certainly pay off a lot more than trying yet another development methodology.
[Giles Bowkett published an excellent take-down of Scrum that ties in with this article. He covers the shortcomings of Scrum and how "agile" practiced in real life in more (and funnier) detail than I got into here. Why Scrum Should Basically Just Die In A Fire]