Software development processes capture—in reusable form–the organization’s best practices and lesson learned, making them sharable across projects. Today, the benefits of process-centric software delivery are well understood. So, why is the industry’s adoption of software development processes so dismal? The lower than expected adoption can be partly explained by a phenomenon called groupthink. For this post I will rely on materials from James Surowiecki widely cited book: The Wisdom of Crowds.

The coinage of the term, groupthink, predates Surowiecki’s work, but he frames it concisely within the larger arena of group decision making, which he shows is more accurate, in most cases, rather than decisions made by subject matter experts. This easy to read book draws from and consolidates various scientific and empirical bodies of work from diverse fields–such as psychology, statistics, and economics–making the subject generally accessible.

For crowds to produce correct decisions, its members must be diverse, independent, and decentralized, and should have a mechanism to consolidate the individual judgments into collective decision. However, the decision making fails when the members of the crowd are too conscious of the opinions of others and begin to emulate each other and conform than think differently. This failure is called groupthink.

I believe commercial software methodologies have been suffering from groupthink. For over a decade, most efforts have centered around Unified Process with all participants—mainly methodology theorist and consultants-emulating each other and conforming rather than thinking differently. Any new development—such as Eclipse Process Framework or SCRUM—has been forced to fit in a UP mold. The practitioners have found these expertly devised methodologies irrelevant, and, hence, have mostly avoided them. At the same time, new practical ideas that arise during actual development projects are prevented from blossoming. The potential methods devised by diverse projects’ practitioners are likely to be more relevant, as they convey the wisdom of crowds (mobs) and consequently have a better chance of wide adoption.

The good news is that with the recent availability of integrated ALM and interactive process asset repository systems, it is now possible to involve practitioners in the end-to-end methodology development effort. I will cover this in more details shortly.

This entry was posted on Tuesday, June 30th, 2009 at 10:57 pm and is filed under ALM, General, Process Asset Repository, Process Modeling, Processes & Methodologies. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.