A few days ago, I had a very interesting conversation with a process improvement expert. He is part of a large multi-national company and for the last 18 months, he has been responsible for the deployment of process improvement best practices throughout his company’s IT department. This work has given him valuable insight on the subject of process improvement deployment, and I very much appreciate him sharing his experience and lessons learned with me. In this post, I will present the process improvement deployment experiences of this customer as a case study, with their lessons learned summarized at the end. This case study illustrates how theoretical concepts of process improvement can be put into practice.

For organizations embarking on the path of achieving a process improvement mindset, the first logical step is to capture their existing processes in a safe, usable, and accessible form. Safe meaning that the processes have the appropriate level of protection from misplacement and loss. Usable meaning that the processes are in a format (or formats) that can be easily used by all members of the organization. Accessible meaning that the processes are accessible organization-wide from a single well-known portal. Once the organization’s internal processes are captured in this manner, a gap analysis can be conducted and the indentified gaps filled by the industry’s best practices. This is a laborious step and requires careful planning and project management, but process authoring and management tools can substantially reduce this effort.

For this customer, the first step involved capturing a large number of processes, many of which were never previously documented. Their strategy relied heavily on tasking various subject matter experts (SMEs), from different IT groups, with process definition. The SMEs were provided with an intuitive tool and training. The SMEs, working from their own various geographic locations, were then able to enter their group’s processes in the central system. The process group constantly reviewed the entered processes and provided feedback to the SMEs. Once the initial processes were captured, the process group structured them into a reusable process architecture. SMEs, in turn, were able to review the process architecture and provide feedback. Employing this strategy, the customer was able to distribute the process definition effort across its IT department, and created a comprehensive process architecture in a matter of months. Moreover, this strategy made the process improvement adoption (institutionalization) more likely, as the user community was intimately involved from the beginning—versus being dictated to by an external group. It creates a sense of ownership.

At the end of this step, the organization had codified its distributed tacit knowledge.

To further increase process adoption by the organization project teams, the process group, with the help of SMEs, created variations of each process to better suit the various project categories. All the processes (and their variations) were made available for consumption from an organization-wide process portal. The portal also provided sophisticated “search and select” functionality to make sure the most suitable process could be quickly identified for a given project. Process search and selection will form the basis of a comprehensive process tailoring solution in the future.

The customer decided on this approach as opposed to burdening the project managers with the tailoring task. They felt tailoring processes by adjusting them was too complex a task for the project managers at this stage.

Based on my experience, many organizations underestimate the complexities of process tailoring. I feel this simplified tailoring approach is sufficient for most organizations, with the exception of matured process-centric ones.

To effectively improve the processes and the process architecture, the organization relied heavily on input from the end users. The “feedback” and “community” functionality of the process portal was used to collect and capture feedback and lessons learned from the organization’s project teams which, in turn, were used by the process group and the SMEs for ongoing process improvement.

Keep in mind that a single integrated system provided the process capturing, definition, architecture, tailoring, and feedback, with end-to-end traceability.

As the next step, the customer is developing usage models to accurately measure the rate of process adoption by the organization’s project teams. They feel having reliable adoption metrics is critical to sustaining  the support of senior management.

In summary, the following are the lessons learned from this process improvement effort:

• Identify and utilize SMEs for capturing the organization’s processes
• Importance of training SMEs and providing an easy-to-use tool for process definition
• Importance of a process architecture that supports reusability
• Create a sense of ownership as early as possible
• Simplify the task of tailoring
• Minimize the effort required by project managers to use a process in the their project
• Base the ongoing process improvement on user feedback
• Importance of end-to-end traceability
• Importance of usage models in winning and sustaining senior management support

This case study illustrates how theoretical concepts of process improvement can be put into practice.

I feel topics such as usage models and institutionalization/internalization deserve more detailed exploration.

This is the first post in a series detailing out a process we at Osellus have devised for development of iPhone applications. This process incorporates Apple’s best practices for iPhone development, as well as, the many relevant lessons learned from the general software development body of knowledge.

This work was initiated by our iPhone development and project management teams. As we embarked on various customer projects, the project management office wanted to have better project visibility and the ability to make sure the projects met the customers’ objectives. It  has also helped project managers with staffing of each project with the right skill set. We intend to improve this process based on lessons learned from the completed projects.

In the next post of the series, I will cover the overall business objectives, and the process objectives as they relate to each business objective which thereby form the basis of the process.

Software development processes capture—in reusable form–the organization’s best practices and lesson learned, making them sharable across projects. Today, the benefits of process-centric software delivery are well understood. So, why is the industry’s adoption of software development processes so dismal? The lower than expected adoption can be partly explained by a phenomenon called groupthink. For this post I will rely on materials from James Surowiecki widely cited book: The Wisdom of Crowds.

The coinage of the term, groupthink, predates Surowiecki’s work, but he frames it concisely within the larger arena of group decision making, which he shows is more accurate, in most cases, rather than decisions made by subject matter experts. This easy to read book draws from and consolidates various scientific and empirical bodies of work from diverse fields–such as psychology, statistics, and economics–making the subject generally accessible.

For crowds to produce correct decisions, its members must be diverse, independent, and decentralized, and should have a mechanism to consolidate the individual judgments into collective decision. However, the decision making fails when the members of the crowd are too conscious of the opinions of others and begin to emulate each other and conform than think differently. This failure is called groupthink.

I believe commercial software methodologies have been suffering from groupthink. For over a decade, most efforts have centered around Unified Process with all participants—mainly methodology theorist and consultants-emulating each other and conforming rather than thinking differently. Any new development—such as Eclipse Process Framework or SCRUM—has been forced to fit in a UP mold. The practitioners have found these expertly devised methodologies irrelevant, and, hence, have mostly avoided them. At the same time, new practical ideas that arise during actual development projects are prevented from blossoming. The potential methods devised by diverse projects’ practitioners are likely to be more relevant, as they convey the wisdom of crowds (mobs) and consequently have a better chance of wide adoption.

The good news is that with the recent availability of integrated ALM and interactive process asset repository systems, it is now possible to involve practitioners in the end-to-end methodology development effort. I will cover this in more details shortly.

Earlier this month, IBM Rational announced their Measured Capability Improvement Framework (MCIF) offering, vaguely described in the IBM’s oficial news release:

Additionally, with IBM’s new Measured Capability Improvement Framework (MCIF), organizations can also take actions to continuously improve on results by learning from past experiences. Through MCIF, IBM provides organizations with an end-to-end framework that enables them to measure results and manage projects so they can incrementally improve their software delivery capability. 

“In today’s economic climate, businesses are looking for new ways to derive greater value from their investments in software,” said Dr. Daniel Sabbah, general manager, IBM Rational Software. “Up until this point, organizations have been lax in measuring the business value and discipline of the processes [emphasis added] they use to deliver software assets. Classic metrics in software engineering largely ignore the importance of actual business outcomes. Our clients are now beginning to realize that the software they build or assemble must be treated as a strategic business asset. IBM is committed to helping them make the right decisions and improve the successful outcomes of this newly emerging business process discipline.” 

 

Measurement driven improvement is central to CMMI and CobiT has a strong emphasis on alignment of IT processes with organization’s business objectives. If you have read my blogs, you would know that I strongly believe in objective-based process definition. If anything, IBM Rational is playing catch up. But it is still nice to have their confirmation.

I didn’t attend this years Rational Software Conference, but I have been carefully studying white papers on MCIP and Rational Insight offerings. I will discuss Rational Insight in a future post.

MCIP white paper is well written and is an enjoyable read. I fully agree with the framing of the differences between business and manufacturing processes and software development processes on pages 3 and 4.

Unlike most other business processes, such as supply chain management or manufacturing, SSD needs to deal with a range of risk. SSD also differs from many other business processes in that it entails a diseconomy of scale: that is, individual productivity decreases with the size of the SSD effort. …

Software delivery differs from many other business processes by dealing with a broad range of innovation. Some software projects, such as maintenance of existing systems, are reasonably predictable, similar to manufacturing processes. Those projects carry limited innovation and drive limited or no business differentiation. Other projects, such as building unprecedented and large software systems, require high degrees of innovation in addressing problems that have never been solved before on a schedule. Committing to delivering innovation requires assuming risk, since the lack of complete knowledge at project inception is inevitable and uncertainty regarding how to proceed is part of the challenge. This risk is manifested in the statistical variance in the estimate of the time or cost to complete. 

A commitment to assuming risk entailed by bringing innovation to the enterprise provides the opportunity to improve ROI.  

Another major difference between the business process of software delivery and other business processes is the diseconomy of scale. Typically, manufacturing and service delivery processes offer economy of scale: The cost of a unit of software grows nonlinearly (i.e., yields cost reduction) with the size and complexity of the system. But this is not the norm in software production.

On the other hand, some of the insights that have been discovered as part of IBM effort are trivial. For example, on page 15 they say 

Many organizations mistakenly try to make one process fit all circumstances. In our experience, the above type of analysis is required to enable you to drive the appropriate change to the right project types. 

I don’t know of any organization that doesn’t believe this. In fact, it sounds condescending. 

In essence, MCIF is a practice-based approach to software development processes. An approach they first introduced in the last version of EPF (before it became inactive). One can argue that IBM was a later comer to this also., The concept of practice has been widely utilized in CMMI, Microsoft MSF and EssentialUP.  MCIF is a methodology for top-down selection of practices based on the organization’s business objectives.

Although I like objective-based software development process definition, MCIF, however, is top-down and non-collaborative. It relies on Rational Method Composer (RMC) tool, which is a single-user desktop application–requiring a configuration management system for basic maintenance of processes. The white paper, also, falls short in addressing the practical issues of mapping business objectives to different aspects of processes and the mechanics of process tailoring. 

Finally, from Per’s video, it is apparent that MCIF is not a tool empowering users, rather it is a service that requires engagement of IBM consulting services.

My recommendation: best source for software development capabilities improvement is CMMI body of work. As I said before, CMMI is the result of two decades worth of work by various subject matter experts, not a single vendor’s commercial methodology.

A colleague forwarded to me an interesting work by Petter Holmström titled “Ideas for Next Generation Process Authoring Tools”.  It’s a long comprehensive document, and I have just started reading it end-to-end. From a quick scan of the table of contents, abstract and conclusions, I mostly agree with his conclusions and recommendations:  

The tool vendors should shift focus and concentrate on making their tools more collaborative, customizable and scalable to different process sizes. In this thesis, some ideas of how this could be achieved have been presented, of which one of the more interesting ones is a wiki-based authoring tool.

 

As you may have realized from my previous blog postings, I am a strong proponent of collaborative process management tools and the importance of the involvement of developers and other process consumers in the creation of processes–they consume. The industry players and the user community should democratize process authoring and move on from blindly following methodology pundits.