Here is a post in the Eclipse Process Framework newsgroup that describes OpenUP as “a solution looking for a problem”:

“It may sound naive, but to me OpenUP looks like a solution looking for a problem.

There are already many excellent agile and formal methodologies out there. What development teams need is integration of these methodologies into tools they use daily, not another new methodology created by methodology high-priests. Jazz (Team Concert) is a great example of such a tool. It would be even better if IBM just open sources RUP, so we don’t spend any more time coming up with a new open source methodology. We may just have to accept the bitter reality that after two years (of marketing and promotion) there is not enough interest in OpenUp for a genuine community to form around it.”

If I tried, I couldn’t have said it better ! This is exactly what I had been reflecting on yesterday.

In the past I have pointed out the issues with scalability and lack of enactment (both in context of EPF and IBM’s RMC). However, this sentiment goes beyond and actually questions the need for “spinning the wheels” without adequate clarity on a direction to follow. Rather than wasting so much time, IBM could have been generous and open sourced RUP and not some percentage of it. Also it could have ensured that through a more thought out tool implemenation the historically high overhead related to process, could have been lowered and process “high priests” exposed. But for whatever reasons this was not done.

On Jazz, though I welcome this focus on collaborative development environment, first introduced a couple of years ago by Microsoft Visual Studio Team System, I have a feeling that this idea of “Open Commercial” license may end being a repeat of this half-hearted openness demonstrated by EPF.

What is a process architecture ? My last post talked about a project manager tailoring a process or flexibly enacting a tailored process in a project. In this post I would like to zoom out and present how a typical process architecture looks like. Later I would share the litmus test that shows if your process architecture has been created correctly.

In a typical enterprise organization, the top layer in a process architecture consists of process elements. These are elements like roles, work products, activities that go on to constitute a process. These elements could belong to one or more commercial frameworks or could belong to different methodologies or frameworks. These could also include constitutional elements which the organization needs to define and declare in order to meet a internal or external control objective as part of regulatory or quality compliance.

The next layer would consist of process components that are fragments or clusters of related activities. These process components make it convenient to concentrate on specific aspects of a process. Depending on the business or compliancy objectives of the organization you could craft these components to achieve a certain result or describe a desired workflow. One or more components could be linked to create a larger component and eventually an end-to-end process.

Below the process element and process component layer you will have the process layer. This is where you will have the various sub-organizations or departments start crafting their processes in tandem with the software process engineering group. You would not only use elements from more than one packages but in all probability you will be working with other process authors in different time zones or geographical locations. This process layer also ensures that rather than relying on one monolithic process pushed by a vendor or consultant, teams can create processes drawing from different methodologies and process frameworks.

The next layer would make it convenient for the project management office to tailor the process for a category of projects. This domain layer makes it convenient for you to tailor processes for a technology or business domain. Some common examples I have seen in this layer are tailored processes for web applications or those for mobile applications. This domain specific tailoring makes it convenient for project managers to easily search and use processes in real projects. This takes us to the last and final layer, the engagement layer.

The engagement layer consists of processes that are tailored for a specific project. The project manager does not have to go any higher than the domain layer to look for a process that she can specialize for a project that is about to be initiated. Project managers already have their plate full with all the project start-up tasks and they don’t want any overhead associated with searching for and tailoring a process for their project. They want a tactical intuitive way to tailor a process for a project. If they end up spending more time than needed to prepare a process it is likely that this would end up costing more than the benefits accrued. So the litmus test for a successful process architecture is the time taken by a project manager to tailor and operationalize a process in a real project. The lower this transaction cost of tailoring and moving a process to enactment the higher marks you get for a well managed process architecture.

Almost no discussion on managing processes is complete without the use of the word “tailoring”. It is one of those words that although is understood commonly at a generic level starts giving trouble when applied without enough explanation in process development scenarios. I will attempt to describe how I understand “tailoring” in context of process models. I will provide this explanation using a practical scenario of applying a process in real projects.

A process is of little use unless it is expected to add value resulting in successful delivery of real life projects. Every endeavour or project, be it a software project, science experiment or manufacturing an article of clothing, has a certain process it follows. For us in the knowledge intensive domain of software development, successful project delivery is made tougher due to changing or late arising requirements coupled with involvement of skilled knowledge workers. My earlier post talks a bit more on these challenges. Over time, an organization would accumulate a large number of lessons learned as well as patterns and practices that work consistently well across projects. They will also have a collection of anti-processes (think anti-patterns) which they know don’t work. Some organizations would use a tool such as IRIS Process Author to structure their body of process knowledge for easy reuse.

Let us now add a temporal element relating to a point of time when an expected endeavour/project becomes known. At this time, you would study the characteristics of the known execution environment and try to modify your existing body of process knowledge to come up with a process model that best suits this expected project environment. As much as you can, you would also account for the known unknowns in your process model. Based on different situations, this would mean defining things like - a set of activities and their work breakdown structure or workflow, a set of work products that are created and or consumed in the process, the process roles that are staffed by practioners, the pre-post conditions that might be true within the process and so on. You may also defined the narrative guidance that would be read by the practioners in the project. This tailoring of the available process body of knowledge will result in bespoke processes suitable for a specific project execution environment. This process model would best describe the process framework that is expected to result in a successful execution of the project.

Now we will add the second temporal element relating to a point of time when the project is initiated. At this time you would use the tailored process to create instances of process definition elements as project work items. For example this would mean a project manager creating instances of a change control workflow as and when a change work item is added. I would call this instantiation of process definition as flexible process enactment. Keep in mind that the project manager can also add additional work items that are not defined in the process model. However, these changes to the process model could not have been anticipated as the time of process tailoring before the project is initiated. Some would argue that even this flexible process enactment is a “tailoring” of sorts since the process is being instantiated based on the unfolding project (and in some cases new work items are added to the process model). Personally, I have no problems with that as long as there is clarity on the context – tailoring before project initiation and tailoring after project initiation - is understood.

The key point I want to underline is that you would not leave process tailoring to be done after the project has started. Rather you would attempt to tailor your process, using the lessons learned from the past and the known unknowns of the project, before the project is initiated. Once the project starts you would flexibly instantiate your process elements as project work items driven by the circumstances of the unfolding project. Off course you will also add the unknown unknowns to your process. However these by their very definition would not have been known before project startup so could not have been modeled ahead of time.

In my experience process authoring for software development processes is a highly collaborative activity. It is unnatural to expect a solo effort to come up with a process that could be successfully consumed by a wide team of practioners. The process infrastructure also needs to scale well. Reusing existing process components not only reduces significant overhead but also makes the task of coming up with customized processes agile. Any tool that attempts to support process modeling needs to address these two objectives. Based on the evidence I have noticed, the EPF Composer (and RMC from IBM) fails in this regard.

From the information available, it appears to me that there are significant problems being faced by the Open UP team as they try to come up with a process model. The fundamental issue stems from the fact the EPF Composer is designed as a single-user client tool over-relying on SCM to check-in/check-out complicated process plugins. In my opinion it is unrealistic to manage processes in this manner. You basically end up choosing one of two undesirable options. The first option is to break up a process into multiple constituent plugins that are managed separately in an SCM. The second option is to put everything under one library that is checked in. Here is an on-going EPF-dev discussion thread on this topic. This is a thread questioning the rationale for having a overblown library with every sort of plugin in it.

Most enterprise teams will probably face similar problems when using Rational Method Composer (RMC). If the OpenUP team is having difficulties dealing with the limitations of the tool when trying to add a single (DSDM) plugin to the base OpenUP methodology, imagine the magnitude of problem in an enterprise environment when a blended methodology process architecture consists of a number of methods, regulatory and compliancy standards. Moreover, in that environment there will be a large number of subject matter experts who would likely want to collaborate as they contribute to various aspects of the processes. With the difficulties I am seeing with the usage of EPF Composer within the EPF team one can imagine the problems being intensified with scale.

Providing a server side support for process management is critical for a scalable process architecture and collaborative development of processes. This was a fundamental business objective for the Osellus engineering team that came up with a robust database-backed enterprise architecture for IRIS Process Author. Since every process asset resides in a database, it is easy for IRIS Process Author users to discover, tailor and export processes as needed. All this is done without resorting to complicated check-in/check-out rules associated with handling processes managed as files.

Finally, some would say that it is ironic that the very team that is promoting the “eclipse way” is actually building products that seemingly takes one down the path of single client development rather than team-based development or worse still providing only limited visibility into source process content.