One of the challenges in adopting any methodology in an enterprise is in ascertaining the degree to which it is actually used. Although this sounds simplistic enough it brings a key issue out in the open. Even if an organization has converged on a single methodology to use there are subjective and often acrimonious debates on the extent to which a defined process adheres to the key concepts – either fundamental concepts that outline the methodology or the control objectives that have been bolted on to this methodology. It is not enough to have a methodology preamble announcing that this methodology meets so and so concept or follows this philosophy. A process mature organization needs to go behind the rhetoric (and I am not trying to slam this sort of collateral that sells the methodology) and actually give specific mapping between the process elements and the concepts and or control objectives that are promised to be met with this process.

This data-mapping will not only help in auditing your processes for compliance to your business or regulatory objectives but will also serve as an indisputable baseline to have those debates on what kind of process does the project team really need for an upcoming project. If you know where you are now, it is easier to steer where you want to go. Categorization of your process elements is therefore a simple but powerful way to keep measuring if your process is on track to become what you expected it to be or of you need to make modifications so that you can control its flavor at process authoring stage. Off course another benefit in this approach is that when you are ready to share your process with a peer network they can give their feedback so that you can adjust the mapping of your process elements to the methodology concepts and control objectives and validate this mapping.

There are many examples of this sort of categorization that can be seen in the processes available at the IRIS Process Central Sandbox. You are welcome to try out these processes and provide your comments.

I have been keenly watching the progress on what started as a promising project in the Eclipse foundation. Unfortunately after more than two years of opportunity to shine I would have to give the EPF project a failing grade. I fear that when someone at IBM realizes the wasteful investment of resources in this project they will pull the plug on their participation at EPF. This will inevitably kill the project since IBM is the only committer on the tool aspect of EPF and the major committer and donor to the content aspect of EPF.

In my view there are several reasons for this failure and many lessons that I have learnt simply by following the travails of the project. These reasons are not the lack of aptitude and personal contribution of the EPF team but simply a miscalculated approach to building a vibrant self-organizing process community. Today I will highlight the mind boggling high transaction costs of submitting process content to be consumed and critiqued by a peer network. Even by their own admission, it takes “2-3 months” for the eclipse legal process to approve any process contributions to EPF. This is an anachronistic concept in today’s day and age. Communities are self-organizing. If what I contribute is useless to you, you will not use it. Period. Moreover having been tagged as someone who supplies useless content my user ratings will go down and I will have to significantly improve my game to get back my reputation. Communities give rise to wisdom-of-the-crowd, encourage meritocracy and do not tolerate back room machinations. As articulated by Chris Anderson in The Long Tail while discussing the tools needed to support a long tail, “This is the end of spoon-fed orthodoxy and infallible institutions, and the rise of messy mosaics of information that require - and reward – investigation”. Elsewhere in the book he discusses how “a once-monolithic industry structure where professionals produced and amateurs consumed is now a two-way marketplace, where anyone can be in any camp at any time. This is just a hint of the sort of profound change that the democratized tools of production and distribution can foster”. In the case of software process modeling, these tools take the form of easy to use process modeling and consumption tools. With that, rather than a paradigm of a handful of processes being donated by a few process experts, one could feasibly have hundreds of process variants created, put on wikis, blogged and rated! Now if we multiply the 2-3 months timeframe for these 100 processes the EPF legal staff will be kept busy for 20 years…just on this project! Later I will discuss some other pitfalls to avoid based on the lessons learned from following this project.

Yesterday Omid and I gave a joint Microsoft-Osellus webinar on Effective Software Development Governance with Adam Galant. Thanks to Rini Gahir and Adam Galant from Microsoft Canada for arranging this webinar.

It was a challenge to fit an explanation of the theoretical concepts of SD Governance with a tangible step-by-step approach to achieving it within the 45 minutes we had. In the end we were able to keep to our time and I hope the audience (about 36 attendees) left with an outline of a practical approach to establishing software development governance in an automated yet flexible manner.

Here is a link to a recording of the governance webinar.

You can use this link to register for the joint Microsoft-Osellus Webinar on software development governance. Omid is going to be talking about a lot of things that we have been able to achieve with the collaboration between the osellus process toolset/service offerings and the VSTS integrated ALM.

Here is a preview of what we will be talking about:

Effective governance aligns your software development investment with your business strategy. By ensuring continuous traceability of software development activities, governance leads to enhanced customer satisfaction and compliance with internal and external policies.

While most organizations use some measure of implicit governance, the full potential of a governance framework can be realized by taking three practical and explicit measures. The software development processes in the organization need to be selected and tailored in a manner that supports organization’s business goals. These processes need to be enforced in a streamlined manner. And, they need to be monitored to ensure corrective actions are taken in a timely manner.

In this webinar we outline a practical approach to establish software development governance in an automated yet flexible manner. We will discuss how the business objectives, standards and control mechanisms in the organization should be mapped to process elements such a roles, activities and work products that guide practitioners in a non-intrusive manner. Microsoft Visual Studio Team System (VSTS) and the IRIS process automation suite from Osellus will be used to demonstrate implementation of an effective governance framework in real-world scenarios.

In September I had talked about the ongoing work the Osellus engineering team was doing to address the enterprise need of establishing a sustainable process community.

I had mentioned how:
“This is essential to get feedback and increase process ownership within the project team that would have to ultimately enact it. A process portal would ensure that the processes tailored in the earlier phase are versioned, categorized and available for review. It is important that there is no overhead associated to consume these processes. You should make it convenient for a team of practioners (who may not be process experts) to locate, review and comment on the process they are interested in. The process portal would also be the single location from where users can download VSTS process templates, MS project templates, published online process and printable (document format) process. To encourage a transparent vibrant process community, tools like blogs, ratings, newsgroups should be used extensively.”

Well, Negin has announced the availability of the IRIS Process Central sandbox that introduces this solution.

I am pleased to announce the availability of IRIS Process Central Sandbox. This is an online process community to promote discussions on process improvements through collaborative input. Its goal is to bring together process modelers, process consumers, and other stakeholders to freely share information on existing processes and create new process mash-ups.

To facilitate these discussions, we have included several processes on sandbox including MSF for CMMI, OpenUP, and Agile. Where possible, we have provided variety of formats for each process as well as enactment templates.

You can participate by providing your feedback through blogs, discussions boards or modifying the content of the processes. We are eager to get your feedback and will make every effort to ensure that IRIS Process Central Sandbox becomes a valuable asset to you and your organization. Click IRIS Process Central Sandbox to access the sandbox.

IRIS Process Central Sandbox demonstrates how IRIS Process Central which is included as part of IRIS Process Author can help an organization improve their process tailoring in a collaborative environment.

It is our strong belief that through awareness and involvement of process community, there is a great chance of increasing adaptation of processes across the organization. We feel that IRIS Process Central is an important step towards this goal.

We are looking forward to the community participation in IRIS Process Central Sandbox or any suggestions regarding IRIS Process Central product.

David had this to say (quite disdainfully ) about my theorizing about getting data out of Jazz into MS Excel and MS Project:

Instead of a one-way data transfer from Jazz to Microsoft documents, I think a much more useful exercise would be to extend MS Office to make it Jazz aware. This would allow the Office applications to behave as Jazz clients much like Microsoft and others have done for VSTS. Without Jazz awareness, I feel that these other applications would be limited to reporting functions and be of little use to the project management community.

I see this more as an exercise in extending MS Office (or your application of choice) rather than extending Jazz and can understand why IBM would be looking for partners to provide such a solution for their customers.

Point is good. Point taken. I am doing precisely what Negin was complaining about - missing the project management perspective and looking at it from the developer’s perspective - but I did fool myself for a bit there that I had the answer signed, sealed and delivered.

The project manager would rather work directly from inside MS Excel, MS Project and interfaces she is familiar with - and it is far fetched to expect non-developer stake-holders to log into a Jazz client/web client and export into Project or report into Excel.

Also, Work Item information (for example) could be changing frequently and direct integration into MS Excel and MS Project provides an opportunity to refresh statuses and other work item information directly from inside a report that the project manager has open compared to having to export repeatedly from inside a Jazz client each time.

Not to fear. As with the Jazz-based solution I outlined yesterday, I have a similarly vague outline of how to do this - atleast for Excel:

As mentioned in my previous post, we already have a way to get the data off the Jazz server.

Microsoft has made it equally easy to develop applications that target the Office ecosystem through VSTO (Visual Studio Tools for Office - currently VSTO 2005). This means that we have application level add-in support for Office system applications like MS Excel.

Through VSTO, I can write a plug-in for Excel that puts a Jazz toolbar/ribbon in Excel and interfaces directly with the service interface of the Jazz Server to get the raw data required off Jazz. This VSTO plugin would then take care of the business logic of putting together and rendering different reports and charts based on pre-defined queries by leveraging the runtime support in MS Excel that VSTO allows.

Reverting to the developer perspective for a bit, VSTO is installed as an add-on on Visual Studio and provides very nice design-time support - just like Win-Forms controls. Makes it that much easier to develop these plug-ins.

Depending on the needs, in case we find that the Jazz service interface is too ‘raw’ for consumption, another option would be to have the Excel VSTO plug-in interact with the Jazz server through a custom Jazz service-plugin that takes care of sending the Excel plug-in well filtered and formatted data. This has some obvious advantages but in my opinion, has the decisive disadvantage of mandatorily requiring a Jazz server side plug-in without which the nice MS Excel plugin we developed will not work.

The part I am not sure about is how to implement a similar VSTO plug-in for MS Project. If I remember correctly, VSTO doesn’t offer support for MS Project development. I could be wrong.

Browsing the VSTO site is becoming a pain right now because I am on a train back to Toronto that has wifi access on board - but connectivity is sporadic at best. I’ll check up a little later and update this post. If it does have support, that’s that.
If not, a plugin can be written directly against the MS Project SDK to achieve our end.

[Update] There seem to be a couple of options for developing MS Project plugins:

• The version of VSTO that comes integrated with Visual Studio 2008 (currently in beta) has support for creating MS Project plug-ins.
• I guess you can create a COM add-in for MS Project using the MS Project SDK I mentioned above but that would be something to consider only if I couldn’t use VSTO.

Last week was pretty busy and I can remember only bits and pieces of it- sort of like Memento without the anterograde amnesia. I digress but it’s one of my favorite movies and I recently watched it again - and was looking for a way to bring it up - even if completely irrelevant :). So there!

One of the things I do remember from last week was attending the Jazz sessions on the 23rd and then the BoF session on 25th at CASCON’07

There were quite a few of us Ossies at Cascon (see Kamal’s post) and I think we all agreed that Jazz and Team Concert were a pretty impressive debut effort by IBM in the ALM space - though crippled by the lack of certain features and perhaps by the potentially dangerous mislabeling of a group of disjointed policies as “process” (a word and space held dear at Osellus :)).

While we didn’t get a chance to sit down and discuss our observations after cascon, we all took away strong observations from our respective perspectives. I can get started on what I really liked and also the large gaps I see in these offerings - but that’s not the reason I am posting this on a weekend when I could be flipping through channels.

I just read Negin’s post regarding the lack of project mgmt. support in Team Concert and I thought I would quickly respond to it before returning to another re-run.

I know exactly where she is coming from and understand this is a pretty important requirement once you get down to actually using Team Concert/Jazz in a real environment.

However, given that Jazz is built on the Eclipse platform, I see this as a job for an extension (or extensions) built on top of Jazz instead of as an inherent deficiency in Jazz. A couple of us had a chance to talk to Kartik Kanakasabesan, the product manager on Jazz at their booth at cascon and he said as much - it’s something that can be built by the community - possibly by somebody who is more qualified to build such extensions.

Hypothetically, with knowledge of the high-level architecture of Jazz and Eclipse, here are what ingredients I would need to build a MS Project or MS Excel Export concoction:

A way to extend Jazz by adding my own services or wrap Jazz services to get data off the Jazz server:
Jazz is built on the plugin-based architecture of Eclipse and has exposed extension-points both on the client side and the service side (Both of which are Equinox OSGi based). These extension points are in addition to the Jazz services themselves that are exposed as part of the service interface (and also have corresponding siblings in the client library)

So, getting the data off the Jazz server to put into my MS Excel report or into MS Project is taken care off.

Getting the data into MS Project:
Negin’s point regarding getting the Work Items in the “process” heirarchy in JAZZ/Team Concert into MS Project or into any other PM tool for that matter is perhaps *the* most important practical fine-grained feature that will get JAZZ out of just the research community into a real project manager’s hands.

MS Project projects can be defined as XML and Microsoft has a well-defined schema for the XML definition.

So, if we mash up JAZZ’s Work Item related service interface API to get the data out and transform the data to a MS project XML, we are good to go!
I can’t get my jazz.net username/pwd to work for some reason so am vaguely referring above to the “Work Item related service API” - I would have preferred to use the right nomenclature.

Getting the data into MS Excel
Getting the data into Excel is similarly not a problem technically.
A server side plugin that the Jazz team calls a ‘common’ plug-in can be built that provides the generic functionality of being able to export data in the Excel format. This plug-in has to be written in Java of course considering this is the Eclipse platform - therefore it can leverage one of the many libraries openly available for reading/writing to the OLE 2 Compound Document Format which includes the MS Excel (XLS) and MS Word (DOC) formats.

The one I rely on usually for similar purposes is Apache’s POI but the choice might depend on the specifics we want to achieve.

I would then build a Jazz service plug-in that wraps the business logic of generating the specific reports we want to output in the Excel format. This service plug-in would get the data using Jazz’s service interface, mash the data up into different types of reports and then output them into spreadsheets using the generic ‘common’ plugin I mentioned above.

A Jazz client plug-in that provides a way to trigger the service plug-in for generation of specific reports or export into MS-Project would be the last step.

I have used Negin’s specific complaints above to demonstrate a hypothetical and technical answer. However, MS Project and MS Excel are but two formats. Formats such as the Open Document Format (ODF) can be just as easily supported. These formats will become increasingly prominent given the adoption of ODF based tooling such as Open Office by governments, educational institutions and SMEs - especially since these are the same organizations that might end up as primary Jazz customers.

Therefore, building such extensions to provide an interface to the project management tooling world is technically not an issue. The real work will be to intelligently define what reports one would want that can leverage such extensions. Negin and others like her who have invaluable years of project management experience are best suited to answer that question!

Notice to Jazz developers: Feel free to tell me how you would do it - or to correct gaps in my understanding.

So Negin….when do you want me to get started on this?

On Tuesday October 23rd, I attended the Jazz session in CASCON 07. John Wiegand demonstrated Team Concert which is built on Jazz platform and he took the audience through some project scenarios, mostly from developers’ point of view. I was impressed by Team Concert and liked the user interface, however, as a project manager I was disappointed about the lack of project management support planned for the initial release of Team Concert.

In the beginning of the session, John had mentioned the pain areas that Jazz was addressing, one of which was project management. So I was looking forward to seeing the extend of this support. The functionalities that were demonstrated mostly focused on IDE as well as team collaboration and communication. Project management support was clearly missing, even simple interface to MS Project or MS Excel. I asked John after the session regarding their plan for interfacing to MS Project or MS Excel, he confirmed that it is not part of the initial release of the tool but it is part of their long term plan. He hinted that it is not high in their list of priorities.

From my point of view, for a tool to be used for a small or medium size project, there has to be some project management support. One way to achieve this is by interfacing to MS Project or MS Excel. This would be much simpler than building the support internally and majority of the companies use MS Project and MS Excel. The same way IDE is critical for developer community, interfacing to project management tools is essential for project managements.

We have been using VSTS for enactment of our internal projects and its interface to MS Project is important, although there should be more improvements in this area. I feel that in order for Team Concert to be used in small/medium size team, there needs to be some project management support in place.

In my previous posts I had touched upon an important challenge facing Microsoft and IBM as they introduce “game-changing” integrated ALM offerings. How does one go about convincing IT decision makers to start adopting this ALM infrastructure. Outreach activities and a thriving partner eco-system is one of the ways to get the idea out in the open. One example of this is the Jazz team’s attempt to showcase their offerings at the prestigious CASCON 07. This collection of researchers and academics is a fertile seeding ground for advancing the state of art in these concepts.

Yesterday, I attended a BOF session by Marcellus Mindel that discussed opportunities for using Jazz in an academic setting. There was a good discussion on the challenges instructors face in deploying and using Jazz in a curriculum. One of the questions was on curriculum pre-requisites on using Jazz. Do you use Jazz in a first year course before students are aware of the software engineering processes and practices or do you introduce it in fourth year when they have had a chance to get a grounding on some of these practices in addition to essential technical knowhow? Integrated ALM tools like VSTS and Jazz can be an excellent project simulators where instructors can use different process templates to give challenging assignments (a lot like using case studies). Of course, someone may choose to use the ALM tools simply to encourage collaboration between students as they work on their assignments. There were other discussions around research into areas relating to process templates, reporting and generating best practices by using enactment data – all of the concepts that I am passionate about and have blogged in the past.

Later we had a chance to visit the exhibitor area and chat with the IBM Rational Team Concert folks from the Toronto Lab. For those who don’t know, IBM’s Toronto Lab is one of the important brain trusts collaborating on the Jazz/Team Concert platform/product. We wish them our best as they continue to ramp up their outreach activities and aim for the general availability of IBM Rational Team Concert sometime middle of next year.

Here is the Osellus team at the Jazz booth - notice we are appropriately geared !

From left to right - Dave, Payman, Me, Kartik (IBM), Rakesh, Jim, Jason