David had this to say (quite disdainfully ) about my theorizing about getting data out of Jazz into MS Excel and MS Project:

Instead of a one-way data transfer from Jazz to Microsoft documents, I think a much more useful exercise would be to extend MS Office to make it Jazz aware. This would allow the Office applications to behave as Jazz clients much like Microsoft and others have done for VSTS. Without Jazz awareness, I feel that these other applications would be limited to reporting functions and be of little use to the project management community.

I see this more as an exercise in extending MS Office (or your application of choice) rather than extending Jazz and can understand why IBM would be looking for partners to provide such a solution for their customers.

Point is good. Point taken. I am doing precisely what Negin was complaining about - missing the project management perspective and looking at it from the developer’s perspective - but I did fool myself for a bit there that I had the answer signed, sealed and delivered.

The project manager would rather work directly from inside MS Excel, MS Project and interfaces she is familiar with - and it is far fetched to expect non-developer stake-holders to log into a Jazz client/web client and export into Project or report into Excel.

Also, Work Item information (for example) could be changing frequently and direct integration into MS Excel and MS Project provides an opportunity to refresh statuses and other work item information directly from inside a report that the project manager has open compared to having to export repeatedly from inside a Jazz client each time.

Not to fear. As with the Jazz-based solution I outlined yesterday, I have a similarly vague outline of how to do this - atleast for Excel:

As mentioned in my previous post, we already have a way to get the data off the Jazz server.

Microsoft has made it equally easy to develop applications that target the Office ecosystem through VSTO (Visual Studio Tools for Office - currently VSTO 2005). This means that we have application level add-in support for Office system applications like MS Excel.

Through VSTO, I can write a plug-in for Excel that puts a Jazz toolbar/ribbon in Excel and interfaces directly with the service interface of the Jazz Server to get the raw data required off Jazz. This VSTO plugin would then take care of the business logic of putting together and rendering different reports and charts based on pre-defined queries by leveraging the runtime support in MS Excel that VSTO allows.

Reverting to the developer perspective for a bit, VSTO is installed as an add-on on Visual Studio and provides very nice design-time support - just like Win-Forms controls. Makes it that much easier to develop these plug-ins.

Depending on the needs, in case we find that the Jazz service interface is too ‘raw’ for consumption, another option would be to have the Excel VSTO plug-in interact with the Jazz server through a custom Jazz service-plugin that takes care of sending the Excel plug-in well filtered and formatted data. This has some obvious advantages but in my opinion, has the decisive disadvantage of mandatorily requiring a Jazz server side plug-in without which the nice MS Excel plugin we developed will not work.

The part I am not sure about is how to implement a similar VSTO plug-in for MS Project. If I remember correctly, VSTO doesn’t offer support for MS Project development. I could be wrong.

Browsing the VSTO site is becoming a pain right now because I am on a train back to Toronto that has wifi access on board - but connectivity is sporadic at best. I’ll check up a little later and update this post. If it does have support, that’s that.
If not, a plugin can be written directly against the MS Project SDK to achieve our end.

[Update] There seem to be a couple of options for developing MS Project plugins:

• The version of VSTO that comes integrated with Visual Studio 2008 (currently in beta) has support for creating MS Project plug-ins.
• I guess you can create a COM add-in for MS Project using the MS Project SDK I mentioned above but that would be something to consider only if I couldn’t use VSTO.

Last week was pretty busy and I can remember only bits and pieces of it- sort of like Memento without the anterograde amnesia. I digress but it’s one of my favorite movies and I recently watched it again - and was looking for a way to bring it up - even if completely irrelevant :). So there!

One of the things I do remember from last week was attending the Jazz sessions on the 23rd and then the BoF session on 25th at CASCON’07

There were quite a few of us Ossies at Cascon (see Kamal’s post) and I think we all agreed that Jazz and Team Concert were a pretty impressive debut effort by IBM in the ALM space - though crippled by the lack of certain features and perhaps by the potentially dangerous mislabeling of a group of disjointed policies as “process” (a word and space held dear at Osellus :)).

While we didn’t get a chance to sit down and discuss our observations after cascon, we all took away strong observations from our respective perspectives. I can get started on what I really liked and also the large gaps I see in these offerings - but that’s not the reason I am posting this on a weekend when I could be flipping through channels.

I just read Negin’s post regarding the lack of project mgmt. support in Team Concert and I thought I would quickly respond to it before returning to another re-run.

I know exactly where she is coming from and understand this is a pretty important requirement once you get down to actually using Team Concert/Jazz in a real environment.

However, given that Jazz is built on the Eclipse platform, I see this as a job for an extension (or extensions) built on top of Jazz instead of as an inherent deficiency in Jazz. A couple of us had a chance to talk to Kartik Kanakasabesan, the product manager on Jazz at their booth at cascon and he said as much - it’s something that can be built by the community - possibly by somebody who is more qualified to build such extensions.

Hypothetically, with knowledge of the high-level architecture of Jazz and Eclipse, here are what ingredients I would need to build a MS Project or MS Excel Export concoction:

A way to extend Jazz by adding my own services or wrap Jazz services to get data off the Jazz server:
Jazz is built on the plugin-based architecture of Eclipse and has exposed extension-points both on the client side and the service side (Both of which are Equinox OSGi based). These extension points are in addition to the Jazz services themselves that are exposed as part of the service interface (and also have corresponding siblings in the client library)

So, getting the data off the Jazz server to put into my MS Excel report or into MS Project is taken care off.

Getting the data into MS Project:
Negin’s point regarding getting the Work Items in the “process” heirarchy in JAZZ/Team Concert into MS Project or into any other PM tool for that matter is perhaps *the* most important practical fine-grained feature that will get JAZZ out of just the research community into a real project manager’s hands.

MS Project projects can be defined as XML and Microsoft has a well-defined schema for the XML definition.

So, if we mash up JAZZ’s Work Item related service interface API to get the data out and transform the data to a MS project XML, we are good to go!
I can’t get my jazz.net username/pwd to work for some reason so am vaguely referring above to the “Work Item related service API” - I would have preferred to use the right nomenclature.

Getting the data into MS Excel
Getting the data into Excel is similarly not a problem technically.
A server side plugin that the Jazz team calls a ‘common’ plug-in can be built that provides the generic functionality of being able to export data in the Excel format. This plug-in has to be written in Java of course considering this is the Eclipse platform - therefore it can leverage one of the many libraries openly available for reading/writing to the OLE 2 Compound Document Format which includes the MS Excel (XLS) and MS Word (DOC) formats.

The one I rely on usually for similar purposes is Apache’s POI but the choice might depend on the specifics we want to achieve.

I would then build a Jazz service plug-in that wraps the business logic of generating the specific reports we want to output in the Excel format. This service plug-in would get the data using Jazz’s service interface, mash the data up into different types of reports and then output them into spreadsheets using the generic ‘common’ plugin I mentioned above.

A Jazz client plug-in that provides a way to trigger the service plug-in for generation of specific reports or export into MS-Project would be the last step.

I have used Negin’s specific complaints above to demonstrate a hypothetical and technical answer. However, MS Project and MS Excel are but two formats. Formats such as the Open Document Format (ODF) can be just as easily supported. These formats will become increasingly prominent given the adoption of ODF based tooling such as Open Office by governments, educational institutions and SMEs - especially since these are the same organizations that might end up as primary Jazz customers.

Therefore, building such extensions to provide an interface to the project management tooling world is technically not an issue. The real work will be to intelligently define what reports one would want that can leverage such extensions. Negin and others like her who have invaluable years of project management experience are best suited to answer that question!

Notice to Jazz developers: Feel free to tell me how you would do it - or to correct gaps in my understanding.

So Negin….when do you want me to get started on this?

On Tuesday October 23rd, I attended the Jazz session in CASCON 07. John Wiegand demonstrated Team Concert which is built on Jazz platform and he took the audience through some project scenarios, mostly from developers’ point of view. I was impressed by Team Concert and liked the user interface, however, as a project manager I was disappointed about the lack of project management support planned for the initial release of Team Concert.

In the beginning of the session, John had mentioned the pain areas that Jazz was addressing, one of which was project management. So I was looking forward to seeing the extend of this support. The functionalities that were demonstrated mostly focused on IDE as well as team collaboration and communication. Project management support was clearly missing, even simple interface to MS Project or MS Excel. I asked John after the session regarding their plan for interfacing to MS Project or MS Excel, he confirmed that it is not part of the initial release of the tool but it is part of their long term plan. He hinted that it is not high in their list of priorities.

From my point of view, for a tool to be used for a small or medium size project, there has to be some project management support. One way to achieve this is by interfacing to MS Project or MS Excel. This would be much simpler than building the support internally and majority of the companies use MS Project and MS Excel. The same way IDE is critical for developer community, interfacing to project management tools is essential for project managements.

We have been using VSTS for enactment of our internal projects and its interface to MS Project is important, although there should be more improvements in this area. I feel that in order for Team Concert to be used in small/medium size team, there needs to be some project management support in place.

In my previous posts I had touched upon an important challenge facing Microsoft and IBM as they introduce “game-changing” integrated ALM offerings. How does one go about convincing IT decision makers to start adopting this ALM infrastructure. Outreach activities and a thriving partner eco-system is one of the ways to get the idea out in the open. One example of this is the Jazz team’s attempt to showcase their offerings at the prestigious CASCON 07. This collection of researchers and academics is a fertile seeding ground for advancing the state of art in these concepts.

Yesterday, I attended a BOF session by Marcellus Mindel that discussed opportunities for using Jazz in an academic setting. There was a good discussion on the challenges instructors face in deploying and using Jazz in a curriculum. One of the questions was on curriculum pre-requisites on using Jazz. Do you use Jazz in a first year course before students are aware of the software engineering processes and practices or do you introduce it in fourth year when they have had a chance to get a grounding on some of these practices in addition to essential technical knowhow? Integrated ALM tools like VSTS and Jazz can be an excellent project simulators where instructors can use different process templates to give challenging assignments (a lot like using case studies). Of course, someone may choose to use the ALM tools simply to encourage collaboration between students as they work on their assignments. There were other discussions around research into areas relating to process templates, reporting and generating best practices by using enactment data – all of the concepts that I am passionate about and have blogged in the past.

Later we had a chance to visit the exhibitor area and chat with the IBM Rational Team Concert folks from the Toronto Lab. For those who don’t know, IBM’s Toronto Lab is one of the important brain trusts collaborating on the Jazz/Team Concert platform/product. We wish them our best as they continue to ramp up their outreach activities and aim for the general availability of IBM Rational Team Concert sometime middle of next year.

Here is the Osellus team at the Jazz booth - notice we are appropriately geared !

From left to right - Dave, Payman, Me, Kartik (IBM), Rakesh, Jim, Jason

Last night I had a chance to attend a Jazz demonstration by John Wiegand at CASCON 07. It is always a treat to watch John speak and yesterday’s session was no different. For those who have not seen John and Erich speak about collaborative development, here is a short video to one of their previous videos talking about open source development.

John gave a glimpse of the current state of Jazz technology. Although Erich was unable to join the demonstration concurrently from OOPSLA in Montreal as planned, judging from the reaction of the standing room only audience the session was well received. There are several cool things that were shown including the concept of personal work spaces that amongst other things let developers see the effects of checkins before touching the main code repository. John ran through a scenario in which two new developers join an existing project. The system configured the tool behavior for these newcomers based on the process definition. Living up to the promise of transparency, these newcomer developers get access to all work items in their project and across different projects. They then get down to the brass tracks by working on their own work items while engaging in adhoc collaboration and sharing changes with others in the team.

I had a chance to chat with John after the demonstration. Since my focus was primarily on understanding the process awareness built into this integrated toolset, John was quick to point out their definition of process in Jazz. Process in Jazz means the policy and configuration settings that govern tool behavior. The process definition such as checkin polices and build management can be customized for individual projects. We then had a candid discussion on the need to have balance between having a process definition that spawns project work items without burdening developers with complicated process terminology. I can’t go onto all the details here but we agreed that over-complicated process models (based on over-engineered meta-models that require a steep learning curve) could be harmful to the concept of collaborative development environments. This is the reason the Jazz team wants to be very careful when they talk about process and their definition of it.

John was also generous to help me with a favour. He has promised to see how can he make the EclipseWay available to the open source community. This will mean anyone can take this collection of best practices and mash them up with their own in-house process description or third party process definitions without getting a knock on their door by a team of IBM lawyers. I know many people in the EPF community had been asking for such a thing and will join me in thanking John for helping us with this.

I am back at CASCON on Thursday to join a BOF session on Jazz.

Next week will find me and some other members of my team attending the Jazz sessions at Cascon 07 here in Toronto.

As I had talked about in my earlier blog post, I am really pleased to see the work Erich’s team has been doing on the jazz platform. Moreover it is nice to see so much Canadian talent working on developing this next generation ALM platform!

From what I keep hearing, the Jazz platform and its first product manifestation in IBM Rational Team Concert is sounding more and more like Microsoft’s Visual Studio Team System. This is a good thing and something to take note. In fact I would even term the release of Jazz and VSTS as “game-changing” events that have the potential to alter the state of software development lifecycle tooling. This would also be giving sleepless nights to those vendors who claim to offer traditional ALM tooling consisting of a collection of development tools. Simply put the traditional ad-hoc installation and configuration of a collection of individual applications has just not worked. Moreover, customers were expected to go through an expensive and prolong point-to-point integration of these tool “silos”. Even the most comprehensive customer deployment only resulted in partial integration and never met the promised results. Do I need even need to go further on this point…

The key distinguishing characteristic of these ALM offerings from Microsoft and IBM Jazz is the infrastructure layer. Finding similarities with the application middleware (another game changing event) Payman, calls this layer the “middleware” that glues tools and processes together. The infrastructure layer facilitates concurrent development projects each with its own team and tools. It’s a common repository of projects’ data. Unified management of project data makes governance, visibility, predictability, traceability and other ALM business benefits possible. At the core of the infrastructure layer is a unifying object model. The object model is accessible via a comprehensive set of APIs. The ALM tools utilize these event-based API’s for deep integration with the infrastructure layer. Since these APIs are open and published, any third-party tool can also integrate deeply into a ALM solution. Each tool may have its own object model but as part of integrations the tool’s object model is mapped to the infrastructure object model.

The common infrastructure object model is based on the key concept of work items. Microsoft has a few years lead in introducing this concept by way of Visual Studio Team System. Work items are project specific elements, created and updated throughout the life of the project manually or automatically by tools and processes (based on defined rules and constraints). A work item is an instance of a work item type, which is defined as part of the configuration of the ALM solution. Work item types can be associated with each other by user definable link types. Therefore, the object model is fully configurable which makes the ALM solution very versatile. Additionally, the infrastructure layer provides a powerful reporting and data mining service, supporting user defined dashboards and reports for an individual project or consolidation of multiple projects.

This opens the field for tremendous innovation. A community of plug-in providers can now add missing or extend existing functionality. There is enough room to identify and build competencies on this platform - both product and service opportunities. This means a more thriving community that ultimately results in a better ROI for enterprise customers. But there is still a big challenge left. Remembering that they have heard these “cry wolf” calls on ALM promise before, how can IT decision makers be convinced that this is the real deal!

I will share if I hear something interesting at the Jazz sessions at Cascon 07.

In my view IBM is struggling to align their products and practices with the advancement in the state of around software development process automation and collaborative development environment.

In this post I am going to focus on their release of Rational Method Composer 7.2 (RMC 7.2) and specifically point out why it is difficult for me to be convinced that they have done a root cause analysis of their previous failures (with RPW or earlier versions of RMC) or have recognized the direction where software process automation is headed.

When one thinks of a product release from IBM you would obviously expect a high level of maturity, sophistication and thought leadership. IBM is universally well regarded and respected for pushing concepts like SOA and software-as-service. However, when you look at how the Rational team has implemented the tooling around creating and tailoring processes, one wonders why that thought leadership is not being put in practice here. By choosing to keep RMC 7.2 as a thick client application the rational team seems to be stuck in a time warp. I simply cannot imagine a scenario where an enterprise is told that its process assets (which in many cases have the “secret sauce” of project success) is encased in a few laptops. So I was bit surprised that even while acknowledging this problem of scalability and multi-methodology blending, the rational team has chosen to resolve it by continuing to stick to the flawed design decision of using SCM to check-in/check-out process plug-ins. But now, to make matters even more difficult, process authors (often non-programmers), are required to use the eclipse IDE environment to derive some scalability benefits. This is simply unbelievable. First of all, this still does not solve the problem of scalability (see why). Secondly, SCM is a code repository specially built for handling code and not process assets. Finally there is little logic in making process authoring so complicated that it requires a team of consultants to even setup the multi-layer environment for process authoring. Moreover in this day and age when even videos are being edited online asking a process team to work in this client-only disconnected mode is simply irresponsible. I did notice that the rational team did take note of the overwhelming negative feedback on usability of RMC and I welcome that. As in IBM partner and while exhibiting at the RSDC in June 07 we heard lot of this negative feedback. As you read this please keep in mind that if you don’t have a business need for blended methodology processes (and RUP is all you need) or if you are fine with having one or two process authors, RMC does meet your needs. In fact we have a handful of customers who are in such a situation and would simply like to move their RUP process into a collaborative development environment such as VSTS using Content Bridge for VSTS. I hope to see some improvements in the UI with RMC 7.2 and will share what I find out.