I have been keenly watching the progress on what started as a promising project in the Eclipse foundation. Unfortunately after more than two years of opportunity to shine I would have to give the EPF project a failing grade. I fear that when someone at IBM realizes the wasteful investment of resources in this project they will pull the plug on their participation at EPF. This will inevitably kill the project since IBM is the only committer on the tool aspect of EPF and the major committer and donor to the content aspect of EPF.

In my view there are several reasons for this failure and many lessons that I have learnt simply by following the travails of the project. These reasons are not the lack of aptitude and personal contribution of the EPF team but simply a miscalculated approach to building a vibrant self-organizing process community. Today I will highlight the mind boggling high transaction costs of submitting process content to be consumed and critiqued by a peer network. Even by their own admission, it takes “2-3 months” for the eclipse legal process to approve any process contributions to EPF. This is an anachronistic concept in today’s day and age. Communities are self-organizing. If what I contribute is useless to you, you will not use it. Period. Moreover having been tagged as someone who supplies useless content my user ratings will go down and I will have to significantly improve my game to get back my reputation. Communities give rise to wisdom-of-the-crowd, encourage meritocracy and do not tolerate back room machinations. As articulated by Chris Anderson in The Long Tail while discussing the tools needed to support a long tail, “This is the end of spoon-fed orthodoxy and infallible institutions, and the rise of messy mosaics of information that require - and reward – investigation”. Elsewhere in the book he discusses how “a once-monolithic industry structure where professionals produced and amateurs consumed is now a two-way marketplace, where anyone can be in any camp at any time. This is just a hint of the sort of profound change that the democratized tools of production and distribution can foster”. In the case of software process modeling, these tools take the form of easy to use process modeling and consumption tools. With that, rather than a paradigm of a handful of processes being donated by a few process experts, one could feasibly have hundreds of process variants created, put on wikis, blogged and rated! Now if we multiply the 2-3 months timeframe for these 100 processes the EPF legal staff will be kept busy for 20 years…just on this project! Later I will discuss some other pitfalls to avoid based on the lessons learned from following this project.

Yesterday Omid and I gave a joint Microsoft-Osellus webinar on Effective Software Development Governance with Adam Galant. Thanks to Rini Gahir and Adam Galant from Microsoft Canada for arranging this webinar.

It was a challenge to fit an explanation of the theoretical concepts of SD Governance with a tangible step-by-step approach to achieving it within the 45 minutes we had. In the end we were able to keep to our time and I hope the audience (about 36 attendees) left with an outline of a practical approach to establishing software development governance in an automated yet flexible manner.

Here is a link to a recording of the governance webinar.

You can use this link to register for the joint Microsoft-Osellus Webinar on software development governance. Omid is going to be talking about a lot of things that we have been able to achieve with the collaboration between the osellus process toolset/service offerings and the VSTS integrated ALM.

Here is a preview of what we will be talking about:

Effective governance aligns your software development investment with your business strategy. By ensuring continuous traceability of software development activities, governance leads to enhanced customer satisfaction and compliance with internal and external policies.

While most organizations use some measure of implicit governance, the full potential of a governance framework can be realized by taking three practical and explicit measures. The software development processes in the organization need to be selected and tailored in a manner that supports organization’s business goals. These processes need to be enforced in a streamlined manner. And, they need to be monitored to ensure corrective actions are taken in a timely manner.

In this webinar we outline a practical approach to establish software development governance in an automated yet flexible manner. We will discuss how the business objectives, standards and control mechanisms in the organization should be mapped to process elements such a roles, activities and work products that guide practitioners in a non-intrusive manner. Microsoft Visual Studio Team System (VSTS) and the IRIS process automation suite from Osellus will be used to demonstrate implementation of an effective governance framework in real-world scenarios.

David had this to say (quite disdainfully ) about my theorizing about getting data out of Jazz into MS Excel and MS Project:

Instead of a one-way data transfer from Jazz to Microsoft documents, I think a much more useful exercise would be to extend MS Office to make it Jazz aware. This would allow the Office applications to behave as Jazz clients much like Microsoft and others have done for VSTS. Without Jazz awareness, I feel that these other applications would be limited to reporting functions and be of little use to the project management community.

I see this more as an exercise in extending MS Office (or your application of choice) rather than extending Jazz and can understand why IBM would be looking for partners to provide such a solution for their customers.

Point is good. Point taken. I am doing precisely what Negin was complaining about - missing the project management perspective and looking at it from the developer’s perspective - but I did fool myself for a bit there that I had the answer signed, sealed and delivered.

The project manager would rather work directly from inside MS Excel, MS Project and interfaces she is familiar with - and it is far fetched to expect non-developer stake-holders to log into a Jazz client/web client and export into Project or report into Excel.

Also, Work Item information (for example) could be changing frequently and direct integration into MS Excel and MS Project provides an opportunity to refresh statuses and other work item information directly from inside a report that the project manager has open compared to having to export repeatedly from inside a Jazz client each time.

Not to fear. As with the Jazz-based solution I outlined yesterday, I have a similarly vague outline of how to do this - atleast for Excel:

As mentioned in my previous post, we already have a way to get the data off the Jazz server.

Microsoft has made it equally easy to develop applications that target the Office ecosystem through VSTO (Visual Studio Tools for Office - currently VSTO 2005). This means that we have application level add-in support for Office system applications like MS Excel.

Through VSTO, I can write a plug-in for Excel that puts a Jazz toolbar/ribbon in Excel and interfaces directly with the service interface of the Jazz Server to get the raw data required off Jazz. This VSTO plugin would then take care of the business logic of putting together and rendering different reports and charts based on pre-defined queries by leveraging the runtime support in MS Excel that VSTO allows.

Reverting to the developer perspective for a bit, VSTO is installed as an add-on on Visual Studio and provides very nice design-time support - just like Win-Forms controls. Makes it that much easier to develop these plug-ins.

Depending on the needs, in case we find that the Jazz service interface is too ‘raw’ for consumption, another option would be to have the Excel VSTO plug-in interact with the Jazz server through a custom Jazz service-plugin that takes care of sending the Excel plug-in well filtered and formatted data. This has some obvious advantages but in my opinion, has the decisive disadvantage of mandatorily requiring a Jazz server side plug-in without which the nice MS Excel plugin we developed will not work.

The part I am not sure about is how to implement a similar VSTO plug-in for MS Project. If I remember correctly, VSTO doesn’t offer support for MS Project development. I could be wrong.

Browsing the VSTO site is becoming a pain right now because I am on a train back to Toronto that has wifi access on board - but connectivity is sporadic at best. I’ll check up a little later and update this post. If it does have support, that’s that.
If not, a plugin can be written directly against the MS Project SDK to achieve our end.

[Update] There seem to be a couple of options for developing MS Project plugins:

• The version of VSTO that comes integrated with Visual Studio 2008 (currently in beta) has support for creating MS Project plug-ins.
• I guess you can create a COM add-in for MS Project using the MS Project SDK I mentioned above but that would be something to consider only if I couldn’t use VSTO.

Last week was pretty busy and I can remember only bits and pieces of it- sort of like Memento without the anterograde amnesia. I digress but it’s one of my favorite movies and I recently watched it again - and was looking for a way to bring it up - even if completely irrelevant :). So there!

One of the things I do remember from last week was attending the Jazz sessions on the 23rd and then the BoF session on 25th at CASCON’07

There were quite a few of us Ossies at Cascon (see Kamal’s post) and I think we all agreed that Jazz and Team Concert were a pretty impressive debut effort by IBM in the ALM space - though crippled by the lack of certain features and perhaps by the potentially dangerous mislabeling of a group of disjointed policies as “process” (a word and space held dear at Osellus :)).

While we didn’t get a chance to sit down and discuss our observations after cascon, we all took away strong observations from our respective perspectives. I can get started on what I really liked and also the large gaps I see in these offerings - but that’s not the reason I am posting this on a weekend when I could be flipping through channels.

I just read Negin’s post regarding the lack of project mgmt. support in Team Concert and I thought I would quickly respond to it before returning to another re-run.

I know exactly where she is coming from and understand this is a pretty important requirement once you get down to actually using Team Concert/Jazz in a real environment.

However, given that Jazz is built on the Eclipse platform, I see this as a job for an extension (or extensions) built on top of Jazz instead of as an inherent deficiency in Jazz. A couple of us had a chance to talk to Kartik Kanakasabesan, the product manager on Jazz at their booth at cascon and he said as much - it’s something that can be built by the community - possibly by somebody who is more qualified to build such extensions.

Hypothetically, with knowledge of the high-level architecture of Jazz and Eclipse, here are what ingredients I would need to build a MS Project or MS Excel Export concoction:

A way to extend Jazz by adding my own services or wrap Jazz services to get data off the Jazz server:
Jazz is built on the plugin-based architecture of Eclipse and has exposed extension-points both on the client side and the service side (Both of which are Equinox OSGi based). These extension points are in addition to the Jazz services themselves that are exposed as part of the service interface (and also have corresponding siblings in the client library)

So, getting the data off the Jazz server to put into my MS Excel report or into MS Project is taken care off.

Getting the data into MS Project:
Negin’s point regarding getting the Work Items in the “process” heirarchy in JAZZ/Team Concert into MS Project or into any other PM tool for that matter is perhaps *the* most important practical fine-grained feature that will get JAZZ out of just the research community into a real project manager’s hands.

MS Project projects can be defined as XML and Microsoft has a well-defined schema for the XML definition.

So, if we mash up JAZZ’s Work Item related service interface API to get the data out and transform the data to a MS project XML, we are good to go!
I can’t get my jazz.net username/pwd to work for some reason so am vaguely referring above to the “Work Item related service API” - I would have preferred to use the right nomenclature.

Getting the data into MS Excel
Getting the data into Excel is similarly not a problem technically.
A server side plugin that the Jazz team calls a ‘common’ plug-in can be built that provides the generic functionality of being able to export data in the Excel format. This plug-in has to be written in Java of course considering this is the Eclipse platform - therefore it can leverage one of the many libraries openly available for reading/writing to the OLE 2 Compound Document Format which includes the MS Excel (XLS) and MS Word (DOC) formats.

The one I rely on usually for similar purposes is Apache’s POI but the choice might depend on the specifics we want to achieve.

I would then build a Jazz service plug-in that wraps the business logic of generating the specific reports we want to output in the Excel format. This service plug-in would get the data using Jazz’s service interface, mash the data up into different types of reports and then output them into spreadsheets using the generic ‘common’ plugin I mentioned above.

A Jazz client plug-in that provides a way to trigger the service plug-in for generation of specific reports or export into MS-Project would be the last step.

I have used Negin’s specific complaints above to demonstrate a hypothetical and technical answer. However, MS Project and MS Excel are but two formats. Formats such as the Open Document Format (ODF) can be just as easily supported. These formats will become increasingly prominent given the adoption of ODF based tooling such as Open Office by governments, educational institutions and SMEs - especially since these are the same organizations that might end up as primary Jazz customers.

Therefore, building such extensions to provide an interface to the project management tooling world is technically not an issue. The real work will be to intelligently define what reports one would want that can leverage such extensions. Negin and others like her who have invaluable years of project management experience are best suited to answer that question!

Notice to Jazz developers: Feel free to tell me how you would do it - or to correct gaps in my understanding.

So Negin….when do you want me to get started on this?

In my previous posts I had touched upon an important challenge facing Microsoft and IBM as they introduce “game-changing” integrated ALM offerings. How does one go about convincing IT decision makers to start adopting this ALM infrastructure. Outreach activities and a thriving partner eco-system is one of the ways to get the idea out in the open. One example of this is the Jazz team’s attempt to showcase their offerings at the prestigious CASCON 07. This collection of researchers and academics is a fertile seeding ground for advancing the state of art in these concepts.

Yesterday, I attended a BOF session by Marcellus Mindel that discussed opportunities for using Jazz in an academic setting. There was a good discussion on the challenges instructors face in deploying and using Jazz in a curriculum. One of the questions was on curriculum pre-requisites on using Jazz. Do you use Jazz in a first year course before students are aware of the software engineering processes and practices or do you introduce it in fourth year when they have had a chance to get a grounding on some of these practices in addition to essential technical knowhow? Integrated ALM tools like VSTS and Jazz can be an excellent project simulators where instructors can use different process templates to give challenging assignments (a lot like using case studies). Of course, someone may choose to use the ALM tools simply to encourage collaboration between students as they work on their assignments. There were other discussions around research into areas relating to process templates, reporting and generating best practices by using enactment data – all of the concepts that I am passionate about and have blogged in the past.

Later we had a chance to visit the exhibitor area and chat with the IBM Rational Team Concert folks from the Toronto Lab. For those who don’t know, IBM’s Toronto Lab is one of the important brain trusts collaborating on the Jazz/Team Concert platform/product. We wish them our best as they continue to ramp up their outreach activities and aim for the general availability of IBM Rational Team Concert sometime middle of next year.

Here is the Osellus team at the Jazz booth - notice we are appropriately geared !

From left to right - Dave, Payman, Me, Kartik (IBM), Rakesh, Jim, Jason

Last night I had a chance to attend a Jazz demonstration by John Wiegand at CASCON 07. It is always a treat to watch John speak and yesterday’s session was no different. For those who have not seen John and Erich speak about collaborative development, here is a short video to one of their previous videos talking about open source development.

John gave a glimpse of the current state of Jazz technology. Although Erich was unable to join the demonstration concurrently from OOPSLA in Montreal as planned, judging from the reaction of the standing room only audience the session was well received. There are several cool things that were shown including the concept of personal work spaces that amongst other things let developers see the effects of checkins before touching the main code repository. John ran through a scenario in which two new developers join an existing project. The system configured the tool behavior for these newcomers based on the process definition. Living up to the promise of transparency, these newcomer developers get access to all work items in their project and across different projects. They then get down to the brass tracks by working on their own work items while engaging in adhoc collaboration and sharing changes with others in the team.

I had a chance to chat with John after the demonstration. Since my focus was primarily on understanding the process awareness built into this integrated toolset, John was quick to point out their definition of process in Jazz. Process in Jazz means the policy and configuration settings that govern tool behavior. The process definition such as checkin polices and build management can be customized for individual projects. We then had a candid discussion on the need to have balance between having a process definition that spawns project work items without burdening developers with complicated process terminology. I can’t go onto all the details here but we agreed that over-complicated process models (based on over-engineered meta-models that require a steep learning curve) could be harmful to the concept of collaborative development environments. This is the reason the Jazz team wants to be very careful when they talk about process and their definition of it.

John was also generous to help me with a favour. He has promised to see how can he make the EclipseWay available to the open source community. This will mean anyone can take this collection of best practices and mash them up with their own in-house process description or third party process definitions without getting a knock on their door by a team of IBM lawyers. I know many people in the EPF community had been asking for such a thing and will join me in thanking John for helping us with this.

I am back at CASCON on Thursday to join a BOF session on Jazz.

Next week will find me and some other members of my team attending the Jazz sessions at Cascon 07 here in Toronto.

As I had talked about in my earlier blog post, I am really pleased to see the work Erich’s team has been doing on the jazz platform. Moreover it is nice to see so much Canadian talent working on developing this next generation ALM platform!

From what I keep hearing, the Jazz platform and its first product manifestation in IBM Rational Team Concert is sounding more and more like Microsoft’s Visual Studio Team System. This is a good thing and something to take note. In fact I would even term the release of Jazz and VSTS as “game-changing” events that have the potential to alter the state of software development lifecycle tooling. This would also be giving sleepless nights to those vendors who claim to offer traditional ALM tooling consisting of a collection of development tools. Simply put the traditional ad-hoc installation and configuration of a collection of individual applications has just not worked. Moreover, customers were expected to go through an expensive and prolong point-to-point integration of these tool “silos”. Even the most comprehensive customer deployment only resulted in partial integration and never met the promised results. Do I need even need to go further on this point…

The key distinguishing characteristic of these ALM offerings from Microsoft and IBM Jazz is the infrastructure layer. Finding similarities with the application middleware (another game changing event) Payman, calls this layer the “middleware” that glues tools and processes together. The infrastructure layer facilitates concurrent development projects each with its own team and tools. It’s a common repository of projects’ data. Unified management of project data makes governance, visibility, predictability, traceability and other ALM business benefits possible. At the core of the infrastructure layer is a unifying object model. The object model is accessible via a comprehensive set of APIs. The ALM tools utilize these event-based API’s for deep integration with the infrastructure layer. Since these APIs are open and published, any third-party tool can also integrate deeply into a ALM solution. Each tool may have its own object model but as part of integrations the tool’s object model is mapped to the infrastructure object model.

The common infrastructure object model is based on the key concept of work items. Microsoft has a few years lead in introducing this concept by way of Visual Studio Team System. Work items are project specific elements, created and updated throughout the life of the project manually or automatically by tools and processes (based on defined rules and constraints). A work item is an instance of a work item type, which is defined as part of the configuration of the ALM solution. Work item types can be associated with each other by user definable link types. Therefore, the object model is fully configurable which makes the ALM solution very versatile. Additionally, the infrastructure layer provides a powerful reporting and data mining service, supporting user defined dashboards and reports for an individual project or consolidation of multiple projects.

This opens the field for tremendous innovation. A community of plug-in providers can now add missing or extend existing functionality. There is enough room to identify and build competencies on this platform - both product and service opportunities. This means a more thriving community that ultimately results in a better ROI for enterprise customers. But there is still a big challenge left. Remembering that they have heard these “cry wolf” calls on ALM promise before, how can IT decision makers be convinced that this is the real deal!

I will share if I hear something interesting at the Jazz sessions at Cascon 07.

As soon as I hit ‘Publish’ on my previous post yesterday, I came across this very interesting announcement-
BEA Systems and Adobe Systems announced they will be partnering to provide Adobe’s Flex Builder 2 bundled with BEA’s Workshop Studio - this bundle due out later this year. On their part, Adobe will distribute BEA Weblogic Server evaluation instances with their LifeCycle Enterprise Suite (ES) offering - planned for early 2008.

An understanding is emerging that hosted application infrastructure- Web 2.0 or in the enterprise - cannot be considered complete anymore without inherent support for rich client application frameworks. SOA, web services etc. are enabling technologies but the workflow needs to include a sophisticated client that can leverage them - and vice versa.

Adobe’s also doing the right thing if you look at this from the perspective of competition. Microsoft’s Silverlight is already grounded on the .Net platform - With .Net 3.0 and it’s accompanying middleware technology set including WPF, WCF and WF- it now has a powerful foundation to lean on - not to mention a powerful development environment support in Visual Studio.
Adobe has it’s own middleware-like offerings (ColdFusion or JRun anyone?) but they are not nearly as ‘complete’ as the application servers such as Weblogic, IBM’s Websphere, JBoss and a few others. So they need this kind of bundling.

More on this announcement here.

A sense of validation is always nice - especially when you are so far ahead of the curve. I was scanning through my Google Reader Link Blog recently and noticed the number of new applications being implemented in Flex/Flash and Apollo - Adobe’s desktop runtime for RIAs.

Microsoft recently joined the game officially with the launch of version 1 of it’s Silverlight rich interaction delivery platform and while it’s some way behind Flash/Flex as a platform, it’s gaining popularity too - and has a lot of things going for it (a topic for another post).

I want to focus on Adobe preview of a new product last week at the Photoshop World - Photoshop Express, which is an entry-level Photoshop branded product aimed at drawing customers into the world of Photoshop. What’s special about Photoshop Express is that it’s hosted and implemented using Flex (the development environment for Flash).

This is a continuation of Adobe’s recent focus on the hosted model - Adobe had put out a hosted video editing tool, Remix that has been available as part of the Photobucket tool-set for about 6 months. This video tool has other nearly identical avatars - it also exists as Adobe Premiere Express and is provided on Youtube as Youtube Remixer.

This is the best possible advertisement for the Flash platform’s power. As a developer, there is nothing that screams out a platform’s strength than a complex image/video processing software implemented in it! And there are more coming out everyday!

Flash/Flex has been rounding up some well known names in its corner. I am very impressed with the way Adobe has targeted and honed the direction of the Flex/Flash platform after it adopted Macromedia. The AIR platform (formerly known as Apollo) which has gained a lot of steam since it’s launch is another crucial step in that direction.

What direction is that? Flash is no more seen as the delivery tool of the infamous banner ads. It is now perceived as a serious platform for rich and complex application delivery used by the big boys of the internet and increasingly, the enterprise.

Which brings me to the sense of validation I was talking about - Long before Adobe and the others, Osellus was one of the first “serious” product companies to recognize the power and potential of Flash as a platform and employ it to develop a complex visual enterprise-class product. IRIS Process Author is an enterprise-class visual process management product currently being used by very qualified process groups in very large organizations - and its primary authoring interface is implemented in Flash and the IRIS Flash Framework!! We just happened to start 5 years ago and given our early start, we had to develop our own framework for implementing our Flash-based interface. I was involved in designing and guiding Process Author’s interface over the last few years and so felt the need to make sure Osellus’s foresight was recognized - at least in a blog posting :).

Other examples of companies waking up to the potential of Flex/Flash are Google and Salesforce.com which have been adding some Flash-based products and APIs to their catalogs.

Considering our strong enterprise focus as a company, I am especially peaked by the strong uptake of Salesforce.com’s Adobe Flex Toolkit for Apex. Apex (Salesforce AppExchange) is of course Salesforce’s applications marketplace and one of the first implementations of the SaaS model.

On the consumer-internet front, Google made available the Google Talk Gadget a few months ago which is a Flash implementation of Google Talk that can be added onto your Google Personalized Homepage. The gadget can be even be added to a blog or webpage so visitors can chat with you right there in the context of your blog. Best of all, I noticed that it has the “Call” button - so you can VoIP with your google talk contacts right there - no need for the desktop client! I chatted up a couple of friends and was pretty impressed with the implementation. The gadget has a lot of other features too - like being able to play youtube videos and recognizing picasa albums.
The reason I highlight the Google Talk gadget in particular is that I was hard pressed to find a difference in the user experience between the gadget and the Google Talk desktop application. Of course this is as much as ode to the Google developers as to the Flash platform itself.

I suspect we will see Google employing Flash more and more in their upcoming “betas”.

Another web app that I am really impressed with is Virtual Ubiquity’s Buzzword. VU is calling it “the first real word processor for the web”. It’s written in Flex 2 for Flash 9 and is extremely sophisticated and rich in features.

These are but a few of the examples out there in the enterprise and web 2.0 arenas. All in all, this trend indicates the exciting albeit late recognition of a trend that Osellus had woken up to sometime ago.

I’m pretty sure we’ll be seeing more and more of Flash/Flex and Silverlight based RIAs delivered either in the web browser or in an Apollo like delivery model in the next year or two. I itch to articulate my p.o.v. on the Flash vs. Ajax pseudo-debate that’s been on for the last couple of years but that’s a whole new post.

I am keen to know which other enterprise level products have Flash based interfaces, APIs or atleast components. If you know of any, let me know! I would also be interested in knowing about early adopters of Flash in the enterprise along with Osellus or even perhaps before us!