Our design team has been spending some time the last few weeks at taking another look at our architectural practices and also at standardizing on a core Web application architecture that we can then drop into the heart of any web application and then extend/wrap it based on the needs of the particular web application we may be working on.

Performance is one of the key factors in determining best practices and for data intensive applications, the database and data access layer performance in particular can make or break an application’s responsiveness more than any other layer in the application.

As we do a lot of ASP.Net/ .Net development, one of our designers Sanguanchai who is based in our Bangkok office has been trying to break down the use of LINQ to SQL versus using the Entity Framework 4 (currently in beta 1) across key considerations – not just performance - for use in our core architecture.

You can take a look at his findings in the post he published yesterday - LINQ to SQL vs Entity Framework. No doubt there are other architects and developers out there trying to get a handle on the same question and we hope you find this useful.

On a slightly related note, if you are the podcast listening type, .Net Rocks had Dan Simmons, Dev Manager on the EF4 and LINQ to SQL Team as a guest on the latest show. It’s good listen and you get some background on Microsoft’s decisions – including why EF is at version 4 though it is actually the 2nd release. You also get a view into the decisions behind certain features in EF4 and I personally got a better understanding of how best to leverage certain features of EF4 that were opaque to me thus far.

Again, the direct link to Sanguanchai’s post:

http://www.osellus.com/blogs/2009/06/04/linq-to-sql-vs-entity-framework/

Enjoy!

I was trying to run a load test using Visual Studio Team Test Edition yesterday for a large data set and came across an error that said:

Error occurred running test. (Computer LOADTESTMACHINE) Could not access the load test results repository: The load test results repository is out of space. Allocate more space to the repository (if possible), or delete results of older load test runs.

Visual Studio dumps test results in a local instance of MS SQL Server Express and it was easy enough to guess that this was due to SQL Server Express exceeding the db size cap of 4GB (non-Express editions don’t have this cap). I have been running largish load tests on this machine for 8-9 months now and it made sense that I would run out of space.

I went looking for a solution and saw that Microsoft devs have suggested in forums and in blog posts that the DB be cleaned up via SQL to delete test results older than a time period of your choice.

Two things surprised me:

• There is no UI option to manage this in Visual Studio. Before I ran the sql commands, I tried deleting old test results via the “Test Results” window in VS. This apparently deletes exported test results - I am not sure what else it does but it didn’t seem to clean up the DB for the test results. So this means that if you want to clean up your DB when you run into these kind of issues, you need to talk dirty to SQL Server directly. Feels unclean. I Would like to know if I am missing something because this seems like something that shouldn’t require testers even going near a SQL Script.
• I tried and could not find any formal guidance or documentation on msdn to address this issue, even in Practices & Patterns. I may have missed it totally so I hope somebody corrects me if this is actually there. This would be a pretty common scenario I would think so it deserves better than users needing to search blogs and the forums for an answer.

The script I ran is included with this post. A few things to keep in mind:

1. This script cleans up only Load Test Results
2. If you are using a typical installation of Visual Studio Team Test Edition, you need to run this against the db named “LoadTest” on the locally installed instance of MS SQL Server Express installed during Visual Studio installation.
3. Change the value of the @DaysToKeep variable to a suitable value in days. I have set it to 30 days in the script so that results older than 30 days are deleted.
   --@DaysToKeep - indicates the number of previous days records to keep. For example "30" indicates that I want to delete test results older than 30 days
DECLARE @DaysToKeep int
Set @DaysToKeep = 30
4. This script may take a very long time to run - think in terms of hours rather than minutes. It took about 2-2.5 hours in my case. So, don’t abort the execution because you think the script is stuck.

The script (rename to .sql):
deleteoldloadtestresults.txt

Application Lifecycle Management (ALM) is often wrongly thought to only cover only the Software Development Life cycle (SDLC). David Chappell’s excellent and concise whitepaper focuses on this misconception and explains how ALM is more than just the development of the application though that is certainly an important part of it.

ALM is broken down into the Governance, Development and Operations aspects. More importantly Chappell continues the point I had made briefly in my previous post - that there are ALM tools out there which are largely concentrated on the development aspect but their true value can be achieved by integrating them with tools that cover other aspects of ALM - eg: CRM.

To quote the whitepaper:

The three aspects of ALM—governance, development, and operations—are tightly connected to one another. Doing all three well is a requirement for any organization that aspires to maximize the business value of custom software. But this isn’t an easy goal to achieve. Each of the three is challenging to get
right on its own, and so getting the combination right is even more challenging. The right tools can make this easier. A number of vendors today provide tools that are horizontally integrated, i.e., tools that work together well on one of the three lines. For example, Microsoft’s Visual Studio Team System brings together a range of tools supporting several aspects of the development process. Yet tools should be integrated not just horizontally but vertically as well, helping organizations make connections across the three lines. For instance, project management tools should be connected to development tools, which in turn should have connections to the tools used for operations.

These connections are beginning to appear. Visual Studio Team System, for example, can connect with Microsoft’s Project Server to help project managers get up-to-date information on what developers are doing. There’s still plenty of room for improvement, however, and no vendor today offers a set of ALM tools with full vertical integration across all three lines. Tools should be integrated not just horizontally but vertically too.

Read the complete white paper here.

I have been reading Payman’s posts over the past week and the No new Methodology Please; Let’s focus on Tools post in particular stuck with me because it focuses on something I have been thinking about too. His post talks about the noise in the processes domain where there is a lot of focus on coming up with unnecessarily niche processes rather than focus on actually making processes useful by developing tools that make employing existing mature processes practical.

While Payman’s post focuses on processes, it struck me that even where tools go, often companies employ ALM tools with a narrow focus without fully grasping how they can be fully leveraged and what capabilities these tools pack. Over the last year, we have helped design and deploy solutions based on Microsoft Visual studio Team System (VSTS) for a few customers. One of the key needs that kept leaping out at us from all these engagements is that most customers wanted a way to implement requirements traceability – and they were not always aware that most ALM tools already have the capability to implement end-to-end Requirements Management – in particular traceability.

VSTS for example provides advanced linking capabilities that can be leveraged to allow tracking execution of a requirement from beginning to end (and backwards as well). A number of tools have come up around VSTS to satisfy other aspects of traceability such as the ability to track and link requirements within documents – document traceability.  Traceability has always been this much sought after goal – which many companies have been attempting to address in their tools for many years. Now, the tools exist but most of us are not aware how to leverage them or even that the capability exists.

Over the following few weeks, I am thinking of putting up a series of blog posts focusing on Traceability to think through for myself and others who are interested in Traceability – how to leverage tools – with particular focus on VSTS since I am most familiar with it. A recent solution we implemented for a customer involved integrating VSTS with their CRM solution so that the solution was now truly end-to-end : Right from the time the requirement came in from the customer. In a future post, I hope to outline the lessons we learnt from this particular implementation and also step back to examine the broader implications and advantages of implementing such integrations.

We are also discussing the possibility of perhaps even having a series of webinars that focus on various aspects of Traceability based on our experience over many projects over the years and  how we can leverage VSTS for the purpose.

Watch this space.

David had this to say (quite disdainfully ) about my theorizing about getting data out of Jazz into MS Excel and MS Project:

Instead of a one-way data transfer from Jazz to Microsoft documents, I think a much more useful exercise would be to extend MS Office to make it Jazz aware. This would allow the Office applications to behave as Jazz clients much like Microsoft and others have done for VSTS. Without Jazz awareness, I feel that these other applications would be limited to reporting functions and be of little use to the project management community.

I see this more as an exercise in extending MS Office (or your application of choice) rather than extending Jazz and can understand why IBM would be looking for partners to provide such a solution for their customers.

Point is good. Point taken. I am doing precisely what Negin was complaining about - missing the project management perspective and looking at it from the developer’s perspective - but I did fool myself for a bit there that I had the answer signed, sealed and delivered.

The project manager would rather work directly from inside MS Excel, MS Project and interfaces she is familiar with - and it is far fetched to expect non-developer stake-holders to log into a Jazz client/web client and export into Project or report into Excel.

Also, Work Item information (for example) could be changing frequently and direct integration into MS Excel and MS Project provides an opportunity to refresh statuses and other work item information directly from inside a report that the project manager has open compared to having to export repeatedly from inside a Jazz client each time.

Not to fear. As with the Jazz-based solution I outlined yesterday, I have a similarly vague outline of how to do this - atleast for Excel:

As mentioned in my previous post, we already have a way to get the data off the Jazz server.

Microsoft has made it equally easy to develop applications that target the Office ecosystem through VSTO (Visual Studio Tools for Office - currently VSTO 2005). This means that we have application level add-in support for Office system applications like MS Excel.

Through VSTO, I can write a plug-in for Excel that puts a Jazz toolbar/ribbon in Excel and interfaces directly with the service interface of the Jazz Server to get the raw data required off Jazz. This VSTO plugin would then take care of the business logic of putting together and rendering different reports and charts based on pre-defined queries by leveraging the runtime support in MS Excel that VSTO allows.

Reverting to the developer perspective for a bit, VSTO is installed as an add-on on Visual Studio and provides very nice design-time support - just like Win-Forms controls. Makes it that much easier to develop these plug-ins.

Depending on the needs, in case we find that the Jazz service interface is too ‘raw’ for consumption, another option would be to have the Excel VSTO plug-in interact with the Jazz server through a custom Jazz service-plugin that takes care of sending the Excel plug-in well filtered and formatted data. This has some obvious advantages but in my opinion, has the decisive disadvantage of mandatorily requiring a Jazz server side plug-in without which the nice MS Excel plugin we developed will not work.

The part I am not sure about is how to implement a similar VSTO plug-in for MS Project. If I remember correctly, VSTO doesn’t offer support for MS Project development. I could be wrong.

Browsing the VSTO site is becoming a pain right now because I am on a train back to Toronto that has wifi access on board - but connectivity is sporadic at best. I’ll check up a little later and update this post. If it does have support, that’s that.
If not, a plugin can be written directly against the MS Project SDK to achieve our end.

[Update] There seem to be a couple of options for developing MS Project plugins:

• The version of VSTO that comes integrated with Visual Studio 2008 (currently in beta) has support for creating MS Project plug-ins.
• I guess you can create a COM add-in for MS Project using the MS Project SDK I mentioned above but that would be something to consider only if I couldn’t use VSTO.

Last week was pretty busy and I can remember only bits and pieces of it- sort of like Memento without the anterograde amnesia. I digress but it’s one of my favorite movies and I recently watched it again - and was looking for a way to bring it up - even if completely irrelevant :). So there!

One of the things I do remember from last week was attending the Jazz sessions on the 23rd and then the BoF session on 25th at CASCON’07

There were quite a few of us Ossies at Cascon (see Kamal’s post) and I think we all agreed that Jazz and Team Concert were a pretty impressive debut effort by IBM in the ALM space - though crippled by the lack of certain features and perhaps by the potentially dangerous mislabeling of a group of disjointed policies as “process” (a word and space held dear at Osellus :)).

While we didn’t get a chance to sit down and discuss our observations after cascon, we all took away strong observations from our respective perspectives. I can get started on what I really liked and also the large gaps I see in these offerings - but that’s not the reason I am posting this on a weekend when I could be flipping through channels.

I just read Negin’s post regarding the lack of project mgmt. support in Team Concert and I thought I would quickly respond to it before returning to another re-run.

I know exactly where she is coming from and understand this is a pretty important requirement once you get down to actually using Team Concert/Jazz in a real environment.

However, given that Jazz is built on the Eclipse platform, I see this as a job for an extension (or extensions) built on top of Jazz instead of as an inherent deficiency in Jazz. A couple of us had a chance to talk to Kartik Kanakasabesan, the product manager on Jazz at their booth at cascon and he said as much - it’s something that can be built by the community - possibly by somebody who is more qualified to build such extensions.

Hypothetically, with knowledge of the high-level architecture of Jazz and Eclipse, here are what ingredients I would need to build a MS Project or MS Excel Export concoction:

A way to extend Jazz by adding my own services or wrap Jazz services to get data off the Jazz server:
Jazz is built on the plugin-based architecture of Eclipse and has exposed extension-points both on the client side and the service side (Both of which are Equinox OSGi based). These extension points are in addition to the Jazz services themselves that are exposed as part of the service interface (and also have corresponding siblings in the client library)

So, getting the data off the Jazz server to put into my MS Excel report or into MS Project is taken care off.

Getting the data into MS Project:
Negin’s point regarding getting the Work Items in the “process” heirarchy in JAZZ/Team Concert into MS Project or into any other PM tool for that matter is perhaps *the* most important practical fine-grained feature that will get JAZZ out of just the research community into a real project manager’s hands.

MS Project projects can be defined as XML and Microsoft has a well-defined schema for the XML definition.

So, if we mash up JAZZ’s Work Item related service interface API to get the data out and transform the data to a MS project XML, we are good to go!
I can’t get my jazz.net username/pwd to work for some reason so am vaguely referring above to the “Work Item related service API” - I would have preferred to use the right nomenclature.

Getting the data into MS Excel
Getting the data into Excel is similarly not a problem technically.
A server side plugin that the Jazz team calls a ‘common’ plug-in can be built that provides the generic functionality of being able to export data in the Excel format. This plug-in has to be written in Java of course considering this is the Eclipse platform - therefore it can leverage one of the many libraries openly available for reading/writing to the OLE 2 Compound Document Format which includes the MS Excel (XLS) and MS Word (DOC) formats.

The one I rely on usually for similar purposes is Apache’s POI but the choice might depend on the specifics we want to achieve.

I would then build a Jazz service plug-in that wraps the business logic of generating the specific reports we want to output in the Excel format. This service plug-in would get the data using Jazz’s service interface, mash the data up into different types of reports and then output them into spreadsheets using the generic ‘common’ plugin I mentioned above.

A Jazz client plug-in that provides a way to trigger the service plug-in for generation of specific reports or export into MS-Project would be the last step.

I have used Negin’s specific complaints above to demonstrate a hypothetical and technical answer. However, MS Project and MS Excel are but two formats. Formats such as the Open Document Format (ODF) can be just as easily supported. These formats will become increasingly prominent given the adoption of ODF based tooling such as Open Office by governments, educational institutions and SMEs - especially since these are the same organizations that might end up as primary Jazz customers.

Therefore, building such extensions to provide an interface to the project management tooling world is technically not an issue. The real work will be to intelligently define what reports one would want that can leverage such extensions. Negin and others like her who have invaluable years of project management experience are best suited to answer that question!

Notice to Jazz developers: Feel free to tell me how you would do it - or to correct gaps in my understanding.

So Negin….when do you want me to get started on this?

As soon as I hit ‘Publish’ on my previous post yesterday, I came across this very interesting announcement-
BEA Systems and Adobe Systems announced they will be partnering to provide Adobe’s Flex Builder 2 bundled with BEA’s Workshop Studio - this bundle due out later this year. On their part, Adobe will distribute BEA Weblogic Server evaluation instances with their LifeCycle Enterprise Suite (ES) offering - planned for early 2008.

An understanding is emerging that hosted application infrastructure- Web 2.0 or in the enterprise - cannot be considered complete anymore without inherent support for rich client application frameworks. SOA, web services etc. are enabling technologies but the workflow needs to include a sophisticated client that can leverage them - and vice versa.

Adobe’s also doing the right thing if you look at this from the perspective of competition. Microsoft’s Silverlight is already grounded on the .Net platform - With .Net 3.0 and it’s accompanying middleware technology set including WPF, WCF and WF- it now has a powerful foundation to lean on - not to mention a powerful development environment support in Visual Studio.
Adobe has it’s own middleware-like offerings (ColdFusion or JRun anyone?) but they are not nearly as ‘complete’ as the application servers such as Weblogic, IBM’s Websphere, JBoss and a few others. So they need this kind of bundling.

More on this announcement here.

A sense of validation is always nice - especially when you are so far ahead of the curve. I was scanning through my Google Reader Link Blog recently and noticed the number of new applications being implemented in Flex/Flash and Apollo - Adobe’s desktop runtime for RIAs.

Microsoft recently joined the game officially with the launch of version 1 of it’s Silverlight rich interaction delivery platform and while it’s some way behind Flash/Flex as a platform, it’s gaining popularity too - and has a lot of things going for it (a topic for another post).

I want to focus on Adobe preview of a new product last week at the Photoshop World - Photoshop Express, which is an entry-level Photoshop branded product aimed at drawing customers into the world of Photoshop. What’s special about Photoshop Express is that it’s hosted and implemented using Flex (the development environment for Flash).

This is a continuation of Adobe’s recent focus on the hosted model - Adobe had put out a hosted video editing tool, Remix that has been available as part of the Photobucket tool-set for about 6 months. This video tool has other nearly identical avatars - it also exists as Adobe Premiere Express and is provided on Youtube as Youtube Remixer.

This is the best possible advertisement for the Flash platform’s power. As a developer, there is nothing that screams out a platform’s strength than a complex image/video processing software implemented in it! And there are more coming out everyday!

Flash/Flex has been rounding up some well known names in its corner. I am very impressed with the way Adobe has targeted and honed the direction of the Flex/Flash platform after it adopted Macromedia. The AIR platform (formerly known as Apollo) which has gained a lot of steam since it’s launch is another crucial step in that direction.

What direction is that? Flash is no more seen as the delivery tool of the infamous banner ads. It is now perceived as a serious platform for rich and complex application delivery used by the big boys of the internet and increasingly, the enterprise.

Which brings me to the sense of validation I was talking about - Long before Adobe and the others, Osellus was one of the first “serious” product companies to recognize the power and potential of Flash as a platform and employ it to develop a complex visual enterprise-class product. IRIS Process Author is an enterprise-class visual process management product currently being used by very qualified process groups in very large organizations - and its primary authoring interface is implemented in Flash and the IRIS Flash Framework!! We just happened to start 5 years ago and given our early start, we had to develop our own framework for implementing our Flash-based interface. I was involved in designing and guiding Process Author’s interface over the last few years and so felt the need to make sure Osellus’s foresight was recognized - at least in a blog posting :).

Other examples of companies waking up to the potential of Flex/Flash are Google and Salesforce.com which have been adding some Flash-based products and APIs to their catalogs.

Considering our strong enterprise focus as a company, I am especially peaked by the strong uptake of Salesforce.com’s Adobe Flex Toolkit for Apex. Apex (Salesforce AppExchange) is of course Salesforce’s applications marketplace and one of the first implementations of the SaaS model.

On the consumer-internet front, Google made available the Google Talk Gadget a few months ago which is a Flash implementation of Google Talk that can be added onto your Google Personalized Homepage. The gadget can be even be added to a blog or webpage so visitors can chat with you right there in the context of your blog. Best of all, I noticed that it has the “Call” button - so you can VoIP with your google talk contacts right there - no need for the desktop client! I chatted up a couple of friends and was pretty impressed with the implementation. The gadget has a lot of other features too - like being able to play youtube videos and recognizing picasa albums.
The reason I highlight the Google Talk gadget in particular is that I was hard pressed to find a difference in the user experience between the gadget and the Google Talk desktop application. Of course this is as much as ode to the Google developers as to the Flash platform itself.

I suspect we will see Google employing Flash more and more in their upcoming “betas”.

Another web app that I am really impressed with is Virtual Ubiquity’s Buzzword. VU is calling it “the first real word processor for the web”. It’s written in Flex 2 for Flash 9 and is extremely sophisticated and rich in features.

These are but a few of the examples out there in the enterprise and web 2.0 arenas. All in all, this trend indicates the exciting albeit late recognition of a trend that Osellus had woken up to sometime ago.

I’m pretty sure we’ll be seeing more and more of Flash/Flex and Silverlight based RIAs delivered either in the web browser or in an Apollo like delivery model in the next year or two. I itch to articulate my p.o.v. on the Flash vs. Ajax pseudo-debate that’s been on for the last couple of years but that’s a whole new post.

I am keen to know which other enterprise level products have Flash based interfaces, APIs or atleast components. If you know of any, let me know! I would also be interested in knowing about early adopters of Flash in the enterprise along with Osellus or even perhaps before us!

We are moving to a new blogging platform! Our software of choice - Wordpress. The blog already looks and works so much better but please bear with us for the next couple of weeks while we smith Osellus blogs to make your reading experience more enjoyable and engaging.