Friday, August 22, 2008

Incremental Release Always Advisable? Not so fast!

The benefits of the Agile practice of incrementally releasing software are well established. However, there are certain situations where it may not be the best fit for your client. This post considers one such scenario.

In our example, the client is replacing an existing old system, which is currently being used to run the client's business. The client is launching a new project to adapt a new software package to address many of the shortcomings of the current system. However, the package will require a non trivial amount of customization. At this point, there are two options:
1- Implement all the customizations necessary to the new package, migrate the data, then deploy the new package into production, and start using it instead of the old application.
2- Implement a minimal set of of customization to the new package, and release that into production. Use the new customized functionality in the new system, while completing the business process in the old system. Once the functionality has stabilized, customize the next step in the business process, deploy it to production, and so on.

The benefits of the second approach are well established, and there is no need to reiterate them. I'd like to concentrate on some of the challenges of this option.

To aid the discussion, assume that the business process for the client includes the general steps A, B, and C. And assume that we start by customizing the new package to fulfill the needs of step A in the process, continuing B and C in the old system. To be able to do this, we have to perform the following:
  • Transfer the data captured in the new system to the old system, after completing step A.
  • Figure out how to trigger the data transfer from the new system to the old: at which event should the data be transferred to the old system?
  • Decide what to do if the data may be changed in the new system after it has been transferred to the old system: Should we push the data to the old system every time it changes? If not, at what events should this happen? Is it acceptable to prevent further data changes in the new system after it has been transferred?
  • Work out the changes to the reporting needs of the client: Which system is able to provide which pieces of the data?
  • Figure out the implications of having the data unavailable in the old system until it has been transferred. For example, a data entry to capture the requirements of step A in the process might be captured in the new system, but the data will not be available in the old system until it has been transferred. Will this have any implications on the business?
  • Work out the changes to the client's business process. For example, is there double data entry required to maintain the consistency of the configuration data? How and when should users use the new system as opposed to the old system?

We also may have to answer the following questions:
  • If there is a data validation step that needs to be performed as part of step A, that requires the knowledge of previous transactions on related data from the old system, how will it be performed?
  • Is there a need to sync data changes from the old system back to the new system?

There are also requirements on data maintenance. The data in the old and the new system must be kept in sync. Otherwise, the syncing operations will fail to map the data to the correct business entities.

It should also be noted that the new system may not be amenable to syncing out of the box, and that certain changes to the way it functions may be necessary to allow the integration to work properly. For example, we may need to introduce a new status to mark whether or not the data has been, or is available to be, transferred to the old system.

Implications of the phased release approach
Given the above, we can see that the second approach has the following implications, as opposed to the first approach:
  • The client can only implement a limited data restructuring or data cleanup in the new system, because this may break, or make very difficult, the process of mapping data between the new and the old system.
  • It also follows that there are certain realizations of the benefits of moving to the new system will be delayed, until all the steps in the process have been successfully moved to the new system.
  • In a similar vein, the business benefits that may be gained, or process improvements that can be attained, by moving to the new system will be delayed, because the new system will be constrained by the need to synchronize data to, and conform to the business process imposed by, the old system.
  • The old system will need to be up and running for a longer period of time, because the time required to build the customizations to the new system, plus coding for the synchronization logic, will require more time than simply implementing the customizations, then go live. I am assuming here that the team size remains the same.
  • A careful consideration of the client's budget is due. If the budget is tight, there may not be enough slack to accommodate the extra effort required to add the integration functionality between the new and the old systems.

And you should also consider:
  • The cost to the business, in terms of delayed ROI, and time to market benefits, resulting from delaying release time until all the necessary functionality is developed in the new system. Your client may not have sufficient incentive to warrant the cost.
  • The risk of deploying all the functionality of the new system in one step. It may be the case that your client has ways of reducing the risk to the business, that makes the risk/reward balance weigh in favor of this option. For example, the client may have acceptable manual processes, or is able to revert to the old system, if part of the functionality in the new system is encountering issues.

There are certain situations where an incremental release to production may not be advisable. Although there is evident value associated with releasing fast, the cost and risks associated with this option may outweigh the benefits. Other approaches may weigh better on this scale. This entry introduced a situation where there are factors at play that should be considered before choosing an approach. Such factors include the client's budget, the benefits of going live faster, the cost of developing the integration code, the cost of delaying the release, and the risk in releasing all the functionality of the new system in one step.