Typically the BI environments in an organization grow through an organic path. You will have many stand-alone (BI silo) BI and data-mart environments with their own independent BI components like ETL, staging areas, loaded data-mart area, OLAP, and end user tools. There can be two methods in which you evolve into an enterprise integrated BI + data warehouse solution, where you have all the above mentioned components fully integrated (single Data Warehouse, OLAP, Enterprise reporting environment etc..):

• A big-bang integrated BI initiative- Whereby you redefine the integration, with a new enterprise level design across the board. Most of the discussions are generally centered on this approach.
• A gradual approach - You selectively integrated some components of the BI environments, as you move towards true enterprise platform.

The challenge with Big-bang approach is the sheer change management effort for the enterprise. This change management leads to denial and resistance. Business and IT owners do not want to rock the boat, once their silo BI environment is stabilized. Giving up functional control is another obvious reason. There is also a risk of organizations lacking the expertise and stamina.

BiPM recommendation for any data management and BI initiative is against a big-bang and for a gradual approach. The oft-stated reasons are:

• Easier change management.
• Testing out the business case.
• Gaining expertise, establishing show-cases and gaining confidence: The expertise and skill for BI are generally scarce and they also very organization specific. Therefore it’s advisable to be cautious.
• Early results and therefore getting an easier buy-in from the management.
• A big-bang is always associated with a big hype, which becomes detrimental in case the initiative falling short.

How to go ahead with Gradual integration:

Phase 1- Extraction and Transformation Integration first

More than 50% of the BI environment is the plumbing work, and it can be done in the back-end ‘with extreme care’ by working closely with select business analysts and IT folks. The back-end here is the extraction and transformation.

• Phase 1A- Eliminate redundant Extraction and Transformation Jobs: Let’s say that you have 5 stand-alone BI environments and all of them could be extracting some common data elements.
• Phase 1B- Create common extraction routines: If there are three extraction routines which are extracting customer data from three different systems, one can merge them to be extracting data from the best source system for that data element.
• Phase 1C- Create common Transformation routines: let’s say that you have multiple transformation routines which are transforming extracted location data, in different ways (one generating a 5 character city code and one generating a 3 character city code). In this case you may like to merge the transformation routine into a single one, while still maintaining the differences in the data structures. You may cautiously try some limited opportunities to make some standardization.

The advantage for this level of integration will be:

• Learning on the ETL design of these stand-alone BI environments, which will help in later stages of integration?
• Reduction of the load of ETL on source systems as well as individual BI environments.
• Better data quality as you extract data from the best system for a given data element.

The caution in this exercise should be as much transparency for the end-users and upper-users as possible. Business owners need to be kept in touch and any inevitable impact should be discussed and signed-off by them.

Phase 2- Tool Integration

You can integrate the front end tools, like enterprise reporting, performance reporting, analytics etc

A business generally welcomes a new tool which gives them a greater capability and functionality. The front-end tools integration will mean that you do away with different enterprise reporting tools or/and environments and replace them with an enterprise reporting tool. In this integration, you can have different phases:

• Migrating the reports, views and object repositories (like report templates, common functions...) to the new platform as is.
• Rationalize the reports, and objects to eliminate nearly-duplicate items.

The advantage of this phase:

• Users get used to a common set of tools and front-ends.
• Cost-reduction of maintaining multiple environments.

Phase 3- Integration of data Marts

This is the third and final stage, whereby you come out with the enterprise dimensional model would be standardized and a comprehensive data warehouse and OLAP environment will be planned. With phase 1 and phase 2 being done, the task will be relatively simpler. Within this step also, you may think of gradual integration, whereby you integrate the data marts which are more logically grouped.