Project Management Success Metrics include timeliness, cost-adherence, scope-adherence, process-adherence and quality-adherence. As we have shared in Data warehouse has unique challenges, one will need to adopt different tactics in managing a BI project. The BI project is not limited to data warehouse/data-mart only. Here are the key stages in a data-warehouse initiative:
- Platform identification and evaluation phase
- Data Warehouse business requirement phase
- Data Warehouse modeling phase
- Data Warehouse Design phase
- Data Warehouse Development and configuration phase
- Data Warehouse Testing
- Data Warehouse implementation
- Data Warehouse post-implementation phase
- End-User Tools & application business requirement phase
- EU tools & application modeling phase
- EU tools & application design phase
- EU tools & application development
- EU tools & application testing
- EU tools & application implementation
- EU tools & application post-implementation
Any BI initiative can include following scenarios:
Implementation of plumbing elements (data- warehouse, OLAP, ETL and Metadata) and few end-user applications.
Whenever you implement a data-warehouse for the first time, you would also have first set of end user tools along with it. This is equivalent to laying down the plumbing along with the first set of basins and bath-tubs. Typically, the first set of EU tools includes enterprise reporting and analytics tools.
Projects related to enhancements of BI plumbing elements
This is something, which quite often (though in ideal world it may not be) happens. Due to ever changing information needs and their application (ranging from reporting to business modeling), the plumbing elements modeling and design keeps on changing. If we need to have the information management related to a new set of data elements (means laying down additional water pipes), the entire chain of BI may be impacted. This will transcend across source system mapping, ETL, data-warehouse modeling, olap modeling and metadata updation.
Implementation of new BI end-user tools
Mostly the initial implementation of a data-warehouse is accompanied with one or two end-user tools. As the usage goes wider and deeper, more diverse applications come into play. As one installs new end user tools over the BI platform, it may or may not need any change in the plumbing pieces. For example, if you are installing a data-mining tool, it may just pick-up the data from the DW ‘as is’ and re-models, derives and filters it within the data-mining tool itself. Alternatively, it could mean changes in the plumbing pieces, if you want to have different pre-processing of the data before data-mining tool picks it up. Many of the end user tools have their own data-bases and offline cubes, which becomes an additional layer of data source.
Enhancements of BI applicationa & end-user tools
The end-user tools undergo much more frequent changes, compared to the core components. This is just like changing your taps more often than changing the under-ground pipes. These are mostly small initiatives with few large ones. Small initiatives could include setting-up new Data Mining algorithm and applying specific business theme. For example, doing customer affinity analysis through association. example of a large initiative could mean implementing integrated set of performance dashboards & Scorecards at enterprise level.
Most of the project management metrics are generic in nature. However, we will provide specific examples related to BI to help the readers:
BI Project management Success Metrics
Timeliness of BI project:
The timeliness of the project involves:
- Phase-wise timeliness
- Overall timeliness
Here are some of the tricks one can use to ensure the timeliness:
- A faster decision making process: Lots of decisions are made during scoping, prioritization, requirements and design. A steering committee along with a good decisioning articulation helps. Many a times, the time is lost due to the stakeholders not being able to understand the decision they need to take, the options they have and the impact of their decision. Refer Analyze well, but be decisive.
- Sample testing of the routines: Unless one has implemented a similar routine with a similar platform, it’s a good idea to do an end to end testing of one cube, before you work on the entire data-warehouse. We have already mentioned that end-to-end big-bang enterprise data warehouse is an in-feasible concept. Typically you would test run a cube (or a sizeable portion of it), to understand the unknowns, actual skills of the team etc. before you do a more realistic estimate.
- Include your vendor upfront: There are more than 90% chance that before you start the project, you have chosen vendor(s) for the BI platform, as well as an implementation partner. The prudence demands that you take an input from the vendors before committing an overall timeline.
Cost-Adherence:
There are various elements of cost linked to BI. Cost mentioned here is the project related cost. Here are some of the cost elements:
- License Cost
- Hardware cost
- Data Base and OS layer cost
- Application server cost
- Project Management Cost
- Scoping and analysis cost
- Modeling cost
- Design and Development cost
- Testing and implementation cost
- Employees Charged time
- Disaster Recovery
- Others
The cost management is an important piece as the comprehensive BI initiatives for large organization can be quite costly, with license and tools cost being only a smaller piece. One need not invent a new cost management structure for a BI initiative, but can pretty much ride on the existing one in your organizations (assuming that it is robust). Few tricks on cost-management (excluding the typical commercial aspects):
- Consider using open source for non-core components: Depending upon the size of your organization, one can consider using open source for few peripheral applications like performance manager, reporting. However, one will need to be extra diligent in terms of the support model and capabilities.
- Avoid user-base license for viewers: A BI platform, over the time can get accessed by virtually everyone in an organization. Starting from a customer service executive, who can access a single customer view for his up sell, the usage can go upto the CEO who wants to view the daily enterprise dashboard. For viewer licenses, one should go for enterprise and negotiate hard or have a flexibility to shift from the user based to enterprise license, whenever user based licensing exceeds the enterprise license cost.
- Be prepared before the consultants come in: With due respect to all, organizations end up paying more money to the consultants, as the clients are not ready to fulfill their own responsibilities in the engagement. Before one calls in the full team of analysts, modelers and project managers from a consulting company to land-up in your premises, one will need to prepare. This preparation includes building a medium level of awareness to the core team established for the project. One should also demand from the implementation consultants to provide the list of all pre-work and preparedness one needs to do, prior to the full-scale engagement.
- Focus cost on the core: We always recommend that if you have to compromise on few things to meet cost targets, do that at the level of EU tools. In other words, don't spare pennies on the underground pipes, but do buy cheaper/lesser taps, as they can be upgraded/ added later.
Scope-adherence:
Scope adherence is both ways. Firstly being able to deliver what was scoped out before and secondly not to have a scope creep at the later stage. Scope for a BI project will not be a singular statement. There are two kinds of scopes- The business defined scope, and the 'internal' scope. Here is the difference:
- The business defined scope is purely in the business language in terms of the business themes (refer scoping of Data warehouse)
- The 'internal’ scope as defined after the modeling: Post Data-Warehouse/end-user business requirements & modeling, one can break-down the scope into multiple parts:
- All dimensions, measures to be included,
- All cubes to be created
- All summarizations to be done
- All business configurations to be done
- All historical snap-shots to be done
- All foundation dimensions and measures to be done
It’s extremely important to recognize the second 'internal' part of the scope. It is possible to meet the 'business scope' while taking short-cuts on modeling, which can make it painful at later stage. In other words, you can have two different modeling and design scope with both meeting immediate business scope, but can be very different in flexibility and extensibility over the longer term. To further simplify the statement, you can deliver water in the taps while having faulty pipe-layout; however it will significantly impact your capabilities, when you add more water connections in your house. You can also refer don’t rely only on the business requirements for your DW scope
Few tricks on managing the scope:
- Do not compromise on the scope of the plumbing pieces: Just like cost, one should avoid compromising on the scope related to the core BI components. Do not take short-cuts. Even if you take it, let it be conscious decision by the stakeholders.
- Do not make too large a bite in the first few initiatives: Do not make a grand scope. With an assurance that your DW set-up can easily accommodate the changes, one should phase out the BI project in terms of the scope. Start with few data-marts or with few foundation dimensions. For example, one may go for only customer related data marts and focus on only customer and product foundation dimension to start with.
- Reduce scope on EU tools: Few business users may not like what we say here. If it comes to cutting down the scope, reduce it on the EU tools. For example, As long as the data is made available in the end-user tools, one can reduce the number of reports you will create in the first phase of your enterprise reporting implementation.
- Apply Pareto principle: Diligently look at the impact of the scope on the effort. Sometimes reducing few scope elements can save significant costs. You may be spending 30% of the effort in doing complex ETLs from an unstable and low-quality system. One can find smarter ways to define the scope around time-guzzling areas, so that it meets the 'minimum needs' for the business while saving a lot of effort. For example, one may be taking out data from a 'loose' training system to analyze training-vs.-sales productivity. One may agree with business that one can keep the training analysis at the sales unit level instead of sales person level, which may still serve the business purpose, but take away lot of implementation effort.
Process-Adherence:
This is essentially 'how' element of what you achieve in a BI initiative. Many process quality short-falls may have significant long-term impacts. Sometimes these short-cuts are quite tempting. For example, making a data-mart specific customer dimension may take 50% lesser time compared to investing into a foundation customer dimension (as standard dimension structure common to all cubes). The best way to measure the process quality is to annex the process quality assessment checklists along with the phase completion. You should ideally have the process & quality Guide. This should include the key guidelines and dos & don'ts.
How does one get the data to measure the performance?
Here are the methods by which you can get the data to create your BI project management performance scorecard:
- Daily and weekly Project status reports: These provide the details which can be consolidated for your performance scorecard. A typical project status report will touch upon timelines, cost and scope.
- Process Adherence checklists: These are the checklists used to assess process-adherence.
- Cost Assessment Reports: These are the expense sheets based on the vendor payments and internal effort charge-out
- Steering committee minutes: They provide a view on the management judgment on the project management
The point to note is that your BI scorecard will not be for a given initiative, but a consolidated view of performance across multiple initiatives in progress at that point of time. |
