Human Resource utilization:
This includes two groups of people
- The people involved in BI management from IT perspective. This to include both Vendor and employee staff
- The people using BI: This should support the BI objective of reducing the number of people involved in MIS/Reports generation. Secondly, it’s also a question of how well distributed is the power of BI. For example, ideally, there should be very few people involved in generating reports. The BI should give the power to the end-users to access the information by themselves. However, a BI platform will need few super users and designers who will decide and design on the enhancements in BI, including new reports and views.
Infrastructure Resource Utilization:
This pertains to the whole set of components including:
- Hardware
- OS-Database platforms
- Application servers
- Networks
- Special Desk-tops...
Infrastructure resource utilization is mainly to do with BI consuming resources in line with the expected capacity planning assumptions. You may like to refer infrastructure for BI. One needs to bring infrastructure utilization issues as an exception, only if there is a significant deviation
The speed of the batch processing windows: This one is important, as sometimes the sheer cycle time of ETL and population of OLAP and down-stream end-user tools will take inordinate amount of time. We have seen many organizations, where it takes 7-8 hours to a refresh.
Cost of operation:
This will be a combination of the following (with some overlap)
- Human Resource Cost
- Infrastructure cost
- Change management and production support cost
- Data Centre operations cost
Security conformance:
Security is critical as BI platforms can ideally contain all possible information within the organization. The security will need to work at level of:
- User profiles
- Defined Users
- Defined Entities
- Defined Schemas
- Defined instances within a schema...
The performance assessment on the security piece will be judged on the following counts:
- The security configuration is robust
- The has not been any security breach
- The process related to post-facto security reports audit is being followed
Timeliness of Operations:
The timeliness is related to the adherence to the processing windows by the BI platform. For example- it is expected for the entire end-to-end refresh of data through all the layers of BI environment, starting from the staging area upto the BI applications database, to be completed within the defined windows.
Production Support Quality:
This aspect talks about the adherence of the production support machinery to the agreed SLAs. This should also include the trend analysis on the number of tickets raised, and the quality of root-cause analysis on the production incidents.
Response-time quality:
The expected response time to broadly acceptable category of queries, should be within acceptable range.
Availability:
This is an almost direct derivation of the end-of-day processing windows. Smaller are these windows, higher is the availability. There are now more innovative methods applied by the vendors, whereby the data access is still available for the users (for the previous day), while the processing is going on. So over few years, this should be a non-factor in terms of performance assessment.
The source data to create the scorecard around operational performance
The data gathered is taken from the production support reports, SLA reports, uptime reports etc...
|
