Yet, that's exactly what we're doing as technologists in discussing volume, velocity, and variety of our Big Data initiatives. Do we have to measure Big Data initiatives by profit, revenue, or customer impact?
The answer is yes. Eventually. Ecommerce companies and others that can measure response directly based on clicks may have the easiest, direct approach to measuring impacts. But for others, it may not be so simple and it might take some experimentation to yield the proper metrics.
Developing Metrics on Analytics DeliveredSo a good start might be to define metrics that help organizations determine how effective their big data initiatives are in performing analysis and delivering results. Here is one approach to get started:
- Develop a Big Data Analysis Catalog - This catalog, would define some of the recurring analysis performed by someone (possibly a data scientist) in the organization. Below is a simple catalog.
- Document the type of Analysis Performed - For each Data Analysis, develop some light weight documentation on the type of the analysis performed. You can do this in an unstructured way with a document or wiki page, or more structured by defining a child table in the Analysis Catalog. In the pipeline example above, this could list out specific reports and how the data is analyzed for trends or anomalies.
- Log your activity and results - Put the "science" discipline into the role of the data scientists and require them to maintain a lab book. For each analysis performed whether it's reviewing a report or dashboard or performing an entire study, make sure there is a log of this activity and at least a qualification of the results.
- Value and trend the results - At the time of the analysis activity, it might be difficult to put a quantified, ideally monetary value on the results. So I would suggest a quarterly review of the analysis log to determine what activities provided value and to attempt to put a dollar value against them.