3 Ways Improving Data Catalogs Help to Accelerate Digital Transformation

 Is your organization looking to accelerate digital transformation and become more data-driven? While many tools and practices are required to transform businesses, experienced leaders seek digital transformation force multipliers and select data catalogs to improve data quality, grow analytics capabilities, and help instrument change across the organization.

Almost all organizations pursuing digital transformation strategies have top priorities to improve customer experiences and enable data-driven capabilities. Most business leaders recognize that the products and services they offer today need to progress considerably over the next months, quarters, and years based on changing customer needs and business opportunities.

Data Catalogs Accelerate Digital Transformation - Isaac Sacolick

The drive to experiment and evolve customer experiences requires people across the organization to use data, analytics, and machine learning capabilities. The most progressive C-level executives, including CEO, CIO, CMO, chief digital officers, and chief data officers, seek to empower more people with access to data, analytics, and self-service data technologies.

These leaders recognize the need to challenge the status quo and evolve new ways to serve customers better. It’s critical for people in all roles – from sales, marketing, operations, finance, technology, and human resources – to see patterns, ask questions, and experiment with new models.

So, what technologies should CIO, CDO, IT, and data leaders promote to support the data-driven organization?

Should CIO and IT leaders look to centralize data in Snowflake or other data warehouses and data lakes? Will providing access to data using self-service data visualization tools like Tableau, Microsoft Power BI, and others help people across the organization make better, faster, and smarter decisions with data?

The simple answer is, “Yes, but.”

Centralizing, providing access, and enabling data discovery are just three legs of the stool. The problem is that people need to identify data sources useful to their quests, understand the context behind the data, and connect with data stewards that are the subject matter experts.

Data catalog and data prep tools are the fourth leg of the stool, and the collaboration they enable can be an accelerating force multiplier in digital transformations.

Let’s review three use cases on how data catalogs can accelerate digital transformations.

1. Simplify Finding the Right Data for the Analysis

Here’s a common question raised by analysts and data scientists. As they review data sources, they ask, “Which date and financial measures should I use in my analytics, data visualization, or machine learning model?”

CRMs, ERPs, and other SaaS tools store dozens of dates and currency fields. For example, a sales pipeline might store dates for the pipeline’s creation and start, and the analyst wants to know which one to use for calculating the sales cycle.  In a second example, the ERP likely has many fields for capturing customer revenue, and the analyst wants to determine how to aggregate them and calculate a customer lifetime value metric.

Even in the best cases where the database or SaaS has metadata or documentation providing definitions on dimensions, measures, and dates, chances are, only IT and expert users of each system know where to find this information.

Now add to this complexity when businesses have multiple CRMs, ERPs, marketing platforms, and other SaaS tools replicating data in and out of them. How likely is it that analysts select the correct fields, and how much time do they waste seeking experts to learn how to leverage different data sources? 

And it’s not just financial and sales data that are important to organizations. Leaders deploy data catalogs and prep to improve enterprise data quality, analyze supply chains, and improve patient care in hospitals.

Data catalogs centralize a listing of loaded data sources, provide entry points for people to request access to data, and offer tools for maintaining data dictionaries. These dictionaries include metadata loaded from source systems, supplemental information provided by subject matter experts, and data catalog usage analytics.

The analyst using this data catalog is more likely to find the right source and data fields, and top data catalogs offer natural language processing interfaces for querying them. If the analyst has questions, then the data catalog helps them find, collaborate, and ask questions of subject matter experts. 

Using the data dictionary establishes a feedback loop as increased usage improves its accuracy and utility. It creates a force multiplier because citizen analysts have the tools to find sources, accurate data, and subject matter experts.

2. Deploy New Apps With Supporting Analytics

A big success factor in digital transformation is to develop minimal viable products, release frequent app improvements to end-users, capture their feedback, and then realign priorities and requirements. But, developers working on apps have the same challenge as analysts, and they need to know which existing data sources and fields to tap into with their apps.

Most apps also create new data sources with forms, images, audio, IoT data streams, and other form factors. App usage and alert information may be standardized or have app-specific observability data.

So, the key question is, how can analysts tap into these data sources and ensure development teams receive regular feedback and insights on usage, performance, security, and other metrics?

Development teams that include updating the data catalog during their app release management process helps to maintain documentation and inform analysts. It’s a data governance force multiplier because it enables an agile collaboration between developers and analysts and promotes creating data-driven feedback loops when DevOps teams release new app versions and features.

3. Decide When to Source New Data and When to Consolidate

The first two examples focus on consuming and producing internal data sources. The third force multiplier comes from how organizations identify, procure, utilize, support, and manage third-party data sources.

Analysts often integrate and use third-party data sources to enrich internal data on companies, people, and products. In addition, weather, economic, government, and other contextual information are vital to load into analytical and machine learning models to identify correlations and causalities.

Procuring a data source is only one step in the sourcing lifecycle. We also want analysts to find and review existing data sources before seeking new ones.

Support practices to monitor usage patterns and data source changes are also needed. For example, many organizations end up with duplicate or near-identical data sources, and consolidating the usage to one primary source can often yield cost and quality benefits. In addition, when third-party data sources are underutilized, there may be a business rationale to unsubscribe and reduce costs. 

Data catalogs can be even more beneficial when procuring new data sources, and the data governance team establishes a selection process. With a data catalog in place, they can easily collaborate with analysts and other data consumers to identify requirements, pilot analytics, and select winning solutions. Once selected, they can update the data catalog and enable more consumers to leverage sources in their analytics.

The force multiplier is that the catalog encourages reusing third-party data sources and reducing underutilized ones. The process creates another feedback loop where the most important third-party data sources are institutionalized.

To summarize, a key to digital transformations is running experiments, capturing customer feedback, and leveraging data to realign programs. Enterprises using a data catalog have a game-changing tool and practice that enables more analysts, subject matter experts, developers, and decision-makers to collaborate with higher quality data and analytics.

This post is brought to you by Boomi

The views and opinions expressed herein are those of the author and do not necessarily represent the views and opinions of Boomi.

No comments:

Post a Comment

Comments on this blog are moderated and we do not accept comments that have links to other websites.


About Isaac Sacolick

Isaac Sacolick is President of StarCIO, a technology leadership company that guides organizations on building digital transformation core competencies. He is the author of Digital Trailblazer and the Amazon bestseller Driving Digital and speaks about agile planning, devops, data science, product management, and other digital transformation best practices. Sacolick is a recognized top social CIO, a digital transformation influencer, and has over 900 articles published at InfoWorld, CIO.com, his blog Social, Agile, and Transformation, and other sites. You can find him sharing new insights @NYIke on Twitter, his Driving Digital Standup YouTube channel, or during the Coffee with Digital Trailblazers.