Showing posts with label Unified Data. Show all posts
Showing posts with label Unified Data. Show all posts

Thursday, February 11, 2021

Mastering Fractured Data

Data complexity in companies can be a big obstacle to achieve efficient operations and excellent customer service.

Companies are broken down into various departments. They have hundreds, thousands, or even hundreds of thousands of employees performing various tasks. Adding to the complexity, customer information is stored in so many different applications that wide gaps exist among data sources. Bridging those gaps so every employee in the organization has a consistent view of data is possible and necessary.

Various applications collect customer information in different ways. For example, CRM solutions focus on process management and not on data management.

Consequently, customer data is entered into numerous autonomous systems that were not designed to talk to one another. Client data is housed one way in a sales application, another way in an inventory system, and yet another way in contact center systems.

Other organizational factors further splinter the data, which can vary depending on the products in which a customer is interested, where the product resides, and who (the company or a partner) delivers it.

In addition, information is entered in various ways, including manually, either by the customer or an employee, or via voice recognition. And applications store the information in unique ways. One system might limit the field for customers’ last names to 16 characters while another could allow for 64 characters.

The challenge is further exacerbated by software design and vendors’ focus. CRM vendors concentrate on adding application features and do not spend as much time on data quality.

Customers can input their personal information 10 different ways. Most applications do not check for duplication when new customer information is entered.

Human error creates additional problems. Employees are often quite busy, move frequently and quickly from one task to the next, and, consequently, sometimes do not follow best practices fully.

Data becomes very fractured and there appear different versions of truth. The data features a tremendous amount of duplication, inconsistencies, and inefficiencies.

The inconsistencies exist because fixing such problems is a monumental task, one that requires companies to tackle both technical and organizational issues. Master data management (MDM) solutions, which have been sold for decades, are designed to address the technical issues. They are built to clean up the various inconsistencies, a process dubbed data cleansing.

The work sounds straightforward, but it is time-consuming and excruciatingly complex. The company has to audit all of its applications and determine what is stored where and how it is formatted. In many cases, companies work with terabytes and petabytes of information. Usually, they find many more sources than initially anticipated because cloud and other recent changes enable departments to set up their own data lakes.

Cleansing Process

Cleansing starts with mundane tasks, like identifying and fixing typos. The MDM solution might also identify where necessary information is missing.

To start the process, companies need to normalize fields and field values and develop standard naming conventions.  The data clean-up process can be streamlined in a few ways. If a company chooses only one vendor to supply all of its applications, the chances of data having a more consistent format increase. Typically, vendors use the same formats for all of their solutions. In some cases, they include add-on modules to help customers harmonize their data.

But that is not typically the case. Most companies purchase software from different suppliers, and data cleaning has largely been done in an ad hoc fashion, with companies harmonizing information application by application. Recognizing the need for better integration, suppliers sometimes include MDM links to popular systems, like Salesforce Sales Cloud, Microsoft Dynamics, and Marketo.

Artificial intelligence and machine learning are emerging to help companies grapple with such issues, but the work is still in the very early stages of development.

Still other challenges stem from internal company policies—or a lack thereof—and corporate politics. Businesses need to step back from their traditional departmental views of data and create an enterprise-wide architecture. They must understand data hierarchies and dependencies; develop a data governance policy; ensure that all departments understand and follow that policy; and assign data stewards to promote it.

The relationship between company departments and IT has sometimes been strained. The latter’s objectives to keep infrastructure costs low and to put central policies in place to create data consistency often conflict with the company departments' drivers. And while departments have taken more control over the data, they often lack the technical skills to manage it on their own.

It is a good idea to start with small area and then expand to other areas.

Having clean and organized data would make company's operations much more effective and would enable to optimize customer service. They can take steps to improve their data quality.

Please contact us for more information or for a free consultation.

Tuesday, June 10, 2014

Unified Data Management

In most organizations today, information is managed in isolated silos by independent teams using various tools for data quality, data integration, data governance, metadata and master data management, B2B data exchange, content management, database administration, information life-cycle management, and so on.

In response to this situation, some organizations are adopting Unified Data Management (UDM), a practice that holistically coordinates teams and integrates tools. Other common names for this practice include enterprise data management and enterprise information management.

Regardless of what you call it, the big picture that results from bringing diverse data disciplines together yields several benefits, such as cross-system data standards, cross-tool architectures, cross-team design and development synergies, leveraging data as an organizational asset, and assuring data’s integrity and lineage as it travels across multiple departments and technology platforms.

But unified data management is not purely an exercise in technology. Once data becomes an organizational asset, the ultimate goal of UDM becomes to achieve strategic, data-driven business objectives, such as fully informed operations and business intelligence, plus related goals in governance, compliance, business transformation, and business integration. In fact, the challenge of UDM is to balance its two important goals: uniting multiple data management practices and aligning them with business goals that depend on data for success.

What is UDM? It is the best practice for coordinating diverse data management disciplines, so that data is managed according to enterprise-wide goals that promote efficiency and support strategic, data-oriented business goals. UDM is unification of both technology practices and business management.

For UDM to be considered successful, it should satisfy and balance the following requirements:

1. UDM must coordinate diverse data management areas. This is mostly about coordinating the development efforts of data management teams and enabling greater inter-operability among their participants. UDM may also involve the sharing or unifying of technical infrastructure and data architecture components that are relevant to data management. There are different ways to describe the resulting practice, and users who have achieved UDM call it a holistic, coordinated, collaborative, integrated, or unified practice. The point is that UDM practices must be inherently holistic if you are to improve and leverage data on a broad enterprise scale.

2. UDM must support strategic business objectives. For this to happen, business managers must first know their business goals, then communicate data-oriented requirements to data management professionals and their management. Ideally, the corporate business plan should include requirements and milestones for data management. Hence, although UDM is initially about coordinating data management functions, it should eventually lead to better alignment between data management work and information-driven business goals of the enterprise. When UDM supports strategic business goals, UDM itself becomes strategic.

More Definitions

UDM is largely about best practices from a user’s standpoint. Most UDM work involves collaboration among data management professionals of different specialties (such as data integration, quality, master data, etc.). The collaboration fosters cross-solution data and development standards, inter-operability of multiple data management solutions, and a bigger concept of data and data management architecture.

UDM is not a single type of vendor-supplied tool. Even so, a few leading software vendors are shaping their offerings into UDM platforms. Such a platform consists of a portfolio of multiple tools for multiple data management disciplines, the most common being BI, data quality, data integration, master data management, and data governance.

For the platform to be truly unified, all tools in the portfolio should share a common graphical user interface (GUI) for development and administration, servers should inter-operate in deployment, and all tools should share key development artifacts (such as metadata, master data, data profiles, data models, etc.). Having all these conditions is ideal, but not an absolute requirement.

UDM often starts with pairs of practices. In other words, it’s unlikely that any organization would want or need to coordinate 100% of its data management work via UDM or anything similar. Instead, organizations select combinations of data management practices whose coordination and collaboration will yield appreciable benefits.

The most common combinations are pairs, as with data integration and data quality or data governance and master data management. Over time, an organization may extend the reach of UDM by combining these pairs and adding in secondary, supporting data management disciplines, such as metadata management, data modeling, and data profiling. Hence, the scope of UDM tends to broaden over time into a more comprehensive enterprise practice.

A variety of organizational structures can support UDM. It can be a standalone program or a subset of larger programs for IT centralization and consolidation, IT-to-business alignment, data as an enterprise asset, and various types of business integration and business transformations. UDM can be overseen by a competency center, a data governance committee, a data stewardship program, or some other data-oriented organizational structure.

UDM is often executed purely within the scope of a program for BI and data warehousing (DW), but it may also reach into some or all operational data management disciplines (such as database administration, operational data integration, and enterprise data architecture).

UDM unifies many things. As its name suggests, it unifies disparate data disciplines and their technical solutions. On an organizational level, it also unifies the teams that design and deploy such solutions. The unification may simply involve greater collaboration among technical teams, or it may involve the consolidation of teams, perhaps into a data management competency center.

In terms of deployed solutions, unification means a certain amount of inter-operability among servers, and possibly integration of developer tool GUIs. Technology aside, UDM also forces a certain amount of unification among business people, as they come together to better define strategic business goals and their data requirements. When all goes well, a mature UDM effort unifies both technical and business teams through IT-to-business alignment.

Why Care About UDM?

There are many reasons why organizations need to step up their efforts with UDM:

Technology drivers. From a technology viewpoint, the lack of coordination among data management disciplines leads to redundant staffing and limited developer productivity. Even worse, competing data management solutions can inhibit data’s quality, integrity, consistency, standards, scalability, architecture, and so on. On the upside, UDM fosters greater developer productivity, cross-system data standards, cross-tool architectures, cross-team design and development synergies, and assuring data’s integrity and lineage as it travels across multiple organizations and technology platforms.

Business drivers. From a business viewpoint, data-driven business initiatives (including BI, CRM, regulatory compliance, and business operations) suffer due to low data quality and incomplete information, inconsistent data definitions, non-compliant data, and uncontrolled data usage. UDM helps avoid these problems, plus it enables big picture data-driven business methods such as data governance, data security and privacy, operational excellence, better decision making, and leveraging data as an organizational asset.

To be successful, an organization needs a data strategy that integrates a multitude of sources, case studies, deployment technologies, regulatory and best practices guidelines, and any other operating parameters.

Many organizations have begun to think about the contributions of customer information in a more holistic way as opposed to taking a more fragmented approach. In the past, these efforts were undertaken mainly by organizations traditionally reliant on data for their model (like finance and publishing), but today, as more and more businesses recognize the benefits of a more comprehensive approach, these strategies are gaining much wider deployment.

What should a good strategy look like?

First, identify all data assets according to their background and potential contribution to the enterprise.

Second, outline a set of data use cases that will show how information will support any of a variety of customer marketing functions.

Next, create rules and guidelines for responsible use and access, making sure that the process is flexible and transparent. Keep in mind that not all data should be treated the same way; rather, it should be managed according to its sensitivity and need.

Finally, make sure that this process is ongoing so that tactics can be evaluated and adjusted as needed.

Such a strategy combines the best practices with responsible data governance and smart organization. Everyone wins - the employees who gain quick access to essential information, the enterprise that is running more smoothly; and of course, the customers who are served by a resource-rich organization!

Friday, January 31, 2014

Unified Data Strategy

The amount of data being created, captured, and managed worldwide is increasing at a rate that was inconceivable a few years ago. Data is a collection of discrete units of information but like the stars in the night sky taken together form an organized structure.

Unstructured data comes in many different formats including pictures, videos, audio, PDF files, spreadsheets, documents, email, and many other formats. 

Sometimes unstructured data lives within a database. Sometimes the database acts as an index for the unstructured data. Often the metadata (information about the data) associated with the unstructured data is larger than the data itself. Consider the example of a set of videos. Although the files may be small in size, the information stored regarding the content within a particular video may be very big. Often unstructured data is also called big data.

Certain business functions require analysis of massive amounts of data.

Multiple systems are being utilized to manage different forms of disparate data. Companies need to adopt a comprehensive and holistic approach to managing these many systems and incorporating them into a combined system.

Modern IT systems should be able to ingest, access, store, manipulate and protect data within a wide variety of disparate formats. These multiple data formats may exclude the necessary flexibility, elasticity and alacrity that many modern business functions require. There are situations when data must be accessed so quickly and data management systems should be able to accommodate such situations. Each of these systems recognizes a particular style of data with a fairly well-defined set of attributes and manages that data to satisfy a particular business function.

A Unified Data Strategy (UDS) is a broad concept that describes how massive amounts of data in a multitude of forms can and should be understood and managed. UDS is also a specific individualized methodology developed by each data owner to manage that data in all its forms in a comprehensive but interrelated manner.

By adopting a UDS, data owners will be able to develop comprehensive, customized methodologies to manage their data. By taking into account the interconnected nature of the various sources of data and tailoring the management of that data to the specific business requirements the maximum value can be achieved.

UDS can be used to address the task of comprehensive data management. Cloud computing may provide the solution to this data management and recognition problem. Virtualization, the foundation of cloud computing, is the cornerstone of this strategy. The capabilities and architecture enabled via a virtual/cloud infrastructure can help companies to develop a UDS to address the movement in data management and practice.

Exciting new technologies and methodologies are evolving to address this phenomenon of science and culture creating huge new opportunities. These new technologies are also fundamentally changing the way we look at and use data.

The rush to monetize big data makes various solutions appealing. But companies should perform proper due diligence to fully understand the current state of their data management systems. Companies must learn to recognize the various forms of disparate and seemingly extraneous forms of information as data and develop a plan to manage and utilize all their data assets as a single, more powerful whole.

The transition from traditional relationally-structured data to a UDS could be complicated, but can be navigated effectively with an organized and managed approach to this effort.

To successfully adopt a Unified Data Strategy, companies should focus on the following:

1. Develop a thorough understanding of how the business consumes, produces, manipulates and uses information of all types.

2. Determine how the business can use data to both understand external factors and to assist in making internal decisions, as well as to understand how the data itself is relevant to influencing the business.

3. Analyze the "personality" of each data form so that it can be matched with tools that appropriately acquire, filter, store, safeguard and disperse the data into useful information.

4. Select infrastructure and tools that automate or eliminate traditional high-cost tasks such as import, provisioning, scalability, and disaster tolerance. A highly virtualized infrastructure with complementary tools should provide the majority of these capabilities.

5. Commit to the process of learning as an entirely new approach to technology, and to adopting it in risk-appropriate increments.

Any organization with a significant data infrastructure should be aware of the pitfalls that could occur if a company rushes into acquiring new technologies without understanding their requirements. Thorough analysis will lead to an understanding of the current state of their data management systems, and subsequently to better control of their existing data.

Ultimately, organizations should be able to recognize, manage, and utilize new forms of disparate and seemingly extraneous information as data. Companies, that develop a plan to comprehensively address all their issues around managing and utilizing all useful data, will gain significant strategic advantages.