We’ve written in previous posts about our technology and its benefits in a general sense. Today we’d like to share some more concrete details, with a case study about how we used MIOvantage for complex, large-scale data quality analysis at Deutsche Telekom.
Our customer
With [MIOvantage], we are able to monitor and control the data quality of Deutsche Telekom GmbH.-Marco Glass, Manager of Data Quality, Deutsche Telekom
Deutsche Telekom (DT) is the leading German telecom, headquartered in Bonn, Germany. They provide services to more than 151 million mobile customers, and have more than 38 million fixed-network lines and more than 15 million broadband lines.
According to Forbes, Deutsche Telekom is #70 on the Global 2000 list, with more than 227,000 employees and $83 billion in sales.
Scenario
As Deutsche Telekom grew, it evolved a very decentralized approach to IT. Each department had full responsibility for its own data systems, focused on its own niche and customers. Naturally, this led to different departments developing different operational practices.
However, corporate-level documentation often didn’t reflect the actual day-to-day IT operations of individual departments. And not all department practices were aligned with Deutsche Telekom’s overall understanding of its data and processes.
In 2007, Deutsche Telekom introduced an ambitious Master Data Management (MDM) plan that would allow it to maximize the value of its data. Responsibility for data quality was given to a central data quality department, and new data governance, data quality, and quality assurance protocols were introduced.
Deutsche Telekom planned to use the knowledge gathered from the MDM efforts to make extensive improvements to its information and data management systems. The improved infrastructure would maximize the automation of DT’s business processes while minimizing the disruption to the customer experience.
The data quality initiative was a key component of this effort.
Challenges
There were a variety of challenges to performing data quality on Deutsche Telekom’s department systems. We’ll discuss these challenges through the lens of the revenue assurance system, which encountered issues typical of those throughout DT.
A revenue assurance system is meant to identify billing problems so that customers are not under- or over-billed. Underbilling results in missing revenue, while overbilling reduces customer satisfaction and can, for some enterprises, have legal implications.
Major challenges to the revenue assurance system included frequent changes, complex systems, lost relationships, and massive data.
Frequent changes
Although Deutsche Telekom’s decentralized infrastructure served its individual departments well, it was a significant barrier to company-wide data quality. DT’s departments experienced architectural changes to their systems relatively frequently, and some of these changes were significant enough to affect the data quality effort.
On top of that, each department’s highest priority was its business operations and customers, not data quality. As a result, communication between individual departments and the data quality department was sometimes lacking or belated.
To mitigate the combined effects of frequent infrastructure changes and non-optimal communication, the data quality solution would have to be extremely adaptable.
Complex systems
Deutsche Telekom’s department systems were frequently made up of multiple subsystems, complicating a central data quality effort: even within a single department, the data flowed asynchronously through multiple paths, making it more difficult to find the source of an error.
For example, the revenue assurance system’s data originated from a single CRM subsystem, then traveled through one or more of five other subsystems.
Missing and lost relationships
A single customer’s information was often spread across several departments, and because those departments used different systems, the data often wasn’t directly comparable. This disparate data needed to be reconciled so that a single customer’s information was consistent and complete throughout the company.
Additionally, production systems couldn’t support the intensive analytics needed for data quality alongside normal operations. Data had to be migrated to an analysis-specific system.
Because of the large number of disparate source systems, the relationships between customers and contracts were frequently lost. These missing relationships, however, were essential to performing data quality analysis: assessing the correctness of a customer’s bill without knowing what contracts they have is impossible.
The data quality solution would need to identify and build or rebuild missing and lost relationships before proceeding with the analysis.
Massive data
Deutsche Telekom’s departmental systems had amassed enormous amounts of data: the data quality solution would need to handle 11 billion records.
MIOsoft
For us it was important to buy a solution that on the one hand is able to represent the complexity of the IT-processes and that on the other hand can process the big amount of data performantly and this on a linear scale.Another important factor was the offer of a solution out of one hand inclusively a stable production service.
- Marco Glass, Manager of Data Quality, Deutsche Telekom
Deutsche Telekom faced significant challenges to executing its MDM plan. But MIOvantage could fully describe the complexity of Deutsche Telekom’s IT processes. Its data processing handled large amounts of data with high performance. And it scaled near-linearly.
This complete data quality environment and package of integrated features led Deutsche Telekom to choose MIOvantage for its data quality solution.
Solution
We used MIOvantage to implement the data quality solution that Deutsche Telekom needed.
Step #1:
We created a context model for each of Deutsche Telekom’s production systems. This model represented the real-world relationships between the different entities that each system had data about.
Below is a simplified version of the revenue assurance context model.
Step #2:
We created an adapter that directed the data each department produced into the context model. Using an adapter means that MIOvantage doesn’t need any time or resources for batch preprocessing; data is transformed in flight, and the original source files are untouched.
Step #3:
We loaded the data from the source files via adapters. The context model identified pieces of data that appeared to be about the same entity and pooled the data about each entity together.
Step #4:
Each data pool created in step #3 was synthesized using rules belonging to the context model. The context model judged which data most reliably described each entity and committed that data to the main database. A copy of all the loaded data was retained in the original data pools.
Then, the context model identified and created relationships between different entities. For the revenue assurance system, this included the customers, bills, contracts, and products.
Frequent changes
MIOvantage’s tools for each step of the data quality process are decoupled, so that changes to one don’t automatically require changes in another. For example, an adapter to handle a new subsystem’s data can be added to MIOvantage without changing the context model or its synthesis rules.
The data pool also allowed Deutsche Telekom additional flexibility in choosing its business rules and targets: because data was permanently retained in MIOvantage’s data pool, DT could redefine aspects of its data quality rules without having to reload data afterwards.
MIOvantage’s tool decoupling and data pools helped Deutsche Telekom adapt its data quality efforts to account for the frequently-changing IT infrastructure and non-optimal interdepartmental communication.
Complex systems
Deutsche Telekom was able to retain its existing IT systems without compromising its data quality initiative. MIOvantage performed data quality within DT’s existing IT infrastructure using a series of checkpoints and snapshots to compare each system and subsystem’s actual data with its expected data.
Reconstructing relationships
MIOvantage’s synthesis technology allows data about a single customer to be consolidated from multiple systems, forming a complete picture of that customer’s information.
When information about relationships was lost, MIOvantage’s tools could be used to identify and recreate those relationships.
Massive data
MIOvantage scales linearly on low-cost commodity hardware, even for massive data undertakings. And if an application needs to be rescaled for any reason, the architecture underpinning MIOvantage’s operation can re-scale itself automatically.
Results
Our Goal - to realize cost reduction and hence to pay in for the purpose of higher customer satisfaction - was implemented.- Marco Glass, Manager of Data Qualitidy, Deutsche Telekom
In the revenue assurance system alone, MIOvantage revealed $50 million/year in missing revenue, resulting from underbilling in 4% of billed contracts. Deutsche Telekom was able to use the results from the data quality analysis to recover that revenue going forward.
MIOvantage monitored Deutsche Telekom’s data quality centrally, enforcing data quality across all of its IT systems. Data quality errors were detected before they could impact the customer, resulting in measurably increased customer satisfaction. Improved data quality also made it possible for Deutsche Telekom to automate many processes, resulting in significant cost savings.
After the MDM data quality project was completed, Deutsche Telekom and its subsidiaries continue to rely on MIOsoft technology to deliver mission-critical services in other data-centric initiatives.
For Data Migration