Entity Resolution
How It Works & Why It Matters
This comprehensive introduction to the topic โWhat is Entity Resolution?โ will demystify the concept of entity resolution โ sometimes referred to as data matching or fuzzy matching. We’ll examine what entity resolution is, why it matters, how it works, as well as its benefits, value and significance in the world of data quality analytics, visualization and decision-making.
The most common entity types that entity resolution systems handle are people and organizations, while products, vessels and other real-world entities can also benefit from entity resolution.
Watch Jeff Jonas answer the question, โWhat is entity resolution?โ
Table of Contents
- What is entity resolution?
- How does relationship detection improve entity resolution?
- Why does entity resolution matter?
- What are the benefits of entity resolution?
- What are the risks of ignoring entity resolution or doing it poorly?
- What are the most common entity resolution use cases?
- How does entity resolution work step by step?
- Why is getting entity resolution right important?
- How should you evaluate entity resolution solutions?
- How do you get started with smarter entity resolution?
What Is Entity Resolution?
Entity resolution is the process of determining when real-world entities are the same, despite differences in how they are described or inconsistencies in how data was entered.
The image to the right shows an example of the type of problem that entity resolution helps to solve. The data for these records has many differences, but a sophisticated Entity Resolution system can determine that it is actually the same entity.
But entity resolution isn’t just beneficial for finding undiscovered matches. Entity resolution also determines when real-world entities are different despite similarities in how they are described. In this example, the third record only has a one-letter difference from the second record – “Jr” versus “Sr”. However, this one letter is significant, and provides evidence that these are different people.
This happens for a variety of reasons, for example juniors and seniors, twins, or families with Patricks and Patricias under the same roof. Entity resolution systems are designed to handle these data matching edge cases.
How Does Relationship Awareness Improve Entity Resolution?
Todayโs state-of-the-art entity resolution solutions use relationships to better improve entity resolution accuracy. For example, Figure 3 below shows a derived relationship identified between Entity 1 (“Jr”) and Entity 2 (“Sr”). Note that not all entity resolution systems would be capable of recognizing this relationship.
As relationships are detected between the resolved entities, a network of connected entities is developed. This “entity-resolved graph” helps inform future entity resolution decisions. For example, George Foreman (the famous boxer) named all of his boys George. When entity resolution becomes aware of this, matching new George Foreman records can be done with more care.
Why Does Entity Resolution Matter?
Entity resolution is crucial because it solves many of today’s pressing data quality and analytics problems. Utilizing modern entity resolution, organizations can now accurately identify and link entities within and across multiple data sources, despite incomplete and inconsistent data. Entity resolution can also detect relationships between resolved entities. This results in better customer service, more effective fraud detection, greater efficiency, and leaps in competitive advantage.
Entity resolution tackles pivotal business questions, such as:
โข Is this new customer application simply the same bad guy we got rid of yesterday?
โข Is our supplier actually owned by a globally recognized and sanctioned entity?
โข Are these three customers each with one transaction, or is this one customer with three transactions?
What are the Benefits of Entity Resolution?
Entity resolution is crucial in enhancing the quality and utility of data related to individuals and organizations. By employing techniques like fuzzy matching, data matching and Entity Centric Learningโข, entity resolution offers numerous benefits for organizations in todayโs data-driven world.
โข Enhances Data Quality and Accuracy
โข Creates 360-degree Views of Customers and Other Entities
โข Improves Decision-Making and Builds Trust in Data
โข Bolsters Analytics for Discovery and Insight
โข Enriches Graph Analytics and Visualization
The value and benefits of entity resolution are multifaceted and substantial. From enhancing data quality to improving decision-making and analytics, entity resolution is a key enabler in harnessing the full potential of data in the modern business landscape.
What are the Risks of Ignoring Entity Resolution Or Doing it Poorly?
Ignoring or not properly implementing entity resolution can lead to significant consequences for organizations. Here’s an exploration of some of the potential pitfalls and dangers:
- Negative Customer Experiences and Outcomes: Inaccurate entity resolution can negatively impact customer satisfaction. For example, a loyal customer might be treated as a new customer due to mismatched records, leading to dissatisfaction and potential loss of business.
- Security Risks and Undetected Fraud: Inadequate entity resolution might allow bad actors to evade detection. For instance, slight variations in name spellings across different databases could result in a missed connection, allowing fraudulent or criminal activities to go unnoticed.
- Unjust Credit Denials: Faulty entity resolution can lead to financial injustices. A customer with a solid credit history might be mistakenly denied credit because their positive records were not accurately matched with their profile due to minor discrepancies in data entries.
- Decision-Making Based on Guesswork: Without effective data matching and high data quality, businesses might make strategic decisions based on incomplete or incorrect data, leading to potential financial losses or missed opportunities.
- Use of Flawed Data in Business Processes: Utilizing unresolved data in business processes can lead to confusion and poor decision-making. For example, marketing might target the wrong audience or miss potential customers, resulting in wasted resources and lost revenue.
- Reputational Damage: Organizations failing to resolve entities accurately could face reputational harm, especially if data matching inaccuracies lead to publicized errors, such as wrongful accusations or service denials.
- Legal and Regulatory Compliance Repercussions: In sectors like finance and healthcare, inaccurate entity resolution may lead to legal and regulatory compliance breaches, potentially resulting in hefty fines and legal challenges.
These risks underscore the importance of investing in robust and accurate entity resolution systems. These systems arenโt important just because they improve business efficiency and customer satisfaction; they are also critical for maintaining security, compliance and trust.
What are the Most Common Entity Resolution Use Cases?
These use cases demonstrate the importance of entity resolution in various business processes, highlighting entity resolutionโs role in risk and fraud detection, customer 360, graph analytics compliance, and more.
Risk and Fraud Detection
Entity resolution plays a critical role in risk and fraud detection by figuring out who is who, and who is related to who – often despite international obfuscation. With entity resolution, organizations can proactively pinpoint bad actors and mitigate risks before they escalate. For instance, in financial institutions, entity resolution matches and links disparate pieces of information to help uncover fraud rings or insider trading activities.
CASE STUDY: The United States Citizenship and Immigration Services (USCIS) needed to improve its ability to identify immigration fraud committed by applicants and their representatives. The USCIS had many complex source systems and data models containing duplicate names and addresses as well as human-induced errors. The USCIS team used entity resolution for data matching and relationship detection to identify more fraud, improve the quality of insights and the user experience for its fraud analysts, and realize significant cost reductions.
Customer 360 & Marketing Analytics
Entity resolution is critical to delivering customer 360 solutions that enable more effective customer experience, marketing and other programs. Entity resolution unifies disparate customer identities to form a coherent, accurate 360-degree view of each customer. These solutions are crucial for efficient customer engagement across various touchpoints, ensuring that each interaction with a customer is informed by a complete understanding of their history, preferences and needs, which leads to better service and customer satisfaction.
CASE STUDY: Healthy Alliance is a Troy, NY-based organization with a charter to improve the health of the underserved. They struggled with resolving records from 150+ sources to get a complete, reliable view of community members. The organization used entity resolution to create 360-degree person centric views to better understand an individualโs social care needs, services provided and the outcomes of those services.
Graph Analytics
Combining entity resolution with graph technologies makes complex data networks more comprehensible and actionable. This synergy enhances the clarity of graphs for both humans and machines, leading to more accurate and insightful graph analytics and graph visualizations. Entity resolution also improves the performance and accuracy of machine learning models by providing a richer context for data relationships. In practice, this means more effectively identifying trends, risks and opportunities in large data sets.
CASE STUDY: Aptitude Global โ a technology and data consulting solutions provider โ recognized they needed better data matching and relationship detection for their Data Intelligence Platform. In response, Aptitude added entity resolution to their platform solutions used to combat financial crime and fraud. The Aptitude solution combines the best graph technologies and entity resolution to fight financial crime, identify politically exposed persons and sanctioned entities, and dynamically calculate customer risk.
Financial and Regulatory Compliance
Ensuring compliance with financial and regulatory standards is a complex task requiring precise data matching. Todayโs advanced entity resolution systems can provide the data accuracy and consistency necessary to meet government and industry requirements. They help correctly identify customers and their transactions, ensuring financial institutions can adhere to regulations such as the Bank Secrecy Act (BSA), which require KYC (Know Your Customer) and AML (Anti-Money Laundering) compliance.
CASE STUDY: NICE Actimize, recognized as a leader in anti-money laundering solutions by The Forrester Wave (Q3 2022), incorporates real-time entity resolution into its suite of AML products. This integration allows financial service providers to identify entities accurately, uncover complex relationships and detect suspicious activities, significantly enhancing their anti-money laundering efforts.
Master Data Management (MDM), Customer Relationship Management (CRM), Customer Data Management (CDM) and Other Data Management Software
In the realms of MDM, CRM, CDM and other data management software systems, entity resolution can improve the data quality of historical data and eliminate the entry of new duplicate data in real time. Ultimately, with well-resolved entity data in these core enterprise systems, originations deliver higher-quality outcomes at a lower operational cost.
How Does Entity Resolution Work Step-by-Step?
Watch the video below for clear, step-by-step examples of how records about people are matched, identified as related or determined not to match. Youโll come away with an understanding of the intricacies of entity resolution and the power of Entity Centric Learning.
Why Is It Important to Get Entity Resolution Right?
With the rapidly expanding volumes of data, the lack of consistency in this data, and an ever-widening range of use cases, traditional approaches to entity resolution (e.g., batch processing and rule-based tuning) are no longer sufficient.
Here are the top 5 things to get right when shopping for your next commercial-grade entity resolution engine:
- Simple to Use. Overly complex and inelegant products that require outside expertise or in-depth training to bring on new team members result in project overruns and much higher total cost of ownership. The greater the complexity, the longer it will take to integrate new data sources, both initially and for ongoing operations.
- Accuracy. This is one of the harder things to test for. Small record tests often misrepresent what will happen at scale. Synthetic data rarely, if ever, produces an accurate metric to assess the true ultimate accuracy of a production system. Best practice is to use real data for accuracy testing. If the volume is over 1M records, consider using a vertical slice (e.g., one geography, everyone with a last name starting with the letter โAโ).
- Minimal Moving Parts. Complicated stacks with lots of moving parts have many failure points, come with more security risks, and are much more involved to upgrade.
- Real Time. Batch-only entity resolution systems require periodic reloads to integrate new data or removed data (e.g., GDPR deletes).
- Explainable. Business units making decisions based on entity-resolved data must be able to explain why records matched or, conversely, why they did not. This is not only to enhance their confidence in the system, explainability may very well be required if a business unit is called before a regulator.
Advanced entity resolution technology delivered via an API is easier and quicker to deploy, use, manage and maintain compared to traditional full stack solutions. API-based entity resolution solutions substantially reduce the learning curve to get started, so you can be up and running in minutes and deploy in days or weeks. Advanced entity resolution will also use real time Entity Centric Learningโข, machine learned models and relationship awareness to deliver the best accuracy, time-to-value, and total cost of ownership (TCO) of today’s commercial entity resolution offerings.
How Should You Evaluate Entity Resolution Solutions?
Download the Entity Resolution Buyerโs Guide to learn all about how to evaluate and select an entity resolution solution. This guide will give you the knowledge and tools to make an informed decision on which type of entity resolution solution is right for your needs today and into the future.
Senzing Smarter Entity Resolution
Senzing is the first purpose-built real-time AI for entity resolution. Senzing software makes it easy and affordable to add advanced entity resolution capabilities to your enterprise systems and commercial applications. The Senzing API provides highly accurate data matching and relationship detection to improve analytics and decision-making, without requiring entity resolution experts.
โข Minimal data preparation is required with Senzing entity resolution.
โข No tuning, training or entity resolution experts are needed.
โข The Senzing API runs on premises, in the cloud or hybrid.
โข No data flows to Senzing, Inc.
How Do You Get Started with Smarter Entity Resolution?
If youโd like to know more about Senzing entity resolution:
Consult with an Expert โ Schedule a call with a Senzing entity resolution expert to discuss your requirements.
Try it Yourself โ There are three easy ways to take Senzing entity resolution for a test drive: a simple desktop evaluation tool (for Windows or Mac) and QuickStarts for Linux and Docker. You can install the software, load data and evaluate results in as little as 15 minutes.
When you see how Senzing Entity Resolution AI works, you’ll understand why we provide the most accurate and up-to-date results (that are continually updated in real-time).
Video Transcript For "What Is Entity Resolution?"
0:00 Intro
I thought I would just take a moment to explain what entity resolution is, in the way that we think about it at Senzing. There are a lot of different definitions for entity resolution, and ours is a little bit particular.
0:16 What is Entity Resolution?
So part one is recognizing when two records relate to the same identity despite having been described differently. You might see a whole bunch of differences between names and addresses…and fuzzy matching is the kind of thing thatโs necessary to be able to draw those two records together.
The second part is recognizing when two records donโt relate to the same identity, despite having been described similarly. You can have two records with virtually everything the same except one letter is off. One record can have โjrโ and one can have โsrโ (for a junior and a senior), and, they are actually different people, not the same entity.
And finally, if you want to do a really good job with entity resolution, you need to be able to track and manage the relationships between records. This might include a possible match, which is a form of a relationship, or a disclosed relationship (for example, when youโre informed that these two people are twins), as well as being able to recognize undisclosed relationships, which helps complete whatโs necessary to do high-quality entity resolution.
There you go, entity resolution, defined Senzing style.
If youโd like to know more about Senzing entity resolution:
Consult with an Expert โ Schedule a call with a Senzing entity resolution expert to discuss your requirements.
Try it Yourself โ There are three easy ways to take Senzing entity resolution for a test drive: a simple desktop evaluation tool (for Windows or Mac) and QuickStarts for Linux and Docker. You can install the software, load data and evaluate results in as little as 15 minutes.