Guide to Evaluating Entity Resolution Solutions
The Definitive Step-By-Step Guide to Evaluating Entity Resolution Software
The purpose of this Buyer’s Guide is to provide an overview of different types of entity resolution solutions and help you understand what to look for when evaluating and selecting one. By reading the guide, you will be better equipped to make an informed decision about which type of entity resolution solution is right for your needs today, and into the future.
This guide will cover:
• The value of entity resolution
• Different types of solutions available
• Key capabilities for evaluation
• How to get started with entity resolution
Table Of Contents
-
Introduction
-
What Is Entity Resolution
-
5 Primary Business Use Cases
-
Primary Entity Resolution Deployment Methods
-
Full Stack vs. API
-
Entity Resolution Accuracy
-
Entity Resolution Time To Value
-
Entity Resolution Total Cost of Ownership
-
Additional Evaluation Criteria
-
Entity Resolution Side-by-Side Comparison
-
How To Get Started
Got Questions?
Get Answers! Ask An Entity Resolution Expert…
Rather talk to a person? Have ER related questions? We can help. Click below to chedule a free, no-pressure consultation with a Senzing Entity Resolution expert.
What is Entity Resolution?
Entity resolution is the process of determining when real-world entities are the same, despite differences in how they are described or inconsistencies in how data was entered. Entity resolution also determines when real-world entities are different despite similarities in how they are described. For more in-depth information, take a look at our primer, “What Is Entity Resolution?”
Entity resolution can help an organization gain a competitive advantage. It matches and links data about people and organizations to create
360-degree views of entities.
Entity Resolution Powers Discovery & Insight
• Improves decision-making
• Bolsters analytics and insights
• Enhances data quality and accuracy
• Augments fraud detection & prevention
The Steps To Evaluating Entity Resolution
Define your organization’s use case for entity resolution before evaluating solutions.
Step 2: Determine Your Deployment Method
Clearly understand how you plan to use your entity resolution system.
Make sure the solutions you evaluate support all the deployment options you’ll need.
Evaluate how much time and effort is needed to achieve and maintain the accuracy levels your project requires.
Five Primary Business Use Cases
Step 1: Define your organization’s use case for entity resolution before evaluating solutions.
The five primary business use cases for entity resolution today include:
1. Risk and Fraud Detection
Identifying anomalies to proactively find bad actors, mitigate risk and curb fraudulent activities.
2. Financial and Regulatory Compliance
Providing the data necessary to meet government and industry requirements and standards.
3. Customer Data Management
Enhancing customer matching to increase accuracy and insights within and across systems.
4. Marketing and Customer Analytics
Gaining complete 360-degree views of customers to improve and refine marketing strategies.
5. Enhance MDM, CRM, CDI and Other Software
Increasing accuracy, enabling new capabilities, and enhancing downstream analytics.
Entity Resolution Deployment Methods
Step 2: Clearly understand how you plan to use your entity resolution system.
There are four primary ways organizations use entity resolution today:
Adding
entity resolution to a new or existing application
Integrating
entity resolution into a composable architecture
Replacing
a home-grown or commercial entity resolution solution
Augmenting
an existing data management system with entity resolution
Let’s take a closer look at the ways that enterprises are using entity resolution:
Adding to a New or Existing Application
Many enterprises and ISVs want to add entity resolution to a new or existing product to improve data matching, eliminate manual processes, or support new deployment models.
Replacing a Homegrown or Commercial Solution
The need to replace existing, limited entity resolution capabilities is driving many organizations to look for options that are more accurate, cost effective and enable business expansion.
Integrating Entity Resolution into a Composable Architecture
Enterprises building data fabrics and data pipelines, or implementing digital transformation initiatives, often deploy entity resolution as an enterprise-wide service in their architectures.
Augmenting an Existing Data Management System
Some organizations want to run entity resolution systems alongside applications that handle large amounts of data about people and organizations to improve data matching and extend capabilities.
Full Stack vs. API Technologies
Step 3: Define and rank your project’s requirements to better assess whether a full stack or API approach will meet your needs.
API-Based Solutions
API-based entity resolution solutions are modern, modular components that are usually easy to manage and quick to deploy. They can be added into enterprise or commercial applications and services or deployed as part of an enterprise-wide composable architecture. API solutions can offer integration and deployment flexibility and other operational advantages. In addition, they can substantially reduce the learning curve to get started versus full-stack solutions.
Full Stack Solutions
Full stack entity resolution solutions are comprehensive offerings that deliver a wide range of functionality but can be costly and complex to integrate and manage. They often provide more capabilities than you need and have specific infrastructure requirements that can impact how you architect and deploy your products or services. Despite these issues, full stack solutions have been the norm until recently, and are still widely used.
Cloud Based vs. On-Prem
Step 4: Make sure the solutions you evaluate support all the deployment options you’ll need.
Choose Between Cloud or On-Prem Infrastructure
You’ll need to determine where you will initially deploy your entity resolution system. Many organizations start with a cloud based infrastructure due to lower initial costs but may eventually transition to on-premises. Others deploy on-premises first but plan to move to the cloud over time.
Anticipating the future trajectory of your entity resolution system will enable you to select a solution that provides the flexibility you need to meet evolving needs.
Top 3 Critical Criteria for Evaluation
How to Evaluate the Capabilities of Different Systems
As you compare different entity resolution options, there are numerous criteria that should guide your evaluation.
We’ll first delve into the Top 3 Evaluation Criteria:
• Accuracy
• Time-To-Value
• Total Cost of Ownership
Then we’ll explore seven Additional Criteria to consider in your evaluation of entity resolution software solutions for your organization and use case.
Critical Criteria: Accuracy
Step 5: Evaluate how much time and effort is needed to achieve and maintain the accuracy levels your project requires.
What’s involved in time, effort and costs to achieve target accuracy?
Accuracy is one of the top factors to evaluate when considering entity resolution systems. Accuracy is critical because low error rates ensure your organization’s data is dependable and credible.
Pre-Configured Accuracy vs. Tuned and Trained Accuracy
It’s important to understand the difference between out-of-thebox accuracy and tuned and trained accuracy. Some solutions come pre-tuned and pre-trained and actively learn to deliver highly accurate results more quickly, and can be microtuned as needed for edge cases.
Other systems require substantial time and effort to reach and maintain target accuracy levels. Rules-based entity resolution systems require experts to write rules that improve accuracy and modify or add rules as new data sources are added. Probabilistic systems require thresholds to be adjusted to tune for accuracy based on the attributes within each data source.
Critical Criteria: Time-to-Value
Step 6: Understand what’s involved in reducing time-to-value.
How long to deliver value in production?
Time-to-value refers to how long it takes to get a system operational from installation to production. Calculations should include the time required for installation, onboarding data sources and training, tuning and testing the system. Results can vary widely depending on the solution.
Hardware Setup and Software Installation
Full-stack solutions typically demand specialized vendor assistance for installation, turning it into a weeks- or months-long process. API solutions often offer user-friendly setup procedures and installation that takes a day or less.
Data Source Onboarding
The time required to onboard initial data sources is crucial when calculating time-to-value. Some systems can take hours or days to onboard all sources. Others may need a month or longer to add just a single source.
Initial Training and Tuning
Certain systems may require a year or more of training and tuning before yielding results accurate enough for production. Determine
how long training and tuning will take (days or months) and how many internal or external resources are required.
Critical Criteria: Total Cost of Ownership
Step 7: Ensure your TCO calculations focus on initial deployment, as well as ongoing operations and maintenance.
How to fully evaluate all costs over time.
The total cost of ownership (TCO) is based on initial deployment costs plus ongoing operation and maintenance costs. Since the solution you choose will be deployed for many years, ongoing costs will be a major part of TCO.
Initial Deployment Costs
Initial deployment costs include all the software you’ll need, hardware or cloud infrastructure, and the resources required to onboard data sources and tune and train the system. Some entity resolution solutions require expensive, vendor-specific hardware and software that can make integration challenging.
Ongoing Operating and Maintenance Costs
When projecting ongoing costs, be sure to include both staffing expenses and additional costs associated with software, infrastructure complexity, and increasing data volumes. Adding a new data source can be complex and costly for many solutions, both to onboard the data and tune and train the system. Some systems also require data to be reloaded regularly to maintain accuracy, which increases maintenance and admin costs.
Additional Evaluation Criteria
Step 8: Assess other evaluation criteria to ensure you select a solution with features that line up with your needs.
Ease of Use
Systems vary significantly in terms of the user-friendliness of the entire entity resolution process. Make sure you understand how easy a system is to use from installation and data source integration to tuning and training.
Relationship Detection
Most entity resolution systems perform record matching to identify people and organizations. Some systems also identify disclosed and discovered relationships between entities for richer context and deeper insights. If your organization would benefit from understanding households or other relationship networks, confirm any system you evaluate supports these capabilities.
Real-Time vs. Batch
While many organizations rely on batch-based entity resolution, the need for real time is becoming increasingly important. Batch data is easily supported by real-time systems, but transforming a batch system to real time can be impossible or expensive. Even if you don’t need it today, we recommend choosing a real-time system to future proof your solution.
If you opt for real time, clearly identify the specific real-time capabilities a solution provides, as different vendors define “real time” differently. Some just mean real-time querying of data that’s already in the system. Others mean continuously ingesting, resolving, querying, deleting and self-correcting streaming data as it is received in real time.
Explainable Results
Systems vary in terms of the types of explanations they provide about how entity resolution decisions are made. Some offer no information at all. Since it’s common for business users, data scientists and auditors to want to know why records matched or didn’t, you want a system that provides details about how specific decisions were made.
Scalability
Your system may need to scale to support larger data volumes, more data complexity or new features. Since scalability is difficult to retrofit, determine whether a solution can meet the volumes of data and types of performance-intensive capabilities you’ll need for current and future requirements.
Privacy
It’s important to know if anyone outside your organization will have access to your data. Some systems are so complex that you may need to give a vendor access. You should also investigate the details about where your data will flow and what data is or isn’t encrypted.
Security
The more complex the solution, the more authentication and authorization mechanisms your team will need to learn and audit, and the more systems you’ll be required to monitor, patch and maintain. Some solutions require you to add security patches to a variety of technologies and services while others need few if any changes to your existing security methods.
Side-by-Side Comparison
Step 9: Use the Entity Resolution Evaluation Checklist to compare options side-by-side.
Evaluation Criteria | ||
|---|---|---|
Quick, Easy Software Installation |  | |
Self Tuning |  | |
No Training Required |  | |
Full-Stack or API Technology |  | |
Minimal data preparation for new data sources |  | |
Highly accurate ‘out of the box’ |  | |
Fast time-to-value |  | |
Built-in AI that gets smarter over time |  | |
Relationship detection to multiple degrees |  | |
Real-time performance with sequence neutrality |  | |
Easily explainable results |  | |
Deployment flexibility for cloud, on-prem & SaaS |  | |
Dual hardware systems required for batch reloads |  | |
Easily scalable to meet all future requirements |  | |
No one outside your org can access your data |  | |
Low total cost of ownership (TCO) |  | |
Rapid return on investment (ROI) |  |
Download This Entire Guide as a PDF:
How To Get Started
Senzing Entity Resolution API
Senzing software makes it easy and affordable to add advanced entity resolution capabilities to your enterprise systems and commercial applications.
The Senzing API provides highly accurate data matching and relationship detection to improve analytics, insights and outcomes with no entity resolution experts required.
You can be up and running in minutes and deploy into production in weeks.
Consult With An Expert
Schedule a call with a Senzing entity resolution expert to discuss your requirements.
Try It Yourself
There are three easy ways to take Senzing entity resolution for a test drive – a simple desktop eval tool (for Windows or Mac) and QuickStarts for Linux and Docker. You can install the software, load data and evaluate results in as little as 15 minutes.