Entity Resolution: Quick & Easy Way to Implement Cost Cutting Measures for Chief Data Officers During COVID-19
Entity resolution has become a key technology for Chief Data Officers, Chief Digital Officers and data strategists no matter which industrial sector they are in. There are many papers and articles written by experts about entity resolution (ER).
As customers, we demand personalization and are getting used to how our favorite companies are reaching us by knowing exactly who we are. They know our name, our current address, our recent online behavior, all digital interactions and most importantly they are not confused about our identities. There’s only one identity for each customer and we appreciate it. This all starts and ends with companies access to accurate algorithms for resolving an individual’s identity also called entity resolution.
Legacy companies have been manually resolving customer identity for ages. However, identities are often resolved with hundreds or thousands of lines of “if” and “then” codes. These “if” and “then” codes have been written over the course of multiple years. They are executed as overnight batch jobs with significant computational tax on the infrastructure and customer identities are resolved with multiple false positives and negatives. The algorithms are as good as the “if” and “then” statements Many CDOs have seen and experienced this first hand for the last few years especially as the need for resolving customer identities have risen in the industry.
Fast forward to the 21st century and you have commercially available algorithms that can resolve entities, real time, in milliseconds. These newer algorithms are not based on “if” and “then” statements. They deduce customer identities to a single entity based on more advanced techniques including artificial intelligence (AI), machine learning (ML), and robust reference data sets. These algorithms rely on data – the more data you have, the better resolution you get.
However, the struggle most CDOs have is the fact that they are so tied in with legacy systems that breaking apart systems that run your business is not a choice a CDO can make. So how should legacy companies think about transforming to new ER systems and focus on quickly reducing costs when most needed, especially during COVID-19 cost cutting measures?
Here’s a few steps we would recommend to start out with if you are looking at an easy way to reduce costs for your company given the COVID-19 cost pressure:
- Identify the need for entity resolution within your company – it’s most often tied to customer experience, fraud or compliance/data governance initiatives.
- Identify how it’s being done today within the company – number of full time employees needed to execute ER, time it takes for full execution and complexity of the “if” and “then” algorithms.
- Identify the number of full time developers needed to constantly write new logic to stay the pace.
- Identify total cost of ownership – quantify hidden software costs and partner/vendor licensing costs needed to resolve addresses and names.
- Ask yourself how critical is postal code verification? Do you always snail mail your customers, if not, then do you need postal code verification licensing costs to resolve addresses? Quantify the cost of licensing postal code verification software…
- Identify how many entity resolution algorithms you have in place within your company. You will be surprised to find that each department may have its own ER algorithm. If yes, then go to step 1 and quantify for each department.
The above six measures should give you the cost of running your existing ER systems in the current state. Bear in mind, this state likely does not provide real time capabilities, detection of relationships, and other features found in modern commercial entity resolution products.
We have put together a robust return on investment (ROI) calculator to help you measure your current costs for ER. After you measure your current costs, our follow up recommendation would be to run a quick POC for ER on your current dataset to determine the value proposition – including accuracy advantages for yourself. Given the COVID-19 cost cutting pressure on the industry, this could be an easy charter for any CDO/data strategists in the industry that will help reduce costs rapidly and remove redundancies.
Good entity resolution algorithms designed for on-prem or cloud are now commercially available at a much cheaper cost then what it used to be in the past – Senzing being one of them.
As a data strategist, our recommendation will be to start out with resolving the basics of your ER. As we stated earlier, upon inspection, ER is likely to be a significant hidden cost.
If you feel like you are being held hostage by your existing legacy ER systems, there is now hope to escape this expensive trap. New solutions can be deployed with an ROI measured in months if not weeks. If you have an interest in computing the ROI and other benefits of modern commercial ER, feel free to reach out to Anju or Jeff and we will be happy to guide you through it.
About authors:
Anju Gupta is a technology executive with more than twenty years of industry experience in the field of analytics. She believes in powering enterprises with data-driven solutions. She has held Chief Data Officer, Digital Innovation Officer, and Technology Strategy Advisor roles in large corporations. A former Monsanto Science Fellow and YWCA Women leader for St Louis, she is a sought after speaker on topics of Digital Innovation and Artificial Intelligence. Anju has a Ph.D. in Quantitative genetics and Statistics from The Ohio State University and has ~17 granted and published patents. She also serves as an Executive Board member of AIWorld.
Jeff Jonas, founder and CEO of Senzing, is an acclaimed data scientist and the leading creator of Entity Resolution systems. For more than three decades, he has been at the forefront of solving complex big data problems for companies and governments. National Geographic recognized him as the Wizard of Big Data. Jonas was awarded an honorary Ph.D. in Science in 2015 from Claremont Graduate University and is the author or co-author of 14 patents. His work has been featured in documentaries airing on networks such as the Discovery Channel, and he has been the subject of prominent chapters in books such as No Place to Hide, Safe: The Race to Protect Ourselves in a Newly Dangerous World, The Numerati and The Watchers: The Rise of America’s Surveillance State.