Entity Resolution: Insights and Implications for AI Applications

By Ben Lorica of Gradient Flow, published June 29, 2023

This article was first published in the newsletter from Gradient Flow, an organization presenting a rich array of high quality content on data analytics, technology and business, with a focus on machine learning and AI.

Software systems may start simple, but adding sophisticated features and ensuring maintainability leads to complexities, contributing to the age-old ‘build versus buy’ dilemma in software acquisition. With every team needing to weigh the pros and cons of developing new technology in-house versus procuring it from third parties, factors such as cost, implementation timeline, and technical risk come into play, necessitating a tailored decision based on individual business needs, resources, and risk tolerance.

Entity resolution (ER), crucial software I’ve included in my “Don’t Try This At Home” list, involves systematically connecting disparate data records representing the same real-world entity, such as customer, product, or company names. This process is extremely important because poor data quality adversely affects downstream analytics and AI applications. Although ER may appear straightforward initially, it’s a complex problem teeming with applications including customer data management, fraud detection, data  quality enhancement, data integration, data governance, and business intelligence.