Entity Resolution: Insights and Implications for AI Applications

Ben Lorica recently wrote an informative article (maybe the best, ever) for his Gradient Flow newsletter about the unique Senzing purpose-built AI for entity resolution. The article discusses the following nine pillars of modern entity resolution systems: scalability, real-time processing and inference, explainability, privacy features, principle-based, sequence neutrality, support for global languages, robustness, and auditable.

Image courtesy of Ben Lorica of Gradient Flow

Ben specifically calls out Sequence Neutrality as one of the more challenging and unique aspects of the real-time learning Senzing^® entity resolution provides:

“Sequence neutrality is an important requirement. This concept refers to the ability of a system to maintain consistent decision-making regardless of the order in which data arrives. It’s crucial because new data can often retrospectively change our understanding of past data. Sequence neutrality means that your entity resolution system can consistently identify the same entities, regardless of the order in which the related data was received, similar to how new information can change our understanding of a past conversation. However, most ER systems lack sequence neutrality, which can lead to inconsistent and inaccurate outcomes. The common solution is to periodically reload all data, a time-consuming process that can lead to ongoing inaccuracies. Designing a system that implements sequence neutrality is exceptionally complex, and it can be a daunting task to undertake, especially when considering the potential need for real-time correction of prior assertions based on new information.”

Between sequence neutrality and the other pillars of modern entity resolution, Ben goes on to summarize the big picture:

“Entity resolution is a powerful example of how big data, real-time processing, and AI can be combined to solve complex problems. The insights garnered from ER’s challenges in maintaining accuracy, managing scale, and dealing with complexity can enrich other AI applications, enhancing their precision, scalability, and sophistication. The principles of ER can directly be applied to any AI application that involves identifying and linking entities. This includes applications such as fraud detection, customer relationship management, and natural language processing.”

We believe next generation AIs must evolve to support explainability, sequence neutrality, real-time/low latency, and cost-efficient scaling.

In the field of AI for entity resolution, Senzing has already tackled these hard problems. The 6th generation entity resolution engine from Senzing is a result of decades of team collaboration, drawing on 300+ collective years of entity resolution experience. This completely unique real-time AI for entity resolution is industrial-strength, immediately available as commercial off-the-shelf (COTS) software and available to developers everywhere working on projects of any size.

Ben acknowledged the advantages of Senzing:

“These days, I recommend Senzing to teams looking for an entity resolution solution. Senzing is a robust, efficient, and flexible entity resolution system that is cost-effective, and can be deployed on premises or in the cloud. Senzing takes data privacy seriously, offering field hashing and application-level encryption. It is also accurate at scale, capable of processing large volumes of data efficiently and in real time. Senzing can efficiently manage thousands of transactions per second, execute entity resolution in just 100 to 200 milliseconds, and conduct queries within tens of milliseconds, all while handling billions of records. Finally, Senzing upholds sequence neutrality, ensuring consistent identification of entities regardless of the order in which data arrives.”

The entire Entity Resolution: Insights and Implications for AI Applications article is worth a read on Gradient Flow. Gradient Flow is an essential source in information if you are interested in Big Data and AI. I highly recommend subscribing to the Gradient Flow newsletter.

Ben and I discussed all this and more on The Data Exchange podcast: Jeff Jonas on how Senzing makes entity resolution easier and more effective.

Reference Links
Why Explainability is Critical for Entity Resolution
Real-Time, Continuous Entity Resolution Explained
Privacy by Design (PbD) History & Features of Senzing
Benefits of the Senzing Principle-Based Approach