10,000 Rules, Oh The Joy!

By Jeff Jonas, published January 22, 2019

I once found myself at a major financial institution learning about their fraud detection system. As they proudly highlighted the system’s 10,000 custom alerting rules, I could not help but think to myself: “WOW! This system detects exactly 10,000 things, no more, no less. Any slight fraud variation will require another new rule.”

I imagined their ever-growing number of rules. I pondered who in their organization even understands all these rules? I wondered if they would still be bragging when they have 20,000 rules?

I saw them one year later. They had 20,000 rules. They were no longer bragging.

Imagine telling your kid one day to quit throwing rocks at cars. Only to realize the next day you have to tell him to quit throwing rocks at SUVs. Then, a few days later, you realize you must tell him to not throw rocks at trucks, fire engines and ambulances. Ummm… 10,000 rules later you are still adding rules like: “Don’t throw golf balls at Teslas.”

Instead of all these rules, why not one simple principle: “Don’t throw things at other people’s stuff.”

Systems benefit from similar principles.

My team and I first implemented principle-based alerting in 1993. The system, called NORA (Non-Obvious Relationship Awareness), was built for a Las Vegas casino. Instead of building alerting rules, we implemented alerts via principles.

Principle 1: “Tell me when the bad guy is the good guy.”
Principle 2: “Tell me when the bad guy knows the good guy.”

With just these two principles, the system started kicking out all kinds of valuable, unanticipated insights including one of my favorites:

True Story: An alert surveillance room operator watching through the eye in the sky (video camera) noticed a man cheating on a roulette table. He was making bets after the ball fell (something called past posting). Dealers are supposed to watch for this, but the dealer at this table kept missing the obvious scam. Casino security detained the cheater. The dealer said: “I can’t believe this happened to me. I am so embarrassed. You surveillance folks are sure doing a good job. It won’t happen again.” During the arrest processing, the cheating player provided a different last name and address than the dealer’s, but the home phone number he provided was the same number the dealer used on her original employment application. They lived together! NORA’s alert: “The cheater is related to the dealer.” The player instantly rolled-over and confessed.

If we had deployed a traditional rules-based alerting system for this casino, it is likely we would have forgotten the rule: if the phone number on an employee’s original job application matches an arrestee’s phone number, surface an alert.

But, since NORA’s alerts were based on principles, the casino caught this corrupt dealer.

When you find yourself managing too many rules, step back for a minute and ask yourself whether you should be using principles instead?

This is exactly what I did in 2009 when I took a step back and imagined: What if we used principles to perform Entity Resolution, not just alerting? Would this approach allow one default configuration to entity-resolve people, companies, vessels, planes, and more entities? Would it support different languages? Would it work without pre-training or tuning?

Principles proved out. Today, our principle-based Entity Resolution is the primary breakthrough that makes Senzing significantly easier to own and operate than other Entity Resolution systems. With Senzing, organizations are no longer beholden to experts as they train, tune, train, tune, train, tune their legacy Entity Resolution algorithms.

P.S.: Some rules are fine e.g., telling the kids: “Always be home before 9 p.m. on weekdays.” Or, if you are a bank: “Report all transactions over $10,000,” a federal law. The point is to favor principles over rules wherever possible. Also, some will argue that principles are simply more abstracted rules. Word.