G2Engine Redo Processing
Redo Processing
Redo records are automatically created when certain conditions occur during entity resolution where additional processing is required.
The most common reasons are:
- A feature value becomes generic and previous decisions need to be revisited
- Clean up after some record deletes
- Detected related entities were being changed at the same time
- A table inconsistency exists, potentially after a non-graceful shutdown
countRedoRecords
countRedoRecords() returns the integer count of the remaining internally queued redo records in the Senzing repository.
g2_engine.countRedoRecords()
Click to expand `countRedoRecords()` example
Example
#! /usr/bin/env python3
from senzing import G2Engine, G2Exception
# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'
g2_engine = G2Engine()
try:
g2_engine.init("G2Engine", senzing_engine_configuration_json)
result_count = g2_engine.countRedoRecords()
g2_engine.destroy()
print(result_count)
except G2Exception as err:
print(g2_engine.getLastException())
Response message:
getRedoRecord
getRedoRecord() retrieves the next internally queued redo record into the Senzing repository.
Unlike processRedoRecord(), getRedoRecord() does not actually process the record.
To process the record, use process()
g2_engine.getRedoRecord(response_bytearray)
Parameters
- response_bytearray: (bytearray) Object to store the output of the method.
Click to expand `getRedoRecord()` example
Example
#! /usr/bin/env python3
from senzing import G2Engine, G2Exception
# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'
g2_engine = G2Engine()
response_bytearray = bytearray()
try:
g2_engine.init("G2Engine", senzing_engine_configuration_json)
g2_engine.getRedoRecord(response_bytearray)
print(response_bytearray.decode())
except G2Exception as err:
print(err)
Output
{
"REASON": "LIB_FEAT_ID[13] of FTYPE_ID[7] went generic for CANDIDATES ECLASS_ID[1] in LENS_ID[1]",
"DATA_SOURCE": "TEST",
"RECORD_ID": "39F84F26E1F85ABCCE20D5FDBCA7D08729DD829A",
"ENTITY_TYPE": "GENERIC",
"DSRC_ACTION": "X"
}
process
process() processes the redo record retrieved using getRedoRecord().
g2_engine.process(response_bytearray)
Parameters
- response_bytearray: (bytearray) the redo record obtained by
getRedoRecord() to process.
Click to expand `process()` example
#! /usr/bin/env python3
from senzing import G2Engine, G2Exception
# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'
g2_engine = G2Engine()
response_bytearray = bytearray()
try:
g2_engine.init("G2Engine", senzing_engine_configuration_json)
g2_engine.getRedoRecord(response_bytearray)
g2_engine.process(response_bytearray)
print(response_bytearray.decode())
except G2Exception as err:
print(err)
processWithInfo
processWithInfo() processes the redo record retrieved using getRedoRecord(), and returns a JSON document containing the ENTITY_ID values of the affected entities.
g2_engine.processWithInfo(response_bytearray, process_response_bytearray)
Parameters
- response_bytearray: (bytearray) the redo record obtained by
getRedoRecord() to process.
- process_response_bytearray: (bytearray) Object to store the output of the method.
Click to expand `processWithInfo()` example
Example
#! /usr/bin/env python3
from senzing import G2Engine, G2Exception
# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'
g2_engine = G2Engine()
response_bytearray = bytearray()
with_info_bytearray = bytearray()
try:
g2_engine.init("G2Engine", senzing_engine_configuration_json)
g2_engine.getRedoRecord(response_bytearray)
g2_engine.processWithInfo(
response_bytearray,
with_info_bytearray)
print(with_info_bytearray.decode())
except G2Exception as err:
print(err)
Output
{
"REASON": "LIB_FEAT_ID[1307] of FTYPE_ID[14] went generic for CANDIDATES AND SCORING ECLASS_ID[1] in LENS_ID[1]",
"DATA_SOURCE": "CUSTOMERS",
"RECORD_ID": "2181",
"ENTITY_TYPE": "GENERIC",
"DSRC_ACTION": "X"
}
processRedoRecord
processRedoRecordWithInfo() processes the next redo record in the redo queue.
If processRedoRecord() returns a blank response_bytearray, then the redo queue is empty at the point of execution.
Additional redo records can be created after reevaluation.
g2_engine.processRedoRecord(response_bytearray)
Parameters
- response_bytearray: (bytearray) Object to store the output of the method.
Click to expand `processRedoRecord()` example
Example
#! /usr/bin/env python3
from senzing import G2Engine, G2Exception
# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'
g2_engine = G2Engine()
try:
g2_engine.init("G2Engine", senzing_engine_configuration_json)
g2_engine.processRedoRecord()
except G2Exception as err:
print(err)
Output
{
"DATA_SOURCE": "CUSTOMERS",
"RECORD_ID": "2192",
"AFFECTED_ENTITIES": [
{
"ENTITY_ID": 105
}
],
"INTERESTING_ENTITIES": {
"ENTITIES": []
}
}
processRedoRecordWithInfo
processRedoRecordWithInfo() processes the next redo record in the redo queue, and returns a JSON document containing the ENTITY_ID values of the affected entities.
g2_engine.processRedoRecordWithInfo(response_bytearray, with_info_bytearray)
Parameters
- response_bytearray: (bytearray) Object to store the output of the method.
- info_bytearray: (bytearray) Returns the “withInfo” section
Click to expand `processRedoRecordWithInfo()` example
Example
#! /usr/bin/env python3
from senzing import G2Engine, G2Exception
# REPLACE /home/user/your_project with the path to your Senzing project
senzing_engine_configuration_json = '{ "PIPELINE": { "CONFIGPATH": "/home/user/your_project/etc", "SUPPORTPATH": "/home/user/your_project/data", "RESOURCEPATH": "/home/user/your_project/resources" }, "SQL": { "CONNECTION": "sqlite3://na:na@/home/user/your_project/var/sqlite/G2C.db" } }'
g2_engine = G2Engine()
response_bytearray = bytearray()
with_info_bytearray = bytearray()
try:
g2_engine.init("G2Engine", senzing_engine_configuration_json)
g2_engine.processRedoRecordWithInfo(
response_bytearray,
with_info_bytearray)
print(with_info_bytearray.decode())
except G2Exception as err:
print(err)
Output
{
"DATA_SOURCE": "CUSTOMERS",
"RECORD_ID": "2192",
"AFFECTED_ENTITIES": [
{
"ENTITY_ID": 105
}
],
"INTERESTING_ENTITIES": {
"ENTITIES": []
}
}