Search This Blog

Tuesday 13 September 2011

Identifying duplicate records by using expression transformation

Identifying Duplicate Records By Using Expression Transformation

To segregate distinct and duplicate records in a source, solution described below uses an expression transformation.
This solution exploits the concept that all input ports are evaluated first, then variable ports are evaluated and all output ports are evaluated in last


SRT_SOURCE_RECORDS
Sorter transformation is used after Source Qualifier to sort source records. All ports are selected as key

EXP_FLAG_DUPLICATE_DISTINCT
Expression transformation is used to flag a record as either duplicate or distinct. Current record (v_CurrentRec) is compared with
previous record (v_PrevRec) and if records match v_IsDuplicate variable is set to ‘Y’ else it is set to ‘N’.


RTR_DUPLICATE_DISTINCT
Router transformation is used to segregate duplicate and distinct records using o_IsDuplicate port.

No comments:

Post a Comment