How to remove duplicate records?
When working with unclean data, you would want to remove any duplicate records in your data. Use the following steps to perform the de-duplication:
Step 1: When you create a new connection, you will get to Preview Screen.
Step 2: On preview screen, select the"remove duplicates" from others section in drop down list as shown
Step 3: It will apply the operation on row basis and will compare the whole row. If the row is having all columns same it will consider it as a duplicate.
Step 4: As and when you click on remove duplicates, the system will apply the opeartion and will delete the duplicates entries by deleting the rows.