After reading your post once again, I now think that your problem is that duplicates remain across batches. I’m still not sure if synthetic or natural ones bother you, but you can fix latter ones with mentioned before cross-batch syntetic deduplication introduced in R88. Bear in mind that it has its costs both in terms of money and complexity. Synthetic cross-batch deduplication though isn’t yet available, but from what I know about shape of end-result data, amount of synthetic duplicates across batches is neglectable.
Please, correct me if I’m wrong with diagnosis.