Filtering events from specific IPs


#1

Hey,

I was wondering if there is a way to filter events from specific IPs BEFORE they are loaded into the target.

Before GDPR we could filter these IPs in the queries, but now we are using the IP anonymization enrichment.

Would love to hear what you think.

Thanks.


#2

@moshesh, you can use JavaScript script enrichment for that purpose. It comes before PII enrichment and hence the IPs are still visible to the enrichment component. You can use data from IP lookup enrichment which produces values such as geo_country to filter based on the country origin of the events.

Do remember that raw data (if batch pipeline used) would still contain the IP address in S3. You might want to set up life cycle rules to get the data flushed after a reasonable number of days.


#3

Thanks @ihor!

The javascript enrichment sounds like a perfect solution.


#4

For a less permanent or retroactive application, you could try this script:

If you know your Salt, IP addresses and push it through the same cryptographic hash function, you will get the same output as the Snowplow enrichment emits.

Then you can just filter by hashed user_ipaddress in SQL.