Who this affects
Any Snowplow user enriching their events with the MaxMind-powered IP Lookups Enrichment:
The original MaxMind GeoIP DAT (legacy) databases are no longer available for new customers, so it’s impossible for a new Snowplow user to get started with those.
Furthermore, MaxMind have announced that:
We will be discontinuing updates to the GeoLite Legacy databases as of April 1, 2018. You will still be able to download the April 2018 release until January 2, 2019. GeoLite Legacy users will need to update their integrations in order to switch to the free GeoLite2 or commercial GeoIP databases by April 2018.
For more information see:
Impact on Snowplow users
The Snowplow IP Lookups Enrichment uses the MaxMind “GeoLite Legacy” databases.
Because the last GeoLite Legacy files will be released on 1st April, effective as of 1st May, your Snowplow installation will be missing out on the latest MaxMind database updates.
Snowplow will continue to function correctly, but the IP Lookups Enrichment will become less and less accurate for incoming events, as IP addresses “drift away” from the 1st April GeoLite Legacy database values.
What we are doing to resolve this
We need to move Snowplow to use the newer format GeoLite2 databases. We have a multiple step plan to do this:
Step 1. Start providing hosting for the GeoLite2 databases
We will start providing this new database file in Amazon S3, as we do the legacy database.
Step 2. Release a new version of scala-maxmind-iplookups
We are working on a new version of this library, with support for the new database file format.
Huge thanks to Snowplow community member Tiago Macedo for providing a PR in support of this work:
Step 3: We will release new Snowplow versions (batch and realtime)
We will update Stream Enrich and Spark Enrich to be able to use the new database file format. You can follow our progress in this milestone:
Preparing your Snowplow installation
As time is relatively tight on this, we strongly recommend open-source users of Snowplow to prepare for this release by upgrading to the most recent available Snowplow release for their respective pipeline (batch or real-time).
Please note that we have no plans to back-port this MaxMind upgrade to any prior versions of Snowplow; if this enrichment is important to your usage of Snowplow, then do upgrade as the first step.
We are globally pinning this thread within Discourse to maximize visibility of this issue, and will post back with updates as our upgrade plan rolls out.
Please let us know any questions.