Ua-parser enrichment - data is in the ..._snowplow_ua_parser_context_1 table but not in events

For data modelers & consumers
1

I’ve got the ua-parser enrichment working and I can see that it’s adding the values to this table com_snowplowanalytics_snowplow_ua_parser_context_1 but shouldn’t this data be then fed back into the events table?

Or will the ua-parser enrichment always maintain the data in this table I need to do further data modeling to join these two together in different tables?

Any help is appreciated.

Thanks

2

Hi @TimLavelle

Events that use self-describing contexts/some enrichments such as ua_parser, performance_timing and custom schemas can optionally be “shredded” out into their own individual tables (as opposed to being stored as one or more JSON objects). You can join these back to the events table using events.collector_tstamp = shredded_table.root_tstamp and events.event_id = shredded_table.root_id.

3

Hey Mike, quick reply, thanks! Is this something I can do via the loader process?

4

Enabling or disabling shredding or something else?

The options for Snowplow EMR ETL runner might be of help here.

5

I use the snowplow runner locally at the moment while I familiarize myself with, just set it up this past weekend.

So I run this command for the runner:
./snowplow-emr-etl-runner --config config/config.yml --targets targets/ --resolver config/resolver.json --enrichments enrichments/

and then run this for the loader
./snowplow-storage-loader --config config/config.yml --resolver config/resolver.json --targets targets/ --skip analyze

Running these two process the logs and I already have the ip_lookups enrichment happening and it shows the data in the main events table, but the ua_parser enrichment does not put the os/device data into events, it just keeps it in the separate table I mentioned earlier.

So what I am trying to find out is if I can have the data merged back into the events table during the loader process or is this something I need to configure myself as a separate SQL enrichment?

Really appreciate the help! I wasn’t expecting responses so quickly. :slight_smile:

6

I don’t think that’s possible as far as I know the ua-parser enrichment generates it’s own table whereas the related user-agent-utils doesn’t (and populate events instead of a dedicated table).

1 Like
7

Ahhh, right ok! I went with ua-parser due to others saying it was more reliable.

Thanks for the help and feedback, I’ll give them both a shot.

Cheers
Tim

8

It’s also worth mentioning that it’s possible (and quite a few people do) run them both at the same time.

9

Ahh, that is good to know! Although the loader is quite fast, but the runner on average takes about 30 minutes for a few dozen entries.

I think I’ll look around the forum and see if there’s some optimisation tips.

Thanks again

10

Found the answer… 30-60 minutes seems like average times. :sunglasses: