Refer_urlhost is null in enriched json


#1

Hi, I am sorry if this is a very basic or stupid question.
But I have noticed this behaviour.
I am using pythnon sdk to convert enriched tsv to json. I noticed in some events page_referrer and refer_urlhost is not present(this behaviour is mostly visible in page_ping event) can someone please help me sorting out this query?

Thanks!


#2

Hi umeshk,

Page pings don’t have referrers - page views do. If you find the page view from which the page ping originated, that’s where that data should be.

Do let us know if you run into issues when looking at the page views though!


#3

Hi Colm,

Thanks for the reply.
I agree most of the page pings do not have referrers but many do, that’s why I got this confusion.

below are two events and I have seen many such events.

{
"collector_tstamp": "2019-02-27T06:59:49.390Z",
"network_userid": "ce3f9f89-fda0-4856-b7de-e90ceeff3b94",
"user_id": "785620",
"platform": "web",
"app_id": "popxo-web",
"page_urlpath": "/2019/02/cast-of-left-right-left-then-vs-now/",
"event_name": "page_ping",
"refr_urlhost": null,
"story_id": 799813,
"remote_content_categories": ["Lifestyle"],
"published_at": "2019-02-27",
"tags": ["celebrities ", "celebrities ", ""],
"post_type_for_filtering": "wordpress_post"
}
{
"collector_tstamp": "2019-02-27T06:59:49.413Z",
"network_userid": "b96afdb6-5582-4da4-858f-9a51cb7810ba",
"user_id": null,
"platform": "web",
"app_id": "popxo-web",
"page_urlpath": "/trending/sara-ali-khan-and-ananya-panday-are-the-most-desi-bffs-799718/",
"event_name": "page_ping",
"refr_urlhost": "l.facebook.com",
"story_id": 799718,
"remote_content_categories": ["Lifestyle"],
"published_at": "2019-02-26",
"tags": ["bestie", "bollywood", "celebrities ", "celebrities ", ""],
"post_type_for_filtering": "wordpress_post"
}

#4

Ah! Serves me right for making assumptions.

Which SDK are you using and which version?
You mentioned Python - which version?

When a page ping is missing these, is the correlated page view also missing the same? (web page context has an ID that is shared across both).

Are there some events which do have these values and some which don’t, or is it the same for all?

Best,


#5

Hi Colm,
I am using snowplow-analytics-sdk(python) version is 0.2.3. and python version is 3.6
This behaviour is random, there are some page_views as well which are missing referrer values.
Regarding correlated page_view missing the value, I need to check that. Will check that and let you know.
However, there is another thing I noticed someday back. Initially we were using scala sdk with gordon, in that if a value for a key is missing then also the key used to be there in the json, however in python, if the value is null, it will not add the key in the JSON.