Brackets in ue_pr get url decoded


#1

Hi all,

I’m having a strange behavior and as such, a lot of bad data.

In this case below are 3 pixel impressions (/i) but the same happens on link clicks (/r/tp2)
Take a look into the following 3 events, all hits from the same user/environment:

bad data:

2017-01-10 17:01:00 - 43 87.XXX.85.1X2 GET 87.XXX.85.1X2 /i 200 - Mozilla%2F5.0+%28iPhone%3B+CPU+iPhone+OS+10_0_2+like+Mac+OS+X%29+AppleWebKit%2F602.1.50+%28KHTML%2C+like+Gecko%29+Mobile%2F14A432 e=ue&ue_pr={%22schema%22%3A%22iglu%3Acom.snowplowanalytics.snowplow%2Funstruct_event%2Fjsonschema%2F1-0-0%22%2C%22data%22%3A{%22schema%22%3A+%22iglu%3Acom.xpto%2Fopen%2Fjsonschema%2F1-0-4%22%2C%22data%22%3A{%22cid%22%3A%2271244%22%2C%22eid%22%3A%22234807%22%2C%22uid%22%3A%2222139562%22%2C%22geo%22%3A%22BE_NL%22}}}&tv=custom&p=web&cv=clj-1.1.0-tom-0.2.0&nuid=3761f80d-db5b-4589-be3c-8cf0aeaed515 - - - - -

good data:

2017-01-10 17:01:02 - 43 87.XXX.85.1X2 GET 87.XXX.85.1X2 /i 200 - Mozilla%2F5.0+%28iPhone%3B+CPU+iPhone+OS+10_0_2+like+Mac+OS+X%29+AppleWebKit%2F602.1.50+%28KHTML%2C+like+Gecko%29+Mobile%2F14A432 e=ue&ue_pr=%7B%22schema%22%3A%22iglu%3Acom.snowplowanalytics.snowplow%2Funstruct_event%2Fjsonschema%2F1-0-0%22%2C%22data%22%3A%7B%22schema%22%3A+%22iglu%3Acom.xpto%2Fopen%2Fjsonschema%2F1-0-4%22%2C%22data%22%3A%7B%22cid%22%3A%2271285%22%2C%22eid%22%3A%22234799%22%2C%22uid%22%3A%2222139562%22%2C%22geo%22%3A%22BE_NL%22%7D%7D%7D&tv=custom&p=web&cv=clj-1.1.0-tom-0.2.0&nuid=81e67a3b-4948-476a-98f7-04fa4447234c - - - - -

bad data:

2017-01-10 17:01:30 - 43 87.XXX.85.1X2 GET 87.XXX.85.1X2 /i 200 - Mozilla%2F5.0+%28iPhone%3B+CPU+iPhone+OS+10_0_2+like+Mac+OS+X%29+AppleWebKit%2F602.1.50+%28KHTML%2C+like+Gecko%29+Mobile%2F14A432 e=ue&ue_pr={%22schema%22%3A%22iglu%3Acom.snowplowanalytics.snowplow%2Funstruct_event%2Fjsonschema%2F1-0-0%22%2C%22data%22%3A{%22schema%22%3A+%22iglu%3Acom.xpto%2Fopen%2Fjsonschema%2F1-0-4%22%2C%22data%22%3A{%22cid%22%3A%2271229%22%2C%22eid%22%3A%22234636%22%2C%22uid%22%3A%2222139562%22%2C%22geo%22%3A%22BE_NL%22}}}&tv=custom&p=web&cv=clj-1.1.0-tom-0.2.0&nuid=5245d2f2-4876-4764-822d-0d838364c396 - - - - -

If you look closely, in the bad data examples, the querystring shows that the brackets from the schema got decoded.

Then the error returned is:
Exception extracting name-value pairs from querystring … with encoding [UTF-8]: Illegal character in query

Is there an explanation, and more important, a solution to this situation?


#2

Hi @T_P - it looks like you are sending in the brackets from the schema un-encoded. Is there a bug in the process you are using to hand-generate your pixels and click redirects?


#3

Hi @alex,

Every schema is encoded the exact same way:

urlencode('{"schema":"iglu:com.snowplowanalytics.snowplow/unstruct_event/jsonschema/1-0-0","data":{"schema": "iglu:com.xpto/click/jsonschema/1-0-4","data":{"cid":"12345","eid":"12345","uid":"12345","geo":"XX"}}}');

Checking tomcat8 access_log files for processing I’m getting like 1 in every 6 events this way…