Not able to run "stream enrich" in local server(ubuntu)?

For engineers AWS real-time pipeline
1

stream enrich installation is completed and in the configuration we used source=stdin and sink=stdouterr, but finally in running stream enrich
using the below command

 ./snowplow-stream-enrich-0.10.0 --config enrich.conf --resolver file:/home/hadoop/snowplow/3-enrich/config/iglu_resolver.json --enrichments file:/home/hadoop/snowplow/3-enrich/config/enrichments/ >> stream_enrich_log_file.txt 2>&1

i am getting below error.

{"line":"","errors":[{"level":"error","message":"Error deserializing raw event: Cannot read. 
Remote side has closed. Tried to read 1 bytes, but only got 0 bytes. (This is often indicative of an internal error on the server side. Please check your server logs.)"}],"failure_tstamp":"2017-07-19T14:41:16.575Z"}

please help me out to resolve this error.

2

Hi @sandesh,

Do you pipe collector output to enrichment input? It looks like issue parsing your keyboard entry…

3

Hey @grzegorzewald thanks for the reply.
How to pipe scala stream collector output to stream enrich?
please help me out.

4

You can either pipe

./snowplow-stream-collector-X.XX.X --config collector.conf | ./snowplow-stream-enrich-0.10.0 --config enrich.conf --resolver file:/home/hadoop/snowplow/3-enrich/config/iglu_resolver.json --enrichments file:/home/hadoop/snowplow/3-enrich/config/enrichments/ >> stream_enrich_log_file.txt 2>&1

Or use named system pipes (look at snowplow mini docker files)

5

Even after doing pipe using below

./snowplow-stream-collector-0.9.0 --config collector.conf | /home/hadoop/snowplow/3-enrich/stream-enrich/target/scala-2.10/snowplow-stream-enrich-0.10.0 --config /home/hadoop/snowplow/3-enrich/stream-enrich/target/scala-2.10/enrich.conf --resolver file:/home/hadoop/snowplow/3-enrich/config/iglu_resolver.json --enrichments file:/home/hadoop/snowplow/3-enrich/config/enrichments/ >> stream_enrich_log_file.txt 2>&1

it is giving below error.

 {"line":"155731897scala+stream+collector+akkaactordefault+dispatcher+3INFOakkaeventslf4jSlf4jLogger+Slf4jLoggerstarte","errors":[{"level":"error","message":"Error deserializing raw event: Cannot read. Remote side has closed. Tried to read 1 bytes, but only got 0 bytes. (This is often indicative of an internal error on the server side. Please check your server logs.)"}],"failure_tstamp":"2017-07-20T10:28:02.897Z"}
6

This is log from collector - it is not a valid event. Remove all the logging in collector configuration. Than verify tracker data - if it is a proper snowplow record.

7

I am using javascript tracker and i have added below script in my webpage

<script type="text/javascript">
	;(function(p,l,o,w,i,n,g){if(!p[i]){p.GlobalSnowplowNamespace=p.GlobalSnowplowNamespace||[];
	p.GlobalSnowplowNamespace.push(i);p[i]=function(){(p[i].q=p[i].q||[]).push(arguments)
	};p[i].q=p[i].q||[];n=l.createElement(o);g=l.getElementsByTagName(o)[0];n.async=1;
	n.src=w;g.parentNode.insertBefore(n,g)}}(window,document,"script","//d1fc8wv8zag5ca.cloudfront.net/2.8.0/sp.js","snowplow"));

	window.snowplow('newTracker', 'cf', '172.16.0.157:8080', { // Initialise a tracker
	  appId: '1',
	  cookieDomain: 'com.unilog.analytics'
	});

	window.snowplow('trackPageView');
	</script>

i am getting following encrypted message in scala stream collector console. kindly let us know whether it is “proper snowplow record”?

 19:46:59.854 [scala-stream-collector-akka.actor.default-dispatcher-24] DEBUG 
s.can.server.HttpServerConnection - Dispatching GET request to 
http://172.16.0.157:8080/istm=1500560190784
&e=pv&url=http://localhost:8080/Snowplow/SampleExampleTracker.html&page
=Fixed+Width+2+Blue&tv=js-2.8.0&tna=cf&aid=1
&p=web&tz=Asia/Kolkata&lang=en-US&cs=UTF-
8&f_pdf=1&f_qt=0&f_realp=0&f_wma=0&f_dir=0&f_fla=0
&f_java=0&f_gears=0&f_ag=0&res=1366x768&cd=24&cookie=1&eid=d245e789-f608-
479e-a67c-
ecf2a7f1dae7&dtm=1500560190781&vp=1517x735&ds=1518x736&vid=1&sid=10ff2114-
f343-42a5-a2e1-acd5d694b23c&duid=341c3ce6-0997-46e0-9555-
58e78aac915a&fp=3280538411 to handler Actor[akka://scala-stream-collector/system/IO-
TCP/selectors/$a/8#648658460]

CwBkAAAACzE3Mi4xNi4yLjI2CgDIAAABXWBb3to
LANIAAAAFVVRGLTgLANwAAAAQc3NjLTAuOS4wLXN0ZG91dAsBLAAAAHNNb3ppbGxhLzUuMCAoV2luZG93cyBOVCAxMC4wOyBXaW42NDsgeDY0KSBBcHBsZVdlYktpdC81MzcuMzYgKEtIVE1MLCBsaWtlIEdlY2tvKSBDaHJvbWUvNTkuMC4zMDcxLjExNSBTYWZhcmkvNTM3LjM2CwE2AAAAOGh0dHA6Ly9sb2NhbGhvc3Q6ODA4MC9Tbm93cGxvdy9TYW1wbGVFeGFtcGxlVHJhY2tlci5odG1sCwFAAAAAAi9pCwFKAAAB43N0bT0xNTAwNTYwMTkwNzg0JmU9cHYmdXJsPWh0dHAlM0ElMkYl

MkZsb2NhbGhvc3QlM0E4MDgwJTJGU25vd3Bsb3clMkZTYW1wbGVFeGFtcGxlVHJhY2tlci5odG1sJnBhZ2U9Rml4ZWQlMjBXaWR0aCUyMDIlMjBCbHVlJnR2PWpzLTIuOC4wJnRuYT1jZiZhaWQ9MSZwPXdlYiZ0ej1Bc2lhJTJGS29sa2F0YSZsYW5nPWVuLVVTJmNzPVVURi04JmZfcGRmPTEmZl9xdD0wJmZfcmVhbHA9MCZmX3dtYT0wJmZfZGlyPTAmZl9mbGE9MCZmX2phdmE9MCZmX2dlYXJzPTAmZl9hZz0wJnJlcz0xMzY2eDc2OCZjZD0yNCZjb29raWU9MSZlaWQ9ZDI0NWU3ODktZjYwOC00NzllLWE2N2MtZWNmMmE3ZjFkYWU3JmR0bT0xNTAwNTYwMTkwNzgxJnZwPTE1MTd4NzM1JmRzPTE1MTh4NzM2JnZpZD0xJnNpZD0xMGZmMjExNC1mMzQzLTQyYTUtYTJlMS1hY2Q1ZDY5NGIyM2MmZHVpZD0zNDFjM2NlNi0wOTk3LTQ2ZTAtOTU1NS01OGU3OGFhYzkxNWEmZnA9MzI4MDUz

8

Thanks @grzegorzewald for your response.
stream enrich i can able to run in my local server.
i made couple of changes while running stream enrich.
First i just commented “Monitoring section” in the enrich conf file and secondly rather running in snowplow/3-enrich/stream-enrich/target/scala-2.10/ i just ran in /snowplow path, now i am getting the data in enrich console.

Below is the output of enrich.

CFe23a	web	2017-07-27 10:36:34.423	2017-07-27 10:36:20.563	2017-07-27 10:35:31.453	page_view	1af023af-82d9-4509-850b-c09c2565527b		cf	js-2.8.0	ssc-0.9.0-stdout	kinesis-0.10.0-common-0.24.0		172.16.2.x	3280538411	67624c4e-d036-4efb-b910-77e26c4b4253	1	a31ab699-91b4-4782-82ae-933b509fa673												http://localhost:8080/ChangesHTML/SampleExampleTracker.html	Fixed Width 2 Blue		http	localhost	8080	/ChangesHTML/SampleExampleTracker.html																																										Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36	Chrome	Chrome	59.0.3071.115	Browser	WEBKIT	en-US	1	0	0	0	0	0	0	0	0	1	24	1517	735	Windows 10	Windows	Microsoft Corporation	Asia/Kolkata	Computer	0	1366	768	UTF-8	1499	840												2017-07-27 10:35:56.538			{"schema":"iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-1","data":[{"schema":"iglu:com.snowplowanalytics.snowplow/ua_parser_context/jsonschema/1-0-0","data":{"useragentFamily":"Chrome","useragentMajor":"59","useragentMinor":"0","useragentPatch":"3071","useragentVersion":"Chrome 59.0.3071","osFamily":"Windows","osMajor":null,"osMinor":null,"osPatch":null,"osPatchMinor":null,"osVersion":"Windows","deviceFamily":"Other"}}]}	b3950f77-d380-4ccc-a380-754bd058426c	2017-07-27 10:35:55.478	com.snowplowanalytics.snowplow	page_view	jsonschema	1-0-0	8fbb60c48e8ae1a14bf6ec0993957a83	
CFe23a	web	2017-07-27 10:36:34.766	2017-07-27 10:36:21.137	2017-07-27 10:35:56.537	page_view	7d7acdc6-3d8b-4a36-a4ba-3de195c4f03b		cf	js-2.8.0	ssc-0.9.0-stdout	kinesis-0.10.0-common-0.24.0		172.16.2.x	3280538411	e802b6f9-b065-40c7-ad6d-b1359b922e79	1	b00fda71-8651-46a2-8046-819cbbd9827b												http://localhost:8080/ChangesHTML/SampleExampleTracker.html	Fixed Width 2 Blue		http	localhost	8080	/ChangesHTML/SampleExampleTracker.html																																										Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36	Chrome	Chrome	59.0.3071.115	Browser	WEBKIT	en-US	1	0	0	0	0	0	0	0	0	1	24	1517	735	Windows 10	Windows	Microsoft Corporation	Asia/Kolkata	Computer	0	1366	768	UTF-8	1499	1081												2017-07-27 10:35:57.628			{"schema":"iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-1","data":[{"schema":"iglu:com.snowplowanalytics.snowplow/ua_parser_context/jsonschema/1-0-0","data":{"useragentFamily":"Chrome","useragentMajor":"59","useragentMinor":"0","useragentPatch":"3071","useragentVersion":"Chrome 59.0.3071","osFamily":"Windows","osMajor":null,"osMinor":null,"osPatch":null,"osPatchMinor":null,"osVersion":"Windows","deviceFamily":"Other"}}]}	244dec20-8c33-4f4c-a30c-0d801d0da4e0	2017-07-27 10:36:20.046	com.snowplowanalytics.snowplow	page_view	jsonschema	1-0-0	fbd1db586f8518e897e5183462548a78	
CFe23a	web	2017-07-27 10:36:34.789	2017-07-27 10:36:22.743	2017-07-27 10:35:59.244	page_view	78f7ae64-5c5f-4ef2-9092-8f86344d8739		cf	js-2.8.0	ssc-0.9.0-stdout	kinesis-0.10.0-common-0.24.0		172.16.2.x	3280538411	8faa2bf5-2a90-43b4-8921-aef4bf34f0cc	1	f5bd4359-a297-4ea4-b1f1-a069de3ec6ac												http://localhost:8080/ChangesHTML/SampleExampleTracker.html	Fixed Width 2 Blue		http	localhost	8080	/ChangesHTML/SampleExampleTracker.html																																										Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36	Chrome	Chrome	59.0.3071.115	Browser	WEBKIT	en-US	1	0	0	0	0	0	0	0	0	1	24	1517	735	Windows 10	Windows	Microsoft Corporation	Asia/Kolkata	Computer	0	1366	768	UTF-8	1518	736												2017-07-27 10:35:59.247			{"schema":"iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-1","data":[{"schema":"iglu:com.snowplowanalytics.snowplow/ua_parser_context/jsonschema/1-0-0","data":{"useragentFamily":"Chrome","useragentMajor":"59","useragentMinor":"0","useragentPatch":"3071","useragentVersion":"Chrome 59.0.3071","osFamily":"Windows","osMajor":null,"osMinor":null,"osPatch":null,"osPatchMinor":null,"osVersion":"Windows","deviceFamily":"Other"}}]}	18d49a39-7c43-4784-955b-98443c65122a	2017-07-27 10:36:22.740	com.snowplowanalytics.snowplow	page_view	jsonschema	1-0-0	9efdc98e40444ea7efbf7bcab614a1ba