Setup configurable enrichments for ad_impression

For engineers Enrichment
1

I am config and run comand for server enrich stream but give an error:

Please, please help me.

run:

./snowplow-stream-enrich-0.9.0 --config ./enrich.conf --resolver file:resolver.json --enrichments file:/opt/enrichments/ &

Exception in thread "main" java.lang.RuntimeException: NonEmptyList(error: NonEmptyList(error: JSON path name not found
    level: "error"
)
    level: "error"
)
        at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp$$anonfun$10.apply(KinesisEnrichApp.scala:145)
        at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp$$anonfun$10.apply(KinesisEnrichApp.scala:145)
        at scalaz.Validation$class.fold(Validation.scala:64)
        at scalaz.Failure.fold(Validation.scala:330)
        at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp$delayedInit$body.apply(KinesisEnrichApp.scala:144)
        at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
        at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
        at scala.App$$anonfun$main$1.apply(App.scala:71)
        at scala.App$$anonfun$main$1.apply(App.scala:71)
        at scala.collection.immutable.List.foreach(List.scala:318)
        at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)
        at scala.App$class.main(App.scala:71)
        at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp$.main(KinesisEnrichApp.scala:71)
        at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp.main(KinesisEnrichApp.scala)
Exception in thread "Thread-1" akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://snowplow-scala-tracker/user/IO-HTTP#679837131]] after [1000 ms]
        at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:334)
        at akka.actor.Scheduler$$anon$7.run(Scheduler.scala:117)
        at scala.concurrent.Future$InternalCallbackExecutor$.scala$concurrent$Future$InternalCallbackExecutor$$unbatchedExecute(Future.scala:694)
        at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:691)
        at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(Scheduler.scala:467)
        at akka.actor.LightArrayRevolverScheduler$$anon$8.executeBucket$1(Scheduler.scala:419)
        at akka.actor.LightArrayRevolverScheduler$$anon$8.nextTick(Scheduler.scala:423)
        at akka.actor.LightArrayRevolverScheduler$$anon$8.run(Scheduler.scala:375)
        at java.lang.Thread.run(Thread.java:745)
2

Configuraion content my file ad_impression:

{
	"schema": "iglu:com.snowplowanalytics.snowplow/ad_impression/jsonschema/1-0-0",

	"data": {
			"impressionId": "{{impressionId}}",
			"zoneId": "{{zoneId}}",
			"bannerId": "{{bannerId}}",
			"campaignId": "{{campaignId}}",
			"advertiserId": "{{advertiserId}}",
			"targetUrl": "{{targetUrl}}",
			"costModel": "cpm",
			"cost": 7
		}
	}
}
3

Hello @james24h,

Could you please paste your enrich.conf and resolver.json?

4

Sorry @james24h, ignore above post. It seems you have invalid enrichment configuration. More specifically - it missing name parameter. Here’s example: https://github.com/snowplow/snowplow/blob/master/3-enrich/config/enrichments/cookie_extractor_config.json#L4

Also it isn’t clear what you’re trying to achiveve with ad_impression. Instances of this schema should be generated by tracker, not by user. You can find more about ad impression tracking in following post: Ad impression and click tracking with Snowplow.

5

Hi @anton

Yes, I have invalid enrichment configuration for ad_impression:

{
	"schema": "iglu:com.snowplowanalytics.snowplow/ad_impression/jsonschema/1-0-0",

	"data": {
			"impressionId": "{{impressionId}}",
			"zoneId": "{{zoneId}}",
			"bannerId": "{{bannerId}}",
			"campaignId": "{{campaignId}}",
			"advertiserId": "{{advertiserId}}",
			"targetUrl": "{{targetUrl}}",
			"costModel": "cpm",
			"cost": 7
		}
	}
}

=> and schema for it: http://iglucentral.com/schemas/com.snowplowanalytics.snowplow/ad_impression/jsonschema/1-0-0 (http://iglucentral.com/)

I haved config follow but not validate and notify the error at the top.

resolver.json

{
  "schema": "iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-0",
  "data": {
    "cacheSize": 500,
    "repositories": [
      {
        "name": "Iglu Central",
        "priority": 0,
        "vendorPrefixes": [ "com.snowplowanalytics" ],
        "connection": {
          "http": {
            "uri": "http://iglucentral.com"
          }
        }
      }
    ]
  }
}

enrich.conf

enrich {
  source = "kinesis"
  sink = "kinesis"

  aws {
    access-key: "{access-key}"
    secret-key: "{secret-key}"
  }

  streams {
    in: {
      raw: good_guy

      maxRecords: 10000

      buffer: {
        byte-limit: 64000 #6KB
        record-limit: 100 
        time-limit: 30
      }
    }

    out: {
      enriched: rich_good_guy
      bad: rich_bad_guy

      backoffPolicy: {
        minBackoff: 20
        maxBackoff: 60
      }
    }

    app-name: "snowplow.kinesis.enrich"
    initial-position = "TRIM_HORIZON"
    region: {region}
  }

akka {
  loglevel = DEBUG # 'OFF' for no logging, 'DEBUG' for all logging.
  loggers = ["akka.event.slf4j.Slf4jLogger"]
}

  monitoring {
    snowplow {
      collector-uri: "localhost"
      collector-port: 8080
      app-id: "{{enrichAppName}}"
      method: "GET"
    }
  }
}

Thanks!

6

@james24h, your resolver.json and enrich.conf seems to be correct. However, problem is in your ad_impression file (if I understand it correct this is /opt/enrichments/ad_impression.json). You don’t need this file to track ad impressions, instead you need to invoke trackAdImpression method in JavaScript Tracker to send this data to collector.

In the enrichments directory you need to store only valid configuration files like following: https://github.com/snowplow/snowplow/tree/master/3-enrich/config/enrichments. If there would be any unexpected file like your ad_impression.json - parsing will fail.

Tracking ad impressions doesn’t have much common with enrichments, because enrichment is process of adding information to collected data, whereas you need to collect (track) it first.

7

Thanks @anton, I am understand.

and i can custom and add /opt/enrichments/ad_impression.json to validate for data other.

8

Hey @james24h,

You don’t need to add or customize ad_impression.json. Your /opt/enrichments shouldn’t contain anything to track ad impressions.

9

@anton ok, thanks very much :innocent: