How to manually build scala-common-enrich

For engineers Enrichment
1

Hi,

I have added my own adapter to the scala-common-enrich project, but I am unsure how to package it up into a jar so that I can add it into my own ivy repository for when I build the stream-enrich project. If I recall awhile back, there used to be instructions for creating a JAR via SBT for the scala-common-enrich project, but I don’t see those in the readme anymore. What is the best way to build the scala-common-enrich project so that I can easily pull it into the stream-enrich project?

2

Here’s what I would do:

  • increment the version of scala-common-enrich in BuildSettings.scala
  • sbt publishLocal of scala common enrich
  • change the version of common-enrich in scala-hadoop-enrich’s Dependencies.scala
  • sbt assembly of scala-hadoop-enrich
  • upload to your bucket in s3 respecting the folder hierarchy of the default bucket (s3://snowplow-hosted-assets) and change the s3-buckets-assets config key in your emr etl runner configuration file (note that you’ll have to upload scala-hadoop-shred as well, and others if you need them)
2 Likes
3

Works like a charm! Thank you