Data not processing in Stream Enrich


#1

Hi All, i have the following set up: Application --> Snowplow scala Collector --> RAW Stream (Kinesis Stream) --> Stream Enrich --> Good events and Bad events (Kinesis streams)
The application is an android app which sends streaming data in json format to the collector. We are using the custom iglu schema in the Stream enrich. The incoming data matches with this schema and the data is validated and sent downstream. This solution works fine and the data is available in our destination which is redshift. However when we use a Javascript tracker in an html page (calling the javascript function on a button click to pass the data onto Collector) the data is stuck with RAW stream and does not show up in the Stream Enrich. Please can someone help.

Below is the config.hocon file for the stream enrich.

enrich {

  streams {

    in {
      raw = DW-SnowplowRaw-staging01

    }

    out {
      enriched = DW-SnowplowGoodStream-staging01
      bad = DW-SnowplowBadStream-staging01
      partitionKey = event_id
    }

    # Configuration shown is for Kinesis, to use another uncomment the appropriate configuration
    # and comment out the other
    # To use stdin, comment or remove everything in the "enrich.streams.sourceSink" section except
    # "enabled" which should be set to "stdin".
    sourceSink {
      enabled = kinesis
      region = us-east-1

      aws {
        accessKey = "iam"
        secretKey = "iam"
      }

      maxRecords = 10000
      initialPosition = LATEST

      # Need to be specified when initial-position is "AT_TIMESTAMP".
      # Timestamp format need to be in "yyyy-MM-ddTHH:mm:ssZ".
      # Ex: "2017-05-17T10:00:00Z"
      # Note: Time need to specified in UTC.
      # initialTimestamp = "{{initialTimestamp}}"

      backoffPolicy {
        minBackoff = 10000
        maxBackoff = 10000
      }
    }

    buffer {
      byteLimit = 100000
      recordLimit = 40
      timeLimit = 1000
    }

    appName = "test"
  }

  # Optional section for tracking endpoints
  #monitoring {
  #  snowplow {
  #    collectorUri = "{{collectorUri}}"
  #    collectorPort = 80
  #    appId = {{enrichAppName}}
  #    method = GET
  #  }
  #}
}