Hi All,
I am using AWS MSK for Managed Kafka and trying to run Snowplow. I get the following error:
[scala-stream-collector-akka.actor.default-dispatcher-5] INFO com.snowplowanalytics.snowplow.collectors.scalastream.KafkaCollector$ - REST interface bound to /0.0.0.0:80
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 ([MYCLUSTERHOST](http://MYCLUSTERHOST.kafka.us-east-1.amazonaws.com)/10.44.1.27:9096) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue.
My Config is:
sink {
enabled = kafka
brokers = "MYCLUSTERHOST.kafka.us-east-1.amazonaws.com:9096"
retries = 0
producerConf {
"sasl.jaas.config" = "org.apache.kafka.common.security.plain.PlainLoginModule required username='myuser' password='mypassword';"
"security.protocol" = "SASL_SSL"
"sasl.mechanism" = "SCRAM-SHA-512"
}
}
I am able to produce and consumer with these details and network connectivity exists e.g.
nc -zv MYCLUSTERHOST.kafka.us-east-1.amazonaws.com 9096
Connection to MYCLUSTERHOST.kafka.us-east-1.amazonaws.com 9096 port [tcp/*] succeeded!
Any ideas. I saw a previous post similar issue - but that was a typo they had on sasl.mechanism. Mine is set correctly.
Best,
Wayne