Have hit the end of the road of google/forum searches, and internal resource knowledge so time to post…
Step 1 - install collector
- Snowplow Scala collector installed on an AWS EC2 instance
- config for the collector using private EC2 IP and port 8000
- TEST: Load http://10…167:8000/com.snowplowanalytics.snowplow/tp2 in a browser shows the pixel - OK
Step 2 - ELB + HTTP
- Setup load balancer (ELB) on AWS
- configure listener on 80 (HTTP)
- setup target on 8000
- create CNAME for new subdomain pointing at ELB.
- TEST: GET request via browser to http://subdomain.example.com/com.snowplowanalytics.snowplow/tp2 - OK
Step 3 - ELB + HTTPS
- configure new listener on 443 (HTTPS) using SSL certificate
- setup target on 8000
- add security group rules to allow 443 traffic.
- TEST: GET request via browser to https://subdomain.example.com/com.snowplowanalytics.snowplow/tp2 - 502 Bad Gateway
Troubleshooting so far
- Have triple-checked security groups (OK) - allowing traffic from 0.0.0.0/0 for EC2 and ELB security groups have no affect - no network ACL issues either
- certificate is valid
- enable/disable HTTP/2 at ELB has same 502 result
- check ELB access logs display no response back from target host for requests coming in as HTTPS
- The same request is being made to the target host on port 8000 for HTTP and HTTPS from ELB, but the HTTP request receives back 200 status from target whereas the HTTPS request receives nothing resulting in 502 response from the load balancer.
- stdout/stderr logs for collector don’t show any results when HTTPS request is made, but do show for HTTP (eg.
INFO com.snowplowanalytics.snowplow.collectors.scalastream.sinks.KinesisSink - Successfully wrote 1 out of 1 records)
- changed config to interface=“0.0.0.0”
INFO com.snowplowanalytics.snowplow.collectors.scalastream.KinesisCollector$ - REST interface bound to /0:0:0:0:0:0:0:0:8000
- same results - http OK, https 502
At this point we’re pretty much stumped because everything appears to be setup correctly. Our working theory is that the ELB is sending encrypted request to the collector and not terminating SSL at ELB but not sure how to prove/disprove this - are there additional logs somewhere we can see the incoming request to the collector?
Any input/questions/comments are greatly appreciated.