X-Forwarded-For in Cloudfront logs

For engineers Enrichment
1

Hello Snowplowers,

I have recently discovered something that puzzled me: doing the request from a client to a Cloudfront collector and to a Scala Stream collector would result in different user_ipaddress fields after enriching the events.

I checked the contents of both the Cloudfront log line and the Thrift event and found that both had the x-forwarded-for field/header set.

After digging into the source code, I think I found the reason. The ThriftLoader used to load events from the Scala Stream collector uses the x-forwarded-for header to determine the user’s IP, but the CloudfrontLoader ignores the x-forwarded-for field from the log line.

Am I missing something? Would you accept PRs to make CloudfrontLoader use the x-forwarded-for field?

Regards,
Dani

2

I think you are right,
there are some fields added to the cloudfront log, since 29 Apr 2014:

  • x-forwarded-for
  • ssl-protocol
  • ssl-cipher
  • x-edge-response-result-type
  • cs-protocol-version
3

Hi both, good catch @danisola - yes a PR sounds good!