Sauna responder configuration

For engineers
1

Hi,

I am playing with sauna 0.1.0 (binary package from bintray) and it seems I am unable to setup SendGrid responder. I have local filesystem observer, working as expected. Configuration as follows:

{
    "schema": "iglu:com.snowplowanalytics.sauna.observers/LocalFilesystemConfig/avro/1-0-0",
    "data": {
        "enabled": true,
        "id": "myLocalObserver",
        "parameters": {
            "saunaRoot": "input_folder"
        }
    }
}

The configuration of the responder is following:

{
    "schema": "iglu:com.snowplowanalytics.sauna.responders/SendgridConfig/avro/1-0-0",
    "data": {
        "id": "mySendgridResponder",
        "enabled": true,
        "parameters": {
            "recipientsEnabled": true,
            "token": "XXXXX"
        }
    }
}

Where XXXXX is an API key generated within Sendgrid (it has all the possible privileges). I place data file in the following dir:

input_folder/com.sendgrid.contactdb/recipients/v1/tsv:email,first_name,last_name,my_field,my_number/test/ge/

Once I start sauna, nothing happens. I have even started to trigger errors (like no email in tsv fields or mock sendgrid host with loopback ip), but nothing happens. What I am doing wrong?

2

Hello @grzegorzewald,

Does it print to stdout that new file is detected?

3

Hi @anton,

Yes, I can see logs form observer (any file with input folder is being picked up almost in real time), but no logs from responders. Even if I break configuration…

Similarly, I had no logs while trying to make Optimizely responder working…

4

Hey @grzegorzewald,

Looks I found a problem - it is in responder’s configuration. Instead of token it should be apiKeyId. You can also find other configuration schemas on Iglu Central.

But it also a bug - Sauna simply ignores invalid configuration files, whereas it should fail immediately. I’ll submit an issue. Thanks for raising.

Please let us know if you have any other issues with Sauna.

UPD

Issue: https://github.com/snowplow/sauna/issues/65

As about Optimizely - I think it is probably same issue. It also explains why it doesn’t show anything on broken configuration.

5

Hi @anton,

Thanks - exchanging token with apiKeyId solved the issue. Sendgrid works. I have been referring iglu schemas, but focused on Optimizely so there is a token field, unfortunately…

There is also issue with Optimizely config - in docs you do not mention about enabled statement. As far as I understand in Avro all the fields are mandatory. Can we have this fixed in documentation as well?

Many thanks

6

You’re right, @grzegorzewald. Thanks for raising, I fixed this in docs.

7

Hi @anton,

I have set of issues. All regard s3 observer:

  1. SQS queue name key in WIKI is sqsQueue whereas IGLU stands sqsQueueName. Of course iglu key name works.
  2. Unfortunately, once I configure S3 bucket, SQS queue and start the process I get flood of errors regarding credentials ((Service: AmazonSQS; Status Code: 403; Error Code: SignatureDoesNotMatch...). The credentials are correct and have all the required privileges (sqs:*). I have tried two regions (eu-west-1 and eu-central-1) and 3 versions of Java (Oracle 8, OJDK 8 and OJDK 7 - this one does not work).

Can you help somehow?

Many thanks

8

Hello @grzegorzewald,

  1. Fixed! Thanks again.
  2. Did you follow Setup Guide precisely and have all same policies and s3:* permissions? I’m asking because we followed this guide and it worked fine. There’s also a more broad guide on AWS docs if you need more details. I think last time I saw SignatureDoesNotMatch was related to not matching regions. Also, what are using Java for? Are you compiling Sauna from the sources? (It was compiled using Oracle JDK 8 and uses some specific for JDK8 stdlib classes).
9

Hi @anton,

I have events configured (can see ObjectPut events in queue). I am using my own user, so i have s3:* and sqs:*. Credentials are ok, as I can use my key/secret with aws cli to list bucket and pull the queue (and list queues). The region is OK - now I use Ireland (so eu-west-1).

I tried both: bintray binary and compiled sauna. Neither works. I started playing with Java versions as I had no idea what can be wrong. Java is in fact required to run Scala code as far as I understand (and downgrading version confirms that).

Update: I have tried with Power User privileges and still the same authentication error. Any thoughts?

10

Hi @anton,

I have a bug in S3 observer code, in credentials object constructor. Have fixed and created a pull request (https://github.com/snowplow/sauna/pull/66).

Now everything works as expected. No exceptions, no bug stack trace.

Waiting for new integrations.

Many thanks.

11

Hello @grzegorzewald,

Thank you very much. Yes indeed, it was a typo and PR will be merged as soon as possible.