Snowplow-bigquery-mutator targets wrong project by default

For engineers Storage targets
1

Hi all,

I just noticed what looks like an issue with bigquery mutator and that is that it seems to ignore the project included in the bigquery_config.json file on table creation.
I worked around this by adding the -e envrionment variable below, but is this the expected behaviour?
Maybe a bit of an edge case…

docker run \   
-v $CONFIGDIR:/snowplow/config \
-e GOOGLE_APPLICATION_CREDENTIALS='/snowplow/config/creds.json' \
**-e GCLOUD_PROJECT='target-project' \**
   snowplow-docker-registry.bintray.io/snowplow/snowplow-bigquery-mutator:0.6.0-rc3 \
   create \
  --config $(cat resources/config/bigquery_config.json | base64 -w 0) \
  --resolver $(cat resources/config/iglu_resolver.json | base64 -w 0)
2

Thanks for the headsup on this @rbkn. By any chance, did you notice which project the mutator was targeting by default? Also, have you seen this issue with any stable releases, eg 0.5.1?

3

No probs.
Given that my default credentials are pointing to another project altogether, it looks like it might be using the project from GOOGLE_APPLICATION_CREDENTIALS.
At least, that would be my guess.

I do think I encountered this same issue previously with a stable release but didn’t note it at the time as I was more focussed on getting it to work.

4

Thanks @rbkn. This is slated to be fixed in 0.6.0: https://github.com/snowplow-incubator/snowplow-bigquery-loader/pull/103.

1 Like