GCP Quick Start pipeline production ready

For engineers GCP pipeline
1

Hi there,

I’ve been able to complete most of the GCP Quick Start Guide. I’m now looking to push this one step further. Reading the Quick Start FAQs there is this question How do I make the pipeline production ready? but the answer is only for AWS. I’d love to have something similar for GCP.

I’m aware that this is an open question and I will probably find some answers here or there but any thoughts would be more than welcome!

1 Like
2

Hi @simonbreton - production ready is a big question but generally it comes to a few key areas (which are mostly cloud agnostic). This is different to just being able to handle higher volume (which you can do by just increasing the size of the instance-groups).

  1. Auto-scaling: Can your pipeline handle shifts in traffic without human intervention (this improves reliability)
  2. Monitoring/Alerting: Have you got sufficient visibility into the pipeline and available metrics to know what “good” looks like and to know when it is not working anymore / requires a human to help it out
  3. Practiced upgrade flow: Do you know how to upgrade components safely without introducing downtime to the system

More broadly than infrastructure you also need to think about how you are managing access to servers, how you have configured the networking layer (i.e. are you using CloudNat to isolate servers from the public internet), and how you are controlling access to the production data you are now saving down to the cloud.

In short production ready means you know what “good” looks like for the infrastructure so you can react when its not, you are confident of your security posture and you have clear control over the data being processed and saved and the access boundaries therein.

Hope this helps!

2 Likes
3

thanks @josh. It helps a lot!

2 Likes