Hey @Piyush_Kukadiya Welcome to the Snowplow Discourse!
The Snowplow Collector will pretty much always return a 200, it only returns other errors when things like Pubsub is unavailable or its unhealthy somehow. Are you sending all your events with the official Snowplow trackers? Any custom schemas that could be causing validation failures, or are you using “standard” event types like
The Snowplow pipeline is typically lossless so those missing events should have gone somewhere. I think it’s worth looking at some of the core concepts first to see if we can find them in an expected location.
Usually events go missing when they do not conform to the tracker protocol (hard to do if you’re using an official tracker to send events) or they fail validation as part of the
enrich step. However, those events should all end up in your
bad pubsub topic, and your
bad GCS bucket if you’re loading your
bad rows to GCS with the GCS Loader (typically what we’d recommend).
Also, if the BigQuery Loader can’t write to the table because of a schema mismatch, your events will need to go to a
failed inserts stream where they will be reinserted by the BigQuery Repeater, after the BigQuery Mutator has created the new types. As described here.