Failed to load recovery configuration for iglu:com.snowplowanalytics.snowplow.badrows/schema_violations/jsonschema/2-0-0. No schema-configuration mapping found

For engineers
1

When I set up the GCP environment with QuickStart. There is a problem with my bad data recovery
my docker run :

sudo docker run   snowplow/snowplow-event-recovery-beam:0.4.1-21-56f1f089-20220607-1024-SNAPSHOT    --runner=DataFlowRunner  --region=asia-southeast1    --job-name=snowplow_recovery    --project=snowplow-335307    --gcpTempLocation=gs://snowplow_temp1/recovery_temp    --inputDirectory=gs://snowplow_raw/**    --outputTopic=projects/snowplow-335307/topics/sp-enriched-bad-self-topic    --failedOutput=gs://snowplow_unrecovered/failed/    --unrecoverableOutput=gs://snowplow_unrecovered/unrecovered    --config=***    --resolver=***

resolver-config.json :

{
    "schema":"iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-3",
    "data":{
       "cacheSize":0,
       "repositories":[
          {
             "name":"Iglu-Central",
             "priority":1,
             "vendorPrefixes":[
                "com.snowplowanalytics"
             ],
             "connection":{
                "http":{
                   "uri":"http://iglu-central.com"
                }
             }
          },
          {
             "name":"Custom Iglu Server",
             "priority":0,
             "vendorPrefixes":[
                "com.snowplowanalytics"
             ],
             "connection":{
                "http":{
                   "uri":"http://34.***.***.**:80/api",
                   "apikey":"0db949ca-4ac6-4763-85e4-75ed7475a25c"
                }
             }
          }
       ]
    }
 }

job-config.json:

{
    "schema": "iglu:com.snowplowanalytics.snowplow/recoveries/jsonschema/4-0-0",
    "data": {
      "iglu:com.snowplowanalytics.snowplow.badrows/schema_violations/jsonschema/2-0-*":[
        {
          "name": "modify-own",
          "conditions": [
            {
            "op": "Test",
            "path": "$.data.failure.messages.[0].schemaKey",
            "value": {
            "regex": "iglu:com.snowplowanalytics/own_21/"
            }
          }],
          "steps": [{
            "op": "Replace",
            "path": "$.data.payload.enriched.contexts",
            "match": "own_21",
            "value": "own_2"
            } ]
        }
      ]
    }
  }

GCP dataflow can run successfully, but there is such an error in my failure file. I don’t know why. Can you help me?

"failure":{"error":"Failed to load recovery configuration for iglu:com.snowplowanalytics.snowplow.badrows/schema_violations/jsonschema/2-0-0. No schema-configuration mapping found.","configName":null}
2

Hi @Charles_Han,

the most likely explanation is that your condition isn’t matching any data to process, so the recovery app cannot construct the configuration for the recovery flow.

We have some example scripts for validating configurations. But I think your problem is probably in how you define the path: I think it should be “$.failure.message[0].schemaKey” (no data). Can you give that a try?

Also, it looks like you’re pulling a work-in-progress version of the artifact: 0.4.1-21-56f1f089-20220607-1024-SNAPSHOT. You might want to use the latest version 0.5.0?