tl;dr I am trying to use “$ref” and “allOf” in my schema (it passes igluctl lint
), but I am not seeing the jsonpaths or sql output I expect when running igluctl static generate
.
My use case is that I have 40 events, and growing, for a system that we are migrating to use snowplow. Most of these events have over 30 properties, and most of those are required and common across subsets of events.
Given all that, I am interested in cleaning up my event specs, using the notion of composition. I was looking at https://github.com/snowplow/iglu/blob/master/2-repositories/scala-repo-server/src/main/resources/valid-schema.json as a starting point, and it makes use of allOf
and $ref
to assemble the schema.
I feel like I am missing something or doing something wrong. Below is an example schema file that highlights what I am doing and the issue I am seeing.
$ igluctl --version
igluctl 0.2.0
$ cat schemas/co.ga/test_event/jsonschema/1-0-0
{
"$schema": "http://iglucentral.com/schemas/com.snowplowanalytics.self-desc/schema/jsonschema/1-0-0#",
"self": {"vendor": "co.ga", "name": "test_event", "format": "jsonschema", "version": "1-0-0"},
"type": "object",
"allOf": [
{"$ref": "#/definitions/global_properties"},
{"$ref": "#/definitions/item_properties"},
{
"properties": {
"foobar": {
"type": "boolean"
}
}
}
],
"definitions": {
"sha1sum": {"type": "string", "minLength": 40, "maxLength": 40, "pattern": "^[0-9a-f]+$"},
"global_properties": {
"properties": {
"widget": {
"type": ["string", "null"],
"maxLength": 1024
},
"widget_hash": {
"oneOf": [
{"type": "null"},
{"$ref": "#/definitions/sha1sum"}
]
}
}
},
"item_properties": {
"properties": {
"item_id": {
"type": "integer"
}
}
}
},
"additionalProperties": true
}
$ igluctl lint schemas/co.ga/test_event/jsonschema/1-0-0
SUCCESS: Schema [/Users/jon/Projects/github/generalassembly/snowplow-events/schemas/co.ga/test_event/jsonschema/1-0-0] is successfully validated
TOTAL: 1 Schemas were successfully validated
TOTAL: 0 invalid Schemas were encountered
TOTAL: 0 errors were encountered
$ igluctl static generate --with-json-paths schemas/co.ga/test_event/jsonschema/1-0-0 --force
File [/Users/jon/Projects/github/generalassembly/snowplow-events/./sql/co.ga/test_event_1.sql] was overridden successfully (no change)!
File [/Users/jon/Projects/github/generalassembly/snowplow-events/./jsonpaths/co.ga/test_event_1.json] was overridden successfully (no change)!
$ cat ./jsonpaths/co.ga/test_event_1.json
{
"jsonpaths": [
"$.schema.vendor",
"$.schema.name",
"$.schema.format",
"$.schema.version",
"$.hierarchy.rootId",
"$.hierarchy.rootTstamp",
"$.hierarchy.refRoot",
"$.hierarchy.refTree",
"$.hierarchy.refParent"
]
}