I want to process the files generated by the S3 Loader, so I’m trying to make sense of the data that is there. I’ve struggled a bit to find some documentation on the topic. When I found this schema in the iglu repository, I thought it was all that I needed, sadly the number of columns were not the same (128 in the schema, 131 in the TSV files).
I also found this issue in which all the 131 properties are mentioned and it seems to match the data in the file. The 3 extra columns that are not included in the iglu schema are
Is it just that the iglu schema is not up to date to reflect those properties? Where can I find the definite source of truth regarding the files schema? Is there any documentation that I missed for the S3 Loader or Enrich applications where this is stated?