Are there still data cube sql files available

[christophe]

Those are all good questions @ryanrozich!

That documentation will need to replaced - I created a ticket for us to do so: #2932.

@ryanrozich

(1) Is the purpose of materializing the views to speed up query execution as opposed to just having them be dynamic views?

That’s indeed the main reason to have materialized views. There few others, which were mentioned in this post: Should I use views in Redshift?

@ryanrozich

(2) Do you have any benchmarks on how much additional cluster resources (disk, etc) it would require to run all of the data cube DDL files you posted? Specifically how much we should allocate on top of what is loaded and stored in the atomic schema.

It shouldn’t have too much of an impact on disk usage (which will increase at most a few percentage points).

@ryanrozich

(3) Whats the recommended way of using these? Is it (a) to drop and recreate the schema every time you want to do a study on web data or (b) Is the intent that we should try to keep these in somewhat in sync with the raw data that is in the atomic schema? If so, should all of these get dropped and recreated in batch on a certain schedule (like every day), or would there be some sort of incremental loading of just new data?

I’d expect most people to run them on a regular schedule. One option is to, for example, have a nightly run. Another one is to run them each time a new batch of events gets loaded into Redshift. If you’re a Looker user, you can use Looker variant of the model. If you use our SQL Runner application to schedule SQL, you can use this version instead.

On the last point:

@ryanrozich

or would there be some sort of incremental loading of just new data?

The current version doesn’t do that, but we plan to make that possible. We have written a tutorial on this topic: Making SQL data models incremental to improve performance [tutorial]

Hope this helps!

Christophe