Saturday, 20 July 2019

Creating a year month dimension

I am kind of going to build my ETL from the ground up as a learning experience. As such I don't really have a specification and I am creating things on the fly. I decided to start by creating a year month dimension. 

I started as in all likelihood, no matter what data set I end up with, a dimension for year and month is bound to be useful. In my professional experience this is always required, everyone wants their data split by month, year or year to date etc. I don't need many columns here and it is something I can generating using just code. 

Next step after creating this will be to create the initial parts of the ETL. 1) The ETL engine to process the results of the view into a staging table and from there into a dimension table. Then 2) to create a control environment to ensure that the data I am putting in looks right, after all data without data quality is worthless. 


No comments:

Post a Comment