One of the broadest uses of Snowflake is building a data warehouse platform or enhancing the existing data lake. It offers all sorts of services to build an efficient Data warehouse with ETL capability and support for various external data partners. Slowly Changing dimensions are a common database modeling technique used to capture data in a table and show how it changes over time. The slowly changing dimension of the warehouse dimension is said to rarely change. However, when they change, there should be a systematic approach to capturing that change. Examples of SCDs are customer and products information. This project explains how to build a Slowly Changing Dimension (SCD) using Snowflake’s Stream functionality and how to automate the process using Snowflake’s Task functionality.
In this project, we will cover in detail the architecture of a transformer used in natural language processing use cases. We will go through the key nlp areas in the pre-transformer stage like bow, word2vec...and then the origin and gradual refinement of transformers. Finally, we will study one of the most popular state of the art transformer models, called BERT and use it for text classification on a large dataset.