Snowflake beginner course

The future of data warehousing has been a popular topic in IT conversations. The outright evolution in technology has seen a recession from on-premise servers and expensive hardware to cloud-based infrastructure devoid of complicated installations. In other words, the entire IT field has been given a facelift.

The move to cloud-based technology in all areas of enterprise software has been trending for years, but migration has never been more pervasive. As modern consumers increasingly take to the convenience of digital experience, companies are progressing through their legacy technology into advanced virtualization and data consumption. Data warehousing (DW), as the backbone of business intelligence (BI) and analytics, is heavily targeted for that initiative.

The future of DW lies in serverless infrastructure. On-premise servers and hardware are becoming antiquated, and as their presence diminishes, the constraints and typical difficulties of acquisition and management go with them.

Usurping the throne of DW are platforms with cloud-based data lake architecture making the real-time data availability simple and efficient. While we contemplate the reality of this new DW environment, one company that comes to mind is Snowflake.

Snowflake has proven to be one of the most compelling players in the game as an up-and-coming leader in Gartner’s Magic Quadrant for Data Management Solutions for Analytics in 2019. Their products have gained traction with companies of all industries because they’re modernizing Data Warehousing-as-a-Service (DWaaS) with real-time analytics in a unique virtual warehouse. The major attraction is its simple, yet powerful, three-layer architecture of database storage, query processing, and cloud services with the ability to scale independently.

When combined with a data lake — for example, Amazon Web Services (AWS) S3, Apache Hadoop, or Microsoft Azure— data is encrypted, compressed, distributed, and geo-redundant, making it exceptionally durable and available for extensible access. Opposed to on-premise warehousing, Snowflake’s virtual warehouse has versatile scaling options and allows for concurrent data loading and querying. Finally, their cloud services alleviate the need for additional resources by including all authentications, sessions, SQL compilation, and more as a part of their offering. (A detailed explanation of Snowflake’s architecture is available here.)

Another reason why Snowflake has become a popular option for DWaaS is that the platform comes as an on-demand run-time service, which means you only pay for the time Snowflake is running. While initial pricing is subject to the amount of data you store, the real savings come from the ability to analyze, report, and perform other business-critical activities without accruing additional fees from an idle, or “paused,” database.

Snowflake is emerging as a strong competitor against the more prominent names in Gartner’s Magic Quadrant — such as Amazon Redshift and Google BigQuery — particularly for their pricing model, which gives companies with smaller budgets an opportunity to leverage a powerful querying system without big name costs. Nonetheless, there’s a time and place for its implementation.

While its cost, architecture, and performance are substantial, there are technical prerequisites that can hinder an immediate migration. Although Snowflake supports a variety of commercial BI tools — for instance, Power BI and Tableau — your specific tool may not be on their radar yet. Additionally, you may not have all your ducks in a row when it comes to data quality and data stacks. Before undertaking DW virtualization, the data pipeline needs to be finely tuned.

Platforms like Snowflake are revolutionizing the way data is stored and consumed. Although there’s still a need for on-premise DW services, the future of DW lives within the vast potential of virtualization. From flexibility and durability to premiere performance and scalability, competing with virtual DWs is becoming harder and harder.

For right now, the technology offered by Snowflake plays as much of a role in modern DW as Teradata and other big-name competitors. The choice between the two depends on your organization’s needs and the style of innovation your team is looking to deploy.