Real-time analytics with Snowflake and dbt

This 2-day training is dedicated to data analysts, analytics engineers & data engineers, who are interested in learning how to build and deploy real-time Snowlake data pipelines.

With the growing demand for real-time information processing, a streaming approach to data pipelines is becoming more and more common. We will use Snowflake's Snowpipe Streaming to ingest a simulated stream, then utilize Dynamic tables to transform and prepare the raw ingested message payloads into ready-for-analytics datasets and create a data product on top of it.

Training outcome

Understand the key data streaming concepts in the context of the Snowflake architecture
Ingest data into Snowflake using Snowflake Snowpipe Streaming
Leverage Dynamic Tables and dbt models for developing an end-to-end Snowflake data streaming analytics project

Course agenda^*

Part 1

Day 1 - Session #1 - Streaming data into Snowflake
- Real-time data processing - key concepts, challenges, limitations
- Ways to ingest data into Snowflake in a streaming manner (Snowflake's Kafka Connector vs. Snowflake Snowpipe Streaming)
- Introduction to Snowflake Snowpipe Streaming and Dynamic Tables
- Recap on dbt concepts and configurations (including the ones dedicated for Snowflake streaming cases) ,
- Processing Snowflake data with dbt - incremental models vs. materialized views - key differences and usage considerations
- Demonstration of a sample streaming application (in Snowflake Native Data Apps or other BI tools)
- Real-time processing monitoring and observability
Day 1 - Session #2 - Development of a streaming app using Snowlake & dbt
- Building an end-to-end streaming application (replication of RDBMS data in real-time using transactions logs) with Snowflake Snowpipe Streaming and dbt
- Dealing with large data volumes & sensitive data (Secure Functions, Secure Views)
- Data quality, monitoring, observability - configuration & troubleshooting
- Hands-on exercises

^* GetInData reserves the right to make any changes and adjustments to the presented agenda.

Klaudia Wachnio

klaudia@getindata.com

off

Testimonials

Completed in half the estimated time and with a fivefold improvement on data collection goals, the robust product has exponentially increased processing capabilities. GetInData’s in-depth engagement, reliability, and broad industry knowledge enabled seamless project execution and implementation.

Wojciech Ptak

CTO

GetInData had been supporting us in building production Big Data infrastructure and implementing real-time applications that process large streams of data. In light of our successful cooperation with GetInData, their unique experience and the quality of work delivered, we recommend the company as a Big Data vendor.

Miłosz Balus

CTO

GetInData delivered a robust mechanism that met our requirements. Their involvement allowed us to add a feature to our product, despite not having the required developer capacity in-house.

Stephan Ewen

CTO

Their consistent communication and responsiveness enabled GetInData to drive the project forward. They possess comprehensive knowledge of the relevant technologies and have an intuitive understanding of business needs and requirements. Customers can expect a partner that is open to feedback.

Wilson Yu Cao

Development Team Manager

We sincerely recommend GetInData as a Big Data training provider! The trainer is a very experienced practitioner and he gave us a lot of tips regarding production deployments, possible issues as well as good practices that are invaluable for a Hadoop administrator.

Mariusz Popko

Platform Manager

The engineers and administrators at GetInData are world-class experts. They have proven experience in many open-source technologies such as Hadoop, Spark, Kafka and Flink for implementing batch and real-time pipelines.

Kostas Tzoumas

CEO

Other Big Data Training

Machine Learning Operations Training (MLOps)
This four-day course will teach you how to operationalize Machine Learning models using popular open-source tools, like Kedro and Kubeflow, and deploy it using cloud computing.
See more
See more
Hadoop Administrator Training
This four-day course provides the practical and theoretical knowledge necessary to operate a Hadoop cluster. We put great emphasis on practical hands-on exercises that aim to prepare participants to work as effective Hadoop administrators.
See more
See more
Advanced Spark Training
This 2-day training is dedicated to Big Data engineers and data scientists who are already familiar with the basic concepts of Apache Spark and have hands-on experience implementing and running Spark applications.
See more
See more
Data Analyst Training
This four-day course teaches Data Analysts how to analyse massive amounts of data available in a Hadoop YARN cluster.
See more
See more
Real-Time Stream Processing
This two-day course teaches data engineers how to process unbounded streams of data in real-time using popular open-source frameworks.
See more
See more
Analytics engineering with Snowflake and dbt
This 2-day training is dedicated to data analysts, analytics engineers & data engineers, who are interested in learning how to build and deploy Snowflake data transformation workflows faster than ever before.
See more
See more
Modern Data Pipelines with DBT
In this one day workshop, you will learn how to create modern data transformation pipelines managed by DBT. Discover how you can improve your pipelines’ quality and workflow of your data team by introducing a tool aimed to standardize the way you incorporate good practices within the data team.
See more
See more
Mastering ML/MLOps and AI-powered Data Applications in the Snowflake Data Cloud
This 2-day training is dedicated to data engineers, data scientists, or a tech enthusiasts. This workshop will provide hands-on experience and real-world insights into architecting data applications on the Snowflake Data Cloud.
See more
See more