Next term
Not scheduled
If you are interested, please contact us
Duration
4-days
Target audience
Engineers
Technology
e.g. Hadoop, Hive, YARN, Spark, Kafka, Sqoob
icon workshop variety

Hadoop Developer Training

This four-day course gives software engineers a practical introduction to Big Data application development using popular projects from the Hadoop ecosystem and beyond.

Training outcome

Participants will gain a detailed understanding of the architecture and role of the most important technologies from the Hadoop Ecosystem. They will be able to independently load and transform huge datasets with the help of technologies like Hive, Spark, Sqoop, Kafka and Oozie.

Course agenda*

Day 1

Introduction to Big Data and Apache Hadoop

  • Description of StreamRock along with all its opportunities and challenges that come from Big Data technologies.

    • Hands-on exercise: Accessing a remote multi-node Hadoop cluster.

  • Introduction to HDFS

    • Hands-on exercise: Importing structured data into the cluster using HUE
    • Hands-on exercise: Interacting with HDFS using HDFS CLI, Snakebite and WebHDFS

  • Introduction to YARN

    • Hands-on exercise: Familiarisation with YARN Web UI

  • A short overview of MapReduce

    • Hands-on exercise: Submitting an example ETL map-reduce job to YARN cluster
Day 2

Providing data-driven answers to business questions using SQL-like solution

  • Introduction to Apache Hive

    • Hands-on exercise: Creating Hive databases and tables using HUE
    • Hands-on exercise: Ad-hoc analysis of structured data with HiveQL

  • Advanced aspects of Hive e.g. partitioning, bucketing, strict-mode, execution plan

    • Hands-on exercise: Hive partitioning

  • Extending Hive with custom UDFs and SerDes

    • Hands-on exercise: Using custom Java UDF and SerDe for JSON

  • Hadoop File Formats (Avro, Parquet, ORC)

    • Hands-on exercise: Interacting with Parquet and Avro in Hive
Day 3

Implementing scalable ETL processes on the Hadoop cluster

  • Introduction to Apache Spark, Spark SQL, and Spark DataFrames

    • Hands-on exercise: Implementation of the ETL job to clean and massage input data using Spark
    • Hands-on exercise: Implementing ad-hoc queries using Spark SQL and DataFrames
    • Hands-on exercise: Visualisation of the results of Spark queries using the Spark Notebook

  • Bonus: Overview of Fast-SQL on Hadoop – solutions like Hive, Spark SQL, Impala, Presto and Tez

Day 4

ETL-related technologies

  • Introduction to Apache Sqoop

    • Hands-on exercise: Importing structured data from MySQL to HDFS and Hive using Sqoop

  • Real-time data collection with Apache Kafka

    • Hands-on exercise: Interacting with a Kafka Cluster to produce and consume messages with CLI scripts
    • Hands-on exercise: Using Kafka Java Producer with Avro Schema Registry

  • Introduction to Apache Oozie

    • Hands-on exercise: Building and executing Oozie workflow
    • Hands-on exercise: Scheduling Oozie workflow with Oozie scheduler
* GetInData reserves the right to make any changes and adjustments to the presented agenda.

Instructors

Our workshops and training programmes are organised by experienced instructors with many years’ of real-life Big Data experience. Get to know our team!

More information

Training materials will be made available to all participants in PDF format.

Contact person

Klaudia Wachnio
off
Piotr Krewski
+48 888 185 137

Testimonials

Completed in half the estimated time and with a fivefold improvement on data collection goals, the robust product has exponentially increased processing capabilities. GetInData’s in-depth engagement, reliability, and broad industry knowledge enabled seamless project execution and implementation.

Wojciech Ptak
CTO

GetInData had been supporting us in building production Big Data infrastructure and implementing real-time applications that process large streams of data. In light of our successful cooperation with GetInData, their unique experience and the quality of work delivered, we recommend the company as a Big Data vendor.

Miłosz Balus
CTO

GetInData delivered a robust mechanism that met our requirements. Their involvement allowed us to add a feature to our product, despite not having the required developer capacity in-house.

Stephan Ewen
CTO

Their consistent communication and responsiveness enabled GetInData to drive the project forward. They possess comprehensive knowledge of the relevant technologies and have an intuitive understanding of business needs and requirements. Customers can expect a partner that is open to feedback.

Wilson Yu Cao
Development Team Manager

We sincerely recommend GetInData as a Big Data training provider! The trainer is a very experienced practitioner and he gave us a lot of tips regarding production deployments, possible issues as well as good practices that are invaluable for a Hadoop administrator.

Mariusz Popko
Platform Manager

The engineers and administrators at GetInData are world-class experts. They have proven experience in many open-source technologies such as Hadoop, Spark, Kafka and Flink for implementing batch and real-time pipelines.

Kostas Tzoumas
CEO

Other Big Data Training

  • Machine Learning Operations Training (MLOps)

    Machine Learning Operations Training (MLOps)

    This four-day course will teach you how to operationalize Machine Learning models using popular open-source tools, like Kedro and Kubeflow, and deploy it using cloud computing.
    See more
    See more
  • Hadoop Administrator Training

    Hadoop Administrator Training

    This four-day course provides the practical and theoretical knowledge necessary to operate a Hadoop cluster. We put great emphasis on practical hands-on exercises that aim to prepare participants to work as effective Hadoop administrators.
    See more
    See more
  • Advanced Spark Training

    Advanced Spark Training

    This 2-day training is dedicated to Big Data engineers and data scientists who are already familiar with the basic concepts of Apache Spark and have hands-on experience implementing and running Spark applications.
    See more
    See more
  • Data Analyst Training

    Data Analyst Training

    This four-day course teaches Data Analysts how to analyse massive amounts of data available in a Hadoop YARN cluster.
    See more
    See more
  • Real-Time Stream Processing

    Real-Time Stream Processing

    This two-day course teaches data engineers how to process unbounded streams of data in real-time using popular open-source frameworks.
    See more
    See more
  • Analytics engineering with Snowflake and dbt

    Analytics engineering with Snowflake and dbt

    This 2-day training is dedicated to data analysts, analytics engineers & data engineers, who are interested in learning how to build and deploy Snowflake data transformation workflows faster than ever before.
    See more
    See more
  • Mastering ML/MLOps and AI-powered Data Applications in the Snowflake Data Cloud

    Mastering ML/MLOps and AI-powered Data Applications in the Snowflake Data Cloud

    This 2-day training is dedicated to data engineers, data scientists, or a tech enthusiasts. This workshop will provide hands-on experience and real-world insights into architecting data applications on the Snowflake Data Cloud.
    See more
    See more
  • Modern Data Pipelines with DBT

    Modern Data Pipelines with DBT

    In this one day workshop, you will learn how to create modern data transformation pipelines managed by DBT. Discover how you can improve your pipelines’ quality and workflow of your data team by introducing a tool aimed to standardize the way you incorporate good practices within the data team.
    See more
    See more
  • Real-time analytics with Snowflake and dbt

    Real-time analytics with Snowflake and dbt

    This 2-day training is dedicated to data analysts, analytics engineers & data engineers, who are interested in learning how to build and deploy real-time Snowlake data pipelines.
    See more
    See more

Contact us

Interested in our solutions?
Contact us!

Together, we will select the best Big Data solutions for your organization and build a project that will have a real impact on your organization.

What did you find most impressive about GetInData?

They did a very good job in finding people that fitted in Acast both technically as well as culturally.
Type the form or send a e-mail: hello@getindata.com
The administrator of your personal data is GetInData Poland Sp. z o.o. with its registered seat in Warsaw (02-508), 39/20 Pulawska St. Your data is processed for the purpose of provision of electronic services in accordance with the Terms & Conditions. For more information on personal data processing and your rights please see Privacy Policy.

By submitting this form, you agree to our Terms & Conditions and Privacy Policy