Data Analyst Training
This four-day course teaches Data Analysts how to analyse massive amounts of data available in a Hadoop YARN cluster.
Participants will gain the ability to effectively work with the huge datasets stored in Hadoop clusters as well an understanding of which processing and analytics needs are best addressed by the individual Big Data frameworks. After the training, participants will be able to independently import data to a Hadoop cluster, store it in Hive tables, use Hive and Spark to transform and analyse the data and Kibana to visualise it.
Introduction to Big Data and Apache Hadoop
Description of the StreamRock along with all its opportunities and challenges that come from the Big Data technologies.
- Hands-on exercise: Accessing a remote multi-node Hadoop cluster.
Introduction to HDFS
- Hands-on exercise: Importing structured data into the cluster using HUE
Introduction to YARN
- Hands-on exercise: Familiarisation with YARN Web UI
A short overview of MapReduce
- Hands-on exercise: Submitting an example ETL map-reduce job to YARN cluster
Providing data-driven answers to business questions using SQL-like solution
Introduction to Apache Hive
- Hands-on exercise: Creating Hive databases and tables using HUE
- Hands-on exercise: Ad-hoc analysis of structured data with HiveQL
Advanced aspects of Hive e.g. partitioning, bucketing, strict-mode, execution plan
- Hands-on exercise: Hive partitioning
Extending Hive with custom UDFs and SerDes
- Hands-on exercise: Using custom Java UDF and SerDe for JSON
Hadoop File Formats (Avro, Parquet, ORC)
- Hands-on exercise: Interacting with Parquet and Avro in Hive
Interactive analysis with Apache Spark
Introduction to Apache Spark
- Spark Core and its advantages over map-reduce
- Basics of working with Spark API
- Spark architecture and integration with YARN
- Hands-on exercise: Interacting with Spark Core API
Doing data analysis with SparkSQL
- Introduction to SparkSQL and DataFrames
- Integration with Hive and other tools
- Introduction to Spark notebooks
- Hands-on exercise: Implementing SparkSQL application to clean the dataset with song records
- Hands-on exercise: Data analysis with SparkSQL and Jupyter
Visualisation and Search
Introduction to ElasticSearch and Kibana
- Most important features of ElasticSearch
- Hands-on exercise: Indexing data with ElasticSearch and visualisations in Kibana
Advanced aspects of working with Spark notebooks
- Hands-on exercise: Visualisation and publishing data with Zeppelin or Jupyter
Completed in half the estimated time and with a fivefold improvement on data collection goals, the robust product has exponentially increased processing capabilities. GetInData’s in-depth engagement, reliability, and broad industry knowledge enabled seamless project execution and implementation.
GetInData had been supporting us in building production Big Data infrastructure and implementing real-time applications that process large streams of data. In light of our successful cooperation with GetInData, their unique experience and the quality of work delivered, we recommend the company as a Big Data vendor.
GetInData delivered a robust mechanism that met our requirements. Their involvement allowed us to add a feature to our product, despite not having the required developer capacity in-house.
Their consistent communication and responsiveness enabled GetInData to drive the project forward. They possess comprehensive knowledge of the relevant technologies and have an intuitive understanding of business needs and requirements. Customers can expect a partner that is open to feedback.
We sincerely recommend GetInData as a Big Data training provider! The trainer is a very experienced practitioner and he gave us a lot of tips regarding production deployments, possible issues as well as good practices that are invaluable for a Hadoop administrator.
The engineers and administrators at GetInData are world-class experts. They have proven experience in many open-source technologies such as Hadoop, Spark, Kafka and Flink for implementing batch and real-time pipelines.
Other Big Data Training
Big Data WorkshopA one-day workshop focused on the practical side of using open-source, Big Data technologies. Participants will learn the basics of the most popular Big Data tools and technologies like: Hadoop, Hive, Spark and Kafka.
Hadoop Developer TrainingThis four-day course gives software engineers a practical introduction to Big Data application development using popular projects from the Hadoop ecosystem and beyond.
Hadoop Administrator TrainingThis four-day course provides the practical and theoretical knowledge necessary to operate a Hadoop cluster. We put great emphasis on practical hands-on exercises that aim to prepare participants to work as effective Hadoop administrators.
Advanced Spark TrainingThis 2-day training is dedicated to Big Data engineers and data scientists who are already familiar with the basic concepts of Apache Spark and have hands-on experience implementing and running Spark applications.
Real-Time Stream ProcessingThis two-day course teaches data engineers how to process unbounded streams of data in real-time using popular open-source frameworks.
Fill out this simple form. Our team will contact you promptly to discuss the next steps.