Big Data Event
6 min read

Five big ideas to learn at Big Data Tech Warsaw 2020

Hello again in 2020. It’s a new year and the new, 6th edition of Big Data Tech Warsaw is coming soon! Save the date: 27th of February. We have put great effort into gathering our A-team of Big Data experts from top-tier global corporations and open-source companies, willing to share some of their vast knowledge on the latest achievements and new trends in the Big Data industry. Take a look at the highlights of the most interesting pieces we’ve arranged for you this year.

Big Data Technology Warsaw Summit

1. Ways to make large-scale ML actually work

This will be one of the hottest topics on our agenda. For example, Josh Baer will talk about their winding road to better ML infrastructure at Spotify, to make the lives of internal ML practitioners easier and more productive. As you might expect, even companies that have been using ML in their products for many years and have cutting edge ML capabilities, are continuously figuring out how to scale and operate these systems and the associated teams of software engineers involved.

This topic will be also discussed by several guest experts during our main conference panel, as well as roundtable discussions.

2. Building large-scale (real-time) data analytics platforms

Many ML models require fast access to data (e.g. to detect where the nearest taxi driver is or provide personalized product recommendations while browsing a website). This is where real-time data analytics platforms come in.

Reza Shiftehfa will share his thoughts on creating a Big Data platform at Uber to handle hundreds of petabytes with real-time access. The good news is that this platform was built with a mix of open-source technology (e.g. Hadoop, Spark, Hive, Presto, Kafka) as well as tech developed internally at Uber and later on open-sourced such as Hudi and Marmaray. This means that you can also use similar technology and techniques at your own company.

We’ll also host Yuan Jiang, who will describe interactive analytics at Alibaba. Yuan will mainly focus on their large-scale real-time data warehouse. This solution is based on Apache Flink (open-source) and is adopted internally by Search, Recommendation, and Ads products.

It’s worth noting that Apache Flink is also utilized by Humn.AI, a company that offers innovative car insurance calculated by real-time algorithms. Wojciech Indyk will present how their system is built, what the advantages and limitations of Apache Flink are, and how it can be used for use-cases such as detection of a car trip (in real-time), that might look trivial at glance, but expose some traps.

3. Using data and ML to build personalized products

Building advanced ML and real-time data platforms is only one side of the coin. The second one is actually the ability to use this data in a meaningful way.

Disney+ is a brand new streaming service (launched in November 2019) with an impressive subscriber growth. Some of you might not yet know that a team in Warsaw is working on its recommendation system. Grzegorz Puchawski, who is Head of Data Science and Recommendation at Disney Streaming Services, will share problems and lessons learned, that his team have dealt with whilst working on the recommender system, which provides personalized recommendations for ESPN+ and Disney+.

Personalization will be also covered by Tomasz Burzyński and Mateusz Krawczyk who will talk about how they personalize user experience for millions of their customers, using over 20 contact channels at Orange.

4. Migrating from on-premise to the public cloud

While many companies continue building and expanding their on-premise data platforms, we have started seeing more and more companies building hybrid platforms or moving over fully to the Cloud.

You will hear about the exciting journey from our own on-premise to the cloud done together by Truecaller and GetInData. Juliana Araujo, Fouad Alsayadi and Tomasz Żukowski will share with us some exciting tech choices they made, in order to build a robust architecture, lower costs and make their data scientists happier by migrating to the Google Cloud Platform (in a series of a few steps) using a mix of on-prem, hybrid and native cloud technology. Their scale is 150+M active users that generate 30B events a day.

For those who have been using the public cloud for a while, the presentation from Adam Kurowski and Kamil Szkoda (both StepStone) can be extremely useful. They will talk about best DevOps practices in the AWS cloud and will focus on three topics: Distributed data processing, costs optimization and security.

5. Organizing and discovering data in large data lakes

Everything that we are going to talk about during the conference wouldn’t be possible without … data. You can find many analogies between data and oil or gold. Similarly to oil and gold extraction, you also need to have efficient tools to find data. This will be the topic of a joint presentation given by ING and GetInData. Verdan Mahmood and Marek Wiewiórka will talk about how they are building an enterprise-grade data discovery and data lineage at ING. Thanks to this, ING’s data scientists can easily discover available datasets in their large data lake and trust them, thanks to powerful features such as data lineage, data quality and data profiling.

6. BONUS

In this article we’ve only mentioned about 9 presentations, while the agenda includes…33! This means that there are a lot of other useful topics that you will learn about by attending the conference (February 27th, 2020) and you can find them here.

Still not convinced? Watch the video relation from previous edition:


This article was jointly written by Adam Kawa and Mikołaj Wiśniewski.
big data
analytics
conference
Warsaw
technology
bigdatatech
bigdatatechwarsaw
getindata
machine learning
13 February 2020

Want more? Check our articles

transfer legacy pipeline modern gitlab cicd kubernetes kaniko
Tutorial

How we helped our client to transfer legacy pipeline to modern one using GitLab's CI/CD - Part 2

Please dive in the second part of a blog series based on a project delivered for one of our clients. If you miss the first part, please check it here…

Read more
getindata grafana loki monitoring
Use-cases/Project

Why are log analytics so important in a monitoring system?

A monitoring system is a necessary component of any data platform. We can find a lot of different services that use different approaches to the same…

Read more
1YkseCzHNQ9Sxsi4BHnoCOQ
Use-cases/Project

Enabling Hive on Spark on CDH 5.14 — a few problems (and solutions)

Recently I’ve had an opportunity to configure CDH 5.14 Hadoop cluster of one of GetInData’s customers to make it possible to use Hive on Spark…

Read more
geospatial analytics hadoop
Use-cases/Project

Geospatial analytics on Hadoop

A few months ago I was working on a project with a lot of geospatial data. Data was stored in HDFS, easily accessible through Hive. One of the tasks…

Read more
flink
Tutorial

ETL 2.0 Why you should switch into stream processing

If you are looking at Nifi to help you in your data ingestions pipeline, there might be an interesting alternative. Let’s assume we want to simply…

Read more
getindata bigdatatech cfp
Big Data Event

How we evaluate the CfP submissions and build the conference agenda at Big Data Technology Warsaw Summit

Big Data Technology Warsaw Summit 2021 is fast approaching. Please save the date - February 25th, 2021. This time the conference will be organized as…

Read more

Contact us

Interested in our solutions?
Contact us!

Together, we will select the best Big Data solutions for your organization and build a project that will have a real impact on your organization.

By submitting this form, you agree to our  Terms & Conditions