Success Stories
2 min read

Success Story: Fintech data platform gets a boost from stream processing

A partnership between iZettle and GetInData originated in the form of a two-day workshop focused on analyzing iZettle’s needs and exploring multiple cloud providers’ offerings. Outcomes of this event led to a year-long collaboration on building a robust, third wave data platform.

The challenge

To ensure undisrupted business growth iZettle was looking for a data platform solution that could meet advanced analytics requirements and address the performance issues caused by rapidly swelling data collection. The platform should minimize the effort spent on maintenance work, allowing specialists to dedicate more time to exploratory data analysis and manufacturing business meaningful insights.

izettle getindata strem processing google cloud platform

The solution

Daily loading jobs were replaced with a streaming ingestion process running on Google DataFlow. Currently, BigQuery takes the role of a central data lake and a query engine. The ingestion process uses an internal message dictionary to validate and route messages to relevant tables. Analytics work is orchestrated with Cloud Composer and utilises BigQuery for SQL and DataFlow for complex scenarios.

The results

  • The introduction of dynamic streaming ingestion greatly reduced the effort and time required to onboard new sources into a data lake (days instead of weeks).
  • The new solution ensures that valid and complete information is ready for use ahead of a reporting day.
  • Thanks to simplified maintenance, teams can focus on data mining and analysis, armed with a wide range of tools that Google Cloud has to offer.
streaming
big data
google cloud platform
google dataflow
stream processing
cloud
BigQuery
27 May 2021

Want more? Check our articles

flink
Tutorial

ETL 2.0 Why you should switch into stream processing

If you are looking at Nifi to help you in your data ingestions pipeline, there might be an interesting alternative.Let’s assume we want to simply dump…

Read more
obszar roboczy 1 100

Towards better Data Analytics - Google Cloud Bootcamp

“Without data, you are another person with an opinion”These words from Edward Deming, a management guru, are the best definition of what means to…

Read more
monitoring data platform
Whitepaper

White Paper: Monitoring and Observability for Data Platform

AboutIn this White Paper, we described a monitoring and observing data platform in case of continuously working processes.What you will find there…

Read more
blog6

5 main data-related trends to be covered at Big Data Tech Warsaw 2021. Part I.

A year is definitely a long enough time to see new trends or technologies that get more traction. The Big Data landscape changes increasingly fast…

Read more
complex event processing apache flink
Tutorial

My experience with Apache Flink for Complex Event Processing

My goal is to create a comprehensive review of available options when dealing with Complex Event Processing using Apache Flink. We will be building a…

Read more
getindata white paper aviation bigdata technologies
Whitepaper

White Paper: Big Data Technologies in the Aviation Industry

AboutIn this White Paper we described use-cases in the aviation industry which are the most prominent examples of Big Data related implementations…

Read more

Contact us

Fill out this simple form. Our team will contact you promptly to discuss the next steps.

hello@getindata.comFist bump illustration

Any questions?

Choose one
By submitting this form, you agree to our  Terms & Conditions