Success Stories
2 min read

Success Story: Fintech data platform gets a boost from stream processing

A partnership between iZettle and GetInData originated in the form of a two-day workshop focused on analyzing iZettle’s needs and exploring multiple cloud providers’ offerings. Outcomes of this event led to a year-long collaboration on building a robust, third wave data platform.

The challenge

To ensure undisrupted business growth iZettle was looking for a data platform solution that could meet advanced analytics requirements and address the performance issues caused by rapidly swelling data collection. The platform should minimize the effort spent on maintenance work, allowing specialists to dedicate more time to exploratory data analysis and manufacturing business meaningful insights.

izettle getindata strem processing google cloud platform

The solution

Daily loading jobs were replaced with a streaming ingestion process running on Google DataFlow. Currently, BigQuery takes the role of a central data lake and a query engine. The ingestion process uses an internal message dictionary to validate and route messages to relevant tables. Analytics work is orchestrated with Cloud Composer and utilises BigQuery for SQL and DataFlow for complex scenarios.

The results

  • The introduction of dynamic streaming ingestion greatly reduced the effort and time required to onboard new sources into a data lake (days instead of weeks).
  • The new solution ensures that valid and complete information is ready for use ahead of a reporting day.
  • Thanks to simplified maintenance, teams can focus on data mining and analysis, armed with a wide range of tools that Google Cloud has to offer.
streaming
big data
google cloud platform
google dataflow
stream processing
cloud
BigQuery
27 May 2021

Want more? Check our articles

deploying serverless mlflow google cloud platform using cloud run machine learning getindata notext
Tutorial

Deploying serverless MLFlow on Google Cloud Platform using Cloud Run

At GetInData, we build elastic MLOps platforms to fit our customer’s needs. One of the key functionalities of the MLOps platform is the ability to…

Read more
getindata amundsen feast machine learining notext
Tutorial

Machine Learning Features discovery with Feast and Amundsen

One of the main challenges of today's Machine Learning initiatives is the need for a centralized store of high-quality data that can be reused by Data…

Read more
runningkedroeverywhereobszar roboczy 1 4
Tutorial

Running Kedro… everywhere? Machine Learning Pipelines on Kubeflow, Vertex AI, Azure and Airflow

Building reliable machine learning pipelines puts a heavy burden on Data Scientists and Machine Learning engineers. It’s fairly easy to kick-off any…

Read more
1 RsDrT5xOpdAcpehomqlOPg
Big Data Event

2³ Reasons To Speak at Big Data Tech Warsaw 2020 (February 27th, 2020)

Big Data Technology Warsaw Summit 2020 is fast approaching. This will be 6th edition of the conference that is jointly organised by Evention and…

Read more
why do big data project fails
Tutorial

Why do Big Data projects fail: Part. 2. The Technological Issues.

In the previous post on our Big Data Blog, we discussed the business reasons behind the failures of Big Data projects. We've listed five major…

Read more
datagenerationobszar roboczy 1 4
Tutorial

Data online generation for event stream processing

In a lot of business cases that we solve at Getindata when working with our clients, we need to analyze sessions: a series of related events of actors…

Read more

Contact us

Interested in our solutions?
Contact us!

Together, we will select the best Big Data solutions for your organization and build a project that will have a real impact on your organization.

The administrator of your personal data is GetInData Poland Sp. z o.o. with its registered seat in Warsaw (02-508), 39/20 Pulawska St. Your data is processed for the purpose of provision of electronic services in accordance with the  Terms & Conditions. For more information on personal data processing and your rights please see  Privacy Policy.

By submitting this form, you agree to our Terms & Conditions and Privacy Policy