Success Story: Fintech data platform gets a boost from stream processing
A partnership between iZettle and GetInData originated in the form of a two-day workshop focused on analyzing iZettle’s needs and exploring multiple cloud providers’ offerings. Outcomes of this event led to a year-long collaboration on building a robust, third wave data platform.
To ensure undisrupted business growth iZettle was looking for a data platform solution that could meet advanced analytics requirements and address the performance issues caused by rapidly swelling data collection. The platform should minimize the effort spent on maintenance work, allowing specialists to dedicate more time to exploratory data analysis and manufacturing business meaningful insights.
Daily loading jobs were replaced with a streaming ingestion process running on Google DataFlow. Currently, BigQuery takes the role of a central data lake and a query engine. The ingestion process uses an internal message dictionary to validate and route messages to relevant tables. Analytics work is orchestrated with Cloud Composer and utilises BigQuery for SQL and DataFlow for complex scenarios.
The introduction of dynamic streaming ingestion greatly reduced the effort and time required to onboard new sources into a data lake (days instead of weeks).
The new solution ensures that valid and complete information is ready for use ahead of a reporting day.
Thanks to simplified maintenance, teams can focus on data mining and analysis, armed with a wide range of tools that Google Cloud has to offer.
google cloud platform
27 May 2021
Google Cloud Platform Architect
Like this post? Spread the word
Want more? Check our articles
Apache Spark with Apache Iceberg - a way to boost your data pipeline performance and safety
SQL language was invented in 1970 and has powered databases for decades. It allows you not only to query the data, but also to modify it easily on the…