The Big Data Survey Summary
On 5th February, we finished receiving completed forms and we are glad to announce that we have received 76 applications! This means there is not only great interest in Big Data, but also growing interest in the largest purely technical Big Data conference in Poland – Big Data Technology Summit 2018. Our Big Data survey was filled out by people from around the world including the USA, Canada, Turkey, China, India, Germany, Greece, England, Thailand, Spain, Peru and more.
We prepared the Big Data survey for two reasons. First of all, to give people the chance to win a ticket for the Big Data Tech Summit 2018, and secondly to examine how important Big Data is for companies, and what problems they have. Anyone who works or would like to work with large amounts of data could take part in survey.
Volume, importance and technology stack
Almost 60% of participants stressed that Big Data is a mission critical for their business. What’s interesting, is about 30% of these respondents said that the total size of datasets which they work with is between 1-10 TB or even less than 1 TB. On the other hand, in the few companies with 1 PB of datasets, Big Data is not as important as we thought it would be.
Hadoop, Spark and Kafka are still the most commonly used Big Data open-source technologies, but in the participants’ responses we also discovered that NoSQL databases such as Cassandra, Hbase or search engines like Elasticsearch and Solr are also very popular.
We also could see a noticeable adoption of Flink, that is probably the most promising open-source framework for real-time stream processing.
30% of companies are already using a cloud for Big Data, whilst and 21% have plans to implement and use it.
“Real-time processing”, “generating reports” and “personalization” are in the top three, but in many answers, we can also find a “social media analysis” or “large-scale search”.
At GetInData we are big fans of real-time processing and we believe that many companies will combine these use-cases e.g. generate reports in real-time (read more) or personalize search results for each user in real-time like Alibaba does (read more).
We also asked participants about their concerns regarding using Big Data tools. The most respondents stressed that they do not have adequate technical (engineering, operations) or analytical know-how in their companies. There are also concerns about data security and privacy. If your company has a problem with Big Data tools or needs a consultation, please have a look at our work and don’t hesitate to contact us!
More To Come In Warsaw
Thanks to all of you for your interest on our Big Data Survey and your willingness to participate in 2018 Big Data Technology Warsaw Summit. Last but not least a huge congratulations to the winner.
We are sure that each next edition of the conference will bring more and more Big Data experts and audiences from all over the world.
See you on 4th edition of Big Data Tech on the 22nd of February 2018!