This year, Sky Betting & Gaming sent three architects from the Data Tribe to Berlin Buzzwords to learn all about storing, processing, streaming and searchability of large amounts of digital data. Focusing on open-source projects, it is a great conference for talking to practitioners of big data rather than vendors.
The event started with an afternoon of barcamp sessions, followed by two days of more formal conference split across four rooms and multiple streams - Scale, Search, Stream, Store.
Our favourite talks
Many of the more vendor-driven conferences tend to start with quite abstract keynotes and sales pitches from the sponsors. That is far from true for Berlin Buzzwords, with the keynotes being some of the best talks. Karen Sandler’s story about the importance of free and open-source software for her (Video), and Duncan Ross’s talk about data evangelism (Video) were both very inspiring talks.
Michael Häusler gave a great talk on the integration patterns for big data applications (Video)at Researchgate. I highly recommend watching the video of this, as there are some unique ideas here which seem to work really well.
Lars Francke gave a good overview of securing Hadoop (Video), encouraging people to start with Kerberos authentication right from the start and adding extra security as required by your needs/regulatory requirements.
Frank Lyaruu’s talk on embracing database diversity (Video) reminded us that putting data into Kafka makes it available not just for your first use-case, but for many others. Once user data is being fed through Kafka you can then plug in elasticsearch, key-value stores, and even caching layers and push updates to web sockets.
I would highly recommend next year’s Berlin Buzzwords, especially as it will be combined with a second conference in 2018 - one on governance and management of open source communities.