diginomica recently profiled how German holiday giant TUI is using Snowflake’s ‘data-as-a-service’ platform to give business users timely access to business-critical data, which runs on the company’s AWS stack.
Now, more details have emerged on how the organization prepares all the masses of data it needs to feed part of that data warehouse: its use of ETL (Extraction, Transformation, and Load) across its hotels business, at scale.
As the firm’s Head of Analytics, Kai Wolffram, explains, TUI owns and operates a range of leisure businesses.
This includes travel agencies, six airlines, several cruise lines and retail outlets, as well as a hotel business concentrated on longer-stay, resort travel.
The latter now covers over 400 properties worldwide, made up of a number of different brands (e.g., TUI Blue, Robinson, and Iberotel).
But when he joined the division five years ago, there was no centralized way of looking at their performance. Wolffram says:
Just one of the brands had something like a data warehouse, but it was on-premise and not very detailed. Every brand was doing it on their own; some were advanced, some very much at the beginning of their data journey, and some didn’t have anything.
There was a huge demand for reporting across the division on one platform. Now, our hotel data warehouse is one of our most important systems, and our single source of truth.
What happened in the intervening five years is that Munich-based Wolffram assembled both an internal TUI hotels analytics team and the right technology to create and stand-up such a service.
A key driver on the tech side was to identify cloud-native solutions that would deliver some quick wins, as he didn’t want to invest too much time in educating his developers and then building and maintaining a system in-house.
At the same time, the hotels team didn’t want to migrate or extend any of the separate in-house solutions for a possible new analytics platform for the division, and instead ‘start from scratch’ with a greenfield approach.
An important component of any proposed unified hotels cloud data warehouse would, he says, be a good way to source and properly prepare the separate brand’s data inputs. This would involve a performant ETL tool.
Another consideration, as Snowflake had already been identified as the main group data warehouse preferred solution, would be that such an ETL tool would have to offer full Snowflake integration. He says:
We didn’t want to have technology silos – that was very important to us. But I also wanted to start small and grow as we wanted to; I didn’t want to engage with a partner and sign a 36-month contract upfront and pay half a million of licences before we’d even started.
Jetting off with Matillion
Wolffram says he found the ideal candidate for all these needs in a Snowflake Premier Partner data transformation for cloud data warehouses tool called Matillion.
Even better, the tool was up and running within five minutes, with test jibs ready in hours and in production mode, processing volumes of more than 150 million rows a day were soon being achieved—levels that Wolffram says is now more like “billions.”
He explains:
That sounds too good to be true, but it really was like that. Of course, we had tested it and run a proof of concept, but we were absolutely able to show very, very good results in a very short time.
I think that is one of the strengths of the product is because everything is in the cloud: you just spin up in instances and you more or less start immediately, it provides you with all the features that you need, it’s very intuitive and so didn’t we didn’t need to send staff on any three months training courses to understand it.
Getting this quickly up to speed also meant that it was very easy to engage TUI hotel business users, as the team was able to show results straight away in days.
As a result, he said, users were soon coming directly to him asking for specific questions to be run on the system.
Productivity has also been enhanced by the product’s built-in integrations. He says:
With other ETL tools, the development cycle is rather long, and it can take weeks to attach new data sources to your warehouse. Bit this has pre-built in connectors to most of the data tools we were already using, from Google Analytics to Salesforce; you just use the connectors that are already there, and off you go.
As stated, this form of ETL-powered cloud data warehouse is a major part of what Wolffram sees as “digitized data-driven decision making” in his part of the company.
An example is that TUI managers can now get detailed insight if a property is not performing well, which enabled them to look at changing day rates for certain room types. He adds:
My division isn’t about planes flying around the world that need to send their status every five seconds; in hospitality, most of our reporting is daily, though some use cases also happen more frequently throughout the day.
The point is that our guys cannot make their decisions accurately without having the analytical insights that provide them the data to properly answer the questions they have.
The cloud aspect of the system also became highly relevant during COVID-19, when TUI’s business model was placed under pressure. He says:
Not surprisingly, COVID hit us very hard. Speaking honestly, we were a company without revenue for two years because people weren’t able or weren’t allowed to travel.
Everybody talks about scaling up in the cloud, but that also needs to work the other way round – and that was something we were able to do in very, very difficult times for the company.
In terms of next steps, Wolffram is happy that his ETL function is stable and meeting all his current data warehouse needs. He concluded,
Of course, there are regular releases that provide additional functionality that improve already existing functionalities, and I think that is good and shows me that there is a lot of R&D investment here.
