A Friday afternoon drink at Hoppenbrouers was severely disrupted on July 2, 2021 due to a computer hack. The technical service provider has fallen victim to a global cyber attack on Kaseya software, for which Hoppenbrouwers has just installed an update. “Trojan horse,” CFO Marcel de Boer called the security system afterward. The company has written a hack book on the subject, especially regarding the emotional side of the attack. Marcel de Boer will give a presentation on the book “Hack” on December 1, 2022 during Dutch IT Security Day, where visitors will receive a free copy while stocks last. People can sign up for this now!
Marcel de Boer has been at Hoppenbrouwers Techniek since 1997 and is the Brabant technical service provider that takes care of installation in almost all types of buildings, from homes to offices and from distribution halls to hospitals, for example. The company provides a variety of technical installations, such as climatic or security installations, and also performs industrial automation for production companies, among others. Hoppenbrouwers has grown from 60 to 1,700 employees in recent decades and is now present in twenty sites throughout the Netherlands, not counting all project sites.
De Boer has been CFO since 2008, but also took over the IT department at Hoppenbrouers until early 2022. “So I remember very well the party that broke out on Friday night, July 2, 2021,” he says. “It was about half past six, and the employee could not log in and call the help desk. At first, the on-duty help desk employee assumed there was something trivial, something with a password. But he couldn’t get into the system by himself, the more he searched and searched, the more colleagues there were And partners who did not understand, it became more and more clear: ransomware. Then they called me and our helpline called the insurance company as the first action. In turn, I called Northwave, a security specialist who found out very early in the evening that the problem was with Kaseya. There has been a global attack on this security system, we’ve seen in the news and on the internet.”
It was soon concluded that Hoppenbrouers had a problem not only with central systems, but also with all endpoints in the country. “This means not only computers in our branches and project sites, but also more than 2,000 laptops in employees’ homes,” said De Boer. So the cyber attack was not only a huge challenge from a technical point of view, but also from a logistical point of view. We packed a lot of people that Friday evening so we could pick up all those laptops in a central location the next day. It is somewhat embarrassing that the special script that was already ready for such situations was “on the computer”. So we couldn’t get to that. Then our manager and owner Henny de Haas set the guidelines. As a technology service provider that employs many engineers in industrial automation, we already had enough people standing on our feet on Saturday mornings who know IT and laptops. Northwave has put together a step-by-step plan: What is everyone doing with their laptop? “
Every Saturday Hoppenbrouwers worked with 200 people under the command of Henny de Haas, with status updates hourly. De Boer: “The other 1,300 employees were informed of a tailored webpage. The outside world was also notified during Saturday. You can’t keep something like that quiet, and we’d rather spread the news ourselves.”
One of the most important “challenges” was the centralized system, as De Boer always calls it. How do we restore backups? The hack came on Saturday morning, when IT vendor HPE suggested using a snapshot from Friday noon on their data storage system. We bought this system a few months before the hack and didn’t know it was possible. We can then think about Monday morning: How do we move forward after Friday afternoon’s work is done? What does this mean for our customers, teams and projects? Major systems were back up and running again on Sunday evening. There were, of course, a lot of “loose ends” the following week, but we were basically able to get back to work on Monday morning, without having to pay any ransom. In the end we only lost half a day’s work, and (only) four tons of damage.”
Hoppenbrouwers wrote Hack a book about experiences over the weekend in question. “It’s not a technical book,” says de Boer. “We think it is special that employees check out on Friday afternoon just before the hack and, despite everything, are still able to go back to work on Monday morning. Hack revolves around core values such as collaboration and is primarily intended for the 1,500 employees who They didn’t experience the weekend. They read about the emotional side of the event.”
But there is also interest outside of Hoppenbrouers, even from organizations that De Boer and his colleagues had never heard of. “We had 600 pre-orders before launch. We are now considering a second print, the first print was good for 3,000 books. Don’t forget: small businesses are especially vulnerable. They look quickly at their IT partners, because they will know, they think. But When needed, they discover that this is a really special experience. Who are you calling on this Friday night? Hence our idea of a private phone number: “115, for a hacked company.” Together with Cyberveilig Nederland, we want to bring this message to the attention of politicians in The Hague. “.
Hoppenbrouers have already identified IT as one of the most important business risks, also because of the dependence on the chain. “We were already ISO 27001 certified before the hack, had good online insurance and raised awareness among our employees,” concludes de Boer. “So basically we had everything in order, I thought. But the main conclusion of the hack is that cybersecurity is not just an IT party. It belongs on the agenda of the ultimate responsible person(s) in the organization. If this happens to you, then you should. Also organizing a lot outside the IT department and communicating a lot. Henny de Haas saw that very well. It was hard for him to say: IT should solve it. That’s the main story.”
Marcel de Boer will give a presentation on the ‘Hack’ book on Thursday, 1 December 2022, during Dutch IT Security Day, where visitors will receive a free copy while stocks last. People can sign up for this now!
By: Jeroen Bordewick (Text) and Witold Kepinski (Interview)