Software Security vs. Food Safety

My friend works in a large restaurant chain in St-Petersburg. She is pretty high up in the command after all these years. We talk about all sorts of things when we meet up and once she told me about how they have to deal with safety and quality inspections and how bothersome and expensive they are. So I teased her: why don’t they just pay off the officials to get a certificate instead of going through all the trouble? And she answered seriously that that was only good for a fly-by-night business that does not care about clients or reputation.

Food-poisoning a customer would have severe consequences. Their chain has been around for more than ten years, she said, and they do not want to risk any accidents to destroy their reputation and client base. In the long run, she said, they are better off establishing the right procedures and enduring the audits that will help them to protect the health and safety of their clients. And she named three kinds of losses that they are working hard to prevent: direct losses from accidents, loss of customers as the result of a scare and a long-term loss of customer base as the result of reputation and trust decay.

I think there is something we could learn. The software industry has become completely careless in the recent years and the protection of the customer is something so far down the to-do list you can’t even see it. Judging by the customer care, most businesses in the software industry are fly-by-night. And if they are not, what makes them behave like if they are? Is there some misunderstanding in the industry that the security problems do not cause costs, perhaps? Evidently, the companies in the software industry do not care about moral values but let us see how the same three kinds of losses apply. Maybe I can convince you to rethink the importance of customer care for the industry?

Sony PlayStation Network had an annual revenue of $500 million in 2011, with about 30% margin, giving them a healthy $150 million in profit a year. That year, their network was broken into and hackers stole thousands of personal records, including credit card numbers. In the two weeks of the accident Sony has lost about $20 million, or 13% of their profit. When the damage compensations of $171 million kicked in, the total shot to about $191 million, making Sony PlayStation Network lose over $40 million that year. Some analysts say that the long-term damages to the company could run as high as $20 billion. How would you like to work for 40 years just to repay a single security accident? And Sony is a large company, they could take the hit. A smaller company might have keeled over.

And these kinds of things can come completely unexpected from all sorts of security accidents. Thanks to the governments’ pressure we hear about companies suffering financial disadvantage from incidents that used to be ignored. The US Department of Health & Human Services has fined Massachusets Eye & Ear $1.5 million for the loss of a single laptop that contained unencrypted information. “Oops” indeed. The same year UK Information Commissioner’s Office fined Welcome Financial Services Ltd. &150,000 for the loss of two backup tapes. Things are heating up.

Now, the Sony PlayStation Network breach did not only cost Sony money. The company Brand Index, specializing on measuring the company image in the game circles, determined that that year Sony PlayStation image became negative for the first time in the company’s history. The gamers actively disliked the Sony brand after the accident. That was enough to relegate Sony from the position of a leader in the gaming industry to “just a member of the pack”.

More interesting tendencies could be seen in the retail industry. TJX, the company operating several large retail chains, suffered a breach back in 2005, when hackers got away with 45 million credit card records. At that time, the analysts were predicting large losses of sales that never materialized. TJX paid $10 million in settlements of charges and promotion and the sales did not dip.

Fast forward to December 2013, now Target suffers a security breach where 70 million customer records and 40 million credit card numbers are stolen. Target did not appear to be too worried and engaged in the familiar promotion and discount offering tactics. And then the inconceivable happened. The customers actually paid attention and walked away. The total holiday spending dropped by 9.4%, sales were down 1.5% despite 10% discounts and free credit monitoring offerings from the company. As the result, the company’s stock dropped 1.8%. In the scale of Target, we are talking about billions upon billions of dollars.

So, what happened? In 2005, the industry worried but customers did not react. In 2013, the industry habitually did not worry, but customers took notice. Things are changing, even in the market and industry where software security was never of any interest to either shops or customers. People are starting to pay attention.

Now, if we talk about customer trust and industry image, the food industry serves as a pretty good role model. They have a lot of experience stretching back hundreds of years, they must have figured out a few useful things we could think of applying to our software industry. Take the dioxin scare of 2011. The tracing abilities of the food industry allowed them to find easily how the industrial oil got into animal feed and traced it to particular farms. Right away, the chickens and pigs were mercilessly culled at those farms. That’s what we call an accident response all right. In the aftermath, the food industry installed a regulation to require physical separation of industrial and food oil production and created a requirement for the labs to publish Dioxins findings in food samples immediately.

The food industry has learned that they will not be perceived well if they kill their customers. They are making an effort to establish long-term trust. That’s why they have good traceability, they are merciless in their accident response and they quickly establish new rules that help to improve customer confidence. Take the story of the horse meat fraud in 2013, where horse meat was sold as beef across Europe. That was not dangerous for health, that was a fraud to sell cheaper meat instead of more expensive. The food industry traced it all back to origin and found out that the liability for this kind of fraud was insufficient. That even after paying the fines the companies that engaged in this fraud were making a handsome profit. But customer confidence suffered immensely. And the industry took a swift action, the proposal to increase the penalties and take tougher measures was already accepted by the European Parliament on the 14th of January.

What can we learn from the food industry? They have great traceability of products, detection of all sorts of misbehavior and dangerous agents, requirements to publish data. The penalties are kept higher than potential gain and the response is swift and merciless: either recall or destruction of contaminated goods. All of this taken together helps the industry to keep their customers’ trust.

Try to imagine that HTC was required to recall and destroy all those millions of mobile phones that were found to have multiple security vulnerabilities in 2012. Well, HTC did not waltz away easily as happened in so many cases before. They had to patch up those millions of mobile phones, pass an independent security audit every two years, and, perhaps most telling, they are obliged to tell truth and nothing but the truth when it comes to security.

And this kind of thing will happen more and more often. The customers and governments take interest in security, they notice when something goes wrong and we have a big problem on our hands now, each company individually and the industry as a whole. We will get more fines, more orders to fix things, more new rules imposed and so on. And you know what? It will all go fast, because we always claim that software is fast, it is fast to produce software, make new technology, the innovation pace and all that. People and organizations are used to thinking about he software industry as being fast. So we will not get much advanced notice. We will just get hit with requirements to fix things and fix them immediately. I think it would do us good to actually take some initiative and start changing things ourselves at the pace that is comfortable to ourselves.

Or do we want to sit around and wait the crisis to break out?