Wednesday, December 2, 2015

When software crosses a line


 by Les Hatton and Michiel van Genuchten 
Editor: Mei Nagappan (@MeiNagappan)


We will expand on this story in an upcoming column in our Software Impact Series by which time we should know more details, but in the light of the rapidly unfolding story at Germany's giant automotive company, VW, we will add a new dimension to our original questions “Software: what's in it and what's it in ?” [1] by asking “What's hidden in it and how many people knew ?”.

At the time of writing, it would appear that aside from the normal and burgeoning functionality in the tens of millions of lines of code embedded in modern automotives (for example, Mossinger [2]), there may in some cases be code intended to 'deceive'.  The question, of course, is when does a feature cross the line from what lawyers refer to as harmless “advertiser's puff”, all the way to deceit?

In the case of VW, the change appears to have been tiny – just a few lines of code in what might be millions.  In essence, the software allegedly monitored steering movement whilst running.  On a test harness, the car wheels move but the steering wheel doesn't, in contrast to normal running where both are continually in motion.  By doing this, the software could detect when the car was in test mode and therefore, control the degree to which catalytic scrubbing was done on the emissions.  Catalytic scrubbers inject a mixture of urea and water into the diesel engine emissions converting harmful nitrogen oxides into the more benign molecules nitrogen, oxygen, water and small amounts of carbon dioxide.  The trade-off in a diesel engine is quite simply one of emission toxicity against car performance.  The software, now known as a 'defeat device' simply turned up the efficiency of the catalytic converters when it thought the car was under test.  To date, this is now believed to have been embedded in around 11 million VW cars and some 2 million Audis.

A cynical observer would claim that if somebody can get away with something, then they will but did the engineers responsible really believe that such a device would never be found?  After all, unless you knew what you were looking for, finding it by inspecting the code is comparable to finding a needle in a haystack, and even if you did know, finding your way around a giant software system is not for the faint-hearted.  However, you cannot defeat the laws of physics, or in this case, chemistry.  The VW defeat device was basically discovered by independent monitoring of exhaust emissions with glaring differences being found between what was observed in normal running and what was being claimed so it seems naïve to think it would not be discovered eventually.  In which case, did the engineers responsible think that people wouldn't mind or that the financial benefit of selling more cars would outweigh any potential downside ?  If they did, they are likely to be in for an unpleasant surprise with VW already setting aside several billion dollars to deal with potential claims.

As of 30-Sept-2015 when this part was written, it appears that over 1 million cars and vans could be affected in the UK, Europe's second biggest diesel user after Germany, but VW do not know.  In fact, they do not appear to know if the software is present or if so, whether it is activated, and nobody seems to have considered the possibility of breaking something else if the software is to be removed or even simply deactivated during software recall, due to unintentional side-effects.  These can occur through, for example, shared global variables, or one of a number of mechanisms which will be familiar to professional software engineers.  In short, it's removal could introduce one or more defects.

Speaking of defects, let's raise an interesting question.  Is this better or worse than releasing inadequately tested automotive software in general?  For example, one of the more recent examples of this is the Toyota unintended acceleration bug [3].  They are not alone as there have been numerous recalls in the automotive industry due to software defects.  When a car manufacturer releases such a bug whilst advertising how safe their cars are, are they not being similarly misleading?  For example, contrast the following two more factually appropriate sentences to cover these  eventualities.

We have adjusted the catalytic converter to behave more efficiently if you drive at constant speed without moving the steering wheel, so your emissions will be much lower.  If you depart from this, you will get better performance but your emissions will be very considerably more noxious.

and

We believe that software innovation is vital in automotive development, however, the systems we release to you are so complicated that they will have defects in them which might sometimes prejudice your safety.  Most of the time, however, we believe they will not.

Would you still buy the car?  It could, of course, be argued that these questions arise from different ethical viewpoints but any software engineer worth their salt will know that the chances of releasing a complicated defect-free software system are effectively negligible.

We await the answers to several obvious questions.  Are any other companies doing this, or if we take a more cynical standpoint, how many?  If not, are they using software practices almost as dubious?  How do we decide what is reasonable given the extraordinary ability of software to give hardware its character?  The CEO has already been replaced but what will happen to the engineers and the managers who were responsible?

We look forward to revisiting the story as more information comes to light.

Contextual Note: The Impact series in IEEE Software describes the impact of software on various industries. Around 30 columns have been published since 2010 by senior technical and business managers from companies companies such as Oracle, Airbus (software in the 380), Hitachi, MicroSoft and Vodafone. Others columns were provided by Cern (software behind the Higgs Boson discovery) and JPL (software in the Mars Lander). Michiel van Genuchten and Les Hatton are editors of the Impact column. The Impact column from Jan-Feb 2016 will contain and updated and more extensive version of this blog.

References

1. Genuchten, M. and Les Hatton (2010). “Software: What's In It and What's It In?”, IEEE Software, Jan/Feb 2010, 27(1): pp. 14-16

2. Mossinger J. (2010) “Software in Automotive Systems”, IEEE Software, 27(2). 

No comments:

Post a Comment