Information Loss Leads to Robustness 

W.B. Langdon, J. Petke, D. Clark
CREST, Department of Computer Science, University College, London 

It is tacitly accepted that real software is far from perfect. That everyday computer systems deliver real economic advantage to their users even though they contain both documented and undocumented bugs. Often computer scientists are embarrassed by this imperfection and teach students that programs should be free from errors. Yet software quality managers know the fastest way to bankrupt their company is to prevent it shipping code until all its errors have been corrected. We turn to information theory to explain why software is resilient to both source code and run time errors and often yields a usable answer despite its flaws [1].

During the second world war Claude Shannon of Bell Labs, New Jersey, formalised the notion of information, defining the amount of information using entropy. The information content of a message defines the minimum number of bits needed to code it (e.g. in order to send it via a telephony link).

Pretty much every operation in a digital computer is irreversible. Meaning that its inputs cannot be determined from its output. From an information theory point of view, every irreversible operation loses information.
Figure 1 shows computer operations as information funnels, with more information going into their wide mouths than leaves via their output.

Figure 1 shows two numbers being added together (addition is irreversible). To make the example more concrete, we assume the two inputs are independent single digits (1–9, using Benford’s law), and therefore their sum lies in the range 2–18. The red bar graphs show the probabilities of the two inputs and of the output. We also show their entropies. Entropy is defined on probability distributions. The more evenly spread the distribution, the more information it contains and the higher the entropy. The output (2–18) being more uniformly spread, has a higher entropy (3.69 bits) than either input, but it is still less than the entropy of the combined inputs together. That is, in this case, the addition information funnel has lost 2.06 bits of information.

Not only is this information loss inevitable, but it is cumulative. That is, once information is lost, it cannot be restored.

Secondly although pretty much everything, bar simple assignment statements, losses information, some operations loose more than others. For example, testing if a value is greater than a threshold outputs a single bit, containing (at most) 1 bit of information, even if the variable being tested could contain 32 bits of information. Similarly a common technique to increase resilience to errors is to use wrappers. E.g. to prevent run-time exceptions by forcing a module’s input to lie in an expected range. Thus the module may have a float input, but in reality expects values to be three digit integers. Hence the developer may design a wrapper to return a default value for negative numbers and numbers greater than 999.0 and round other inputs to the nearest integer. Assuming it is sometimes necessary (e.g. the input is sometimes 5.01), such a robustness increasing wrapper will destroy information. 

Figure 1: An information funnel. All computer operations lose information. Even something as fundamental as addition outputs less information than it took in. Example (see red bar charts): addition of two independent numbers (drawn using Benford’s Law from 1 to 9, each contain 2.88 bits of information, total 5.75 bits). The result, 2 to 18, contains only 3.69 bits. 

Figure 2 shows a large nested function composed of floating point arithmetic operations (+, -, × and ÷). Each function acts as a two input information funnel and loses information. The red subtree at the base of Figure 2 is a bug and the coloured chain shows the execution of nested functions which call the bug. The whole function is executed on 48 test cases and the size of the coloured nodes represents the number of tests when the error and the correct code are not identical at that point in the execution. The colour of these nodes represents the average difference. In this example, the error is visible on all the tests at the function immediately after it. However the number of test cases where the evaluation is not identical relentlessly falls as we move up the chain of coloured nodes towards the outer most function. Indeed, in this example, it falls to zero before we reach the end of the expression. Meaning none of the test cases are able to detect the error.

Figure 2: Impact of error (red subtree) dissipates the further away we measure its change on test values. Size of functional nodes shows number of disrupted test cases. The colour shows, on a log scale, the average difference in evaluation on the remaining disrupted nodes. Brightest yellow shows smallest non-zero difference (RMS 3.1 10−10). Eventually this large change has no impact at all [2] [3]. 

Obviously the loss of information and details of error hiding will depend on the details of the expression and test suite. Nonetheless in functional parts of the code, the monotonic fall in the effectiveness of testing with distance from the error is always true and holds regardless of whether the error is caused by a bug in the source code or a runtime induced fault. Preliminary results hint that the effectiveness of test suites increases only slowly with the number of tests (in proportion to log of the number of tests) and suggests that independent tests are more effective at uncovering faults.

In traditional imperative code the data flow need not follow the execution flow. In particular, the effect of an error may be stored in a variable which is used later. Nonetheless when it is used (computed on) the information it contains about the error may be (partially) lost, particularly if the impact of the error passes through a long chain of operations.

The information funnel view is entirely consistent with Voas’ PIE (propagation, infection, and execution) framework [4] for explaining software errors. Voas says to have any observable effect the error must be executed, that execution must make a difference, i.e. infect the state of the computation, and crucially that state change must propagate to the program’s output. What we see is where the change of state passes through a chain of operations (each of which loses information) information about the error may disappear on one or more test cases. That is, information theory backs up the intuition that deeply nested errors will be more difficult to find using testing but also they may have no impact on many test cases. It also justifies the use of unit testing, as not only may the bug be easier to locate, but also, as the code is less heavily nested, bugs may manifest themselves more readily.

The fact that software is not fragile [5] is only tacitly recognised, is perhaps partially explained by the large number of software engineering phenomena which express it using a range of terminology from different research silos:

• In mutation testing the problem of equivalent mutants is wide spread. And yet, an equivalent mutant is simply a code change which can not be separated from the original code by testing.

• Schulte et al. [6] described software mutational robustness when they showed in examples of production C and assembler code that more than 30% of random code changes made no difference under testing.

• Similarly Harrand et al. [7] found neutral program variants when they mutated Java code.

• Members of the same Stockholm team [8] use the term correctness attraction to describe the imperturbability of ordinary software to injection of minimal changes. That is, the Java program gives the same output (assumed to be correct) even though a value set by executed code has been changed by a small amount at run time.

• We [1] suggest the phrase disruption propagation failure as a generalisation of failed error propagation [9], to recognise that the change in program state need not be an error or traditional source code bug and instead she includes all manner of errors, mutations and run-time perturbations.

• Coincidental correctness and fault masking can be placed in Voas’ PIE framework in that an error is said to have occurred but that it did not manifest (e.g. due to not infecting the state in the first place or failing to propagate). 

In future we will see software systems of increasing complexity. Such systems have long past the point of being comprehensible by any one person and indeed any team of people. Any yet globally we depend on the reliability of software.

In a world addicted to software, managing its quality will become ever more important. Information theory and entropy loss analysis offers a unified view of bugs, faults, transient errors, hardening code to resist tampering and attack, defect resilience and can offer insights into software verification and validation (V&V), particularly testing.

You might also enjoy reading

• Wikipedia article on information theory and Shannon entropy.

• Correctness Attraction: A Study of Stability of Software Behavior Under Runtime Perturbation, 12 November 2018. Martin Monperrus and his co-authors discuss results on 10 Java programs where injection of minimal run time changes made no difference in most (68%) cases.

• Bit-Rot: Computer Software Degrades over Time. IEEE Software blog, 11 March 2020.

• Genetic Improvement. How search is being used to improve existing programs. IEEE Software blog, 2 February 2016.

• Automated bug fixing. An interview with Westley Weimer and Martin Monperrus (published in Ubiquity, 2015, March, pp 1–11).

• Automated bug fixing in Facebook. Mark Harman and his coauthors describe genetic improvement at a global scale [10].

• Benford’s law of anomalous numbers. Wikipedia article on why leading digits are often small. 

 

References

1. [1]  Justyna Petke, William B. Langdon, and David Clark. Software robustness: A survey, a theory, and some prospects. In Paris Avgeriou and Dongmei Zhang, editors, ESEC/FSE 2021, Ideas, Visions and Reflections, Athens, Greece, 23-28 August 2021. ACM.

2. [2]  William B. Langdon. Fitness first. In Wolfgang Banzhaf, Leonardo Trujillo, Stephan Winkler, and Bill Worzel, editors, Genetic Programming Theory and Practice XVIII, East Lansing, MI, USA, 19-21 May 2021. Springer. Forthcoming.

3. [3]  William B. Langdon, Justyna Petke, and David Clark. Dissipative polynomials. In Nadarajen Veerapen, Katherine Malan, Arnaud Liefooghe, Sebastien Verel, and Gabriela Ochoa, editors, 5th Workshop on Landscape-Aware Heuristic Search, GECCO 2021 Companion, Internet, 10-14 July 2021. ACM.

4. [4]  Jeffrey M. Voas. PIE: a dynamic failure-based technique. IEEE Transactions on Software Engineering, 18(8):717–727, Aug 1992.

5. [5]  William B. Langdon and Justyna Petke. Software is not fragile. In Pierre Parrend, Paul Bourgine, and Pierre Collet, editors, Complex Systems Digital Campus E-conference, CS-DC’15, Proceedings in Complexity, pages 203–211. Springer, September 30-October 1 2015. Invited talk.

6. [6]  Eric Schulte, Zachary P. Fry, Ethan Fast, Westley Weimer, and Stephanie Forrest. Software mutational robustness. Genetic Programming and Evolvable Machines, 15(3):281–312, September 2014.

7. [7]  Nicolas Harrand, Simon Allier, Marcelino Rodriguez-Cancio, Martin Monperrus, and Benoit Baudry. A journey among Java neutral program variants. Genetic Programming and Evolvable Machines, 20(4):531–580, December 2019.

8. [8]  Benjamin Danglot, Philippe Preux, Benoit Baudry, and Martin Monperrus. Correctness attraction: a study of stability of software behavior under runtime perturbation. Empirical Software Engineering, 23(4):2086–2119, 1 August 2018.

9. [9]  Gunel Jahangirova, David Clark, Mark Harman, and Paolo Tonella. An empirical study on failed error propagation in Java programs with real faults. ArXiv, 24 Nov 2020. abs/2011.10787.

10. [10]  John Ahlgren, Maria Eugenia Berezin, Kinga Bojarczuk, Elena Dulskyte, Inna Dvortsova, Johann George, Natalija Gucevska, Mark Harman, Ralf Laemmel, Erik Meijer, Silvia Sapora, and Justin Spahr-Summers. WES: Agent-based user interaction simulation on real infrastructure. In Shin Yoo, Justyna Petke, Westley Weimer, and Bobby R. Bruce, editors, GI @ ICSE 2020, pages 276–284, internet, 3 July 2020. ACM. Invited Keynote. 

11.  

What went wrong with Covid-19 Contact Tracing Apps - PART III

The Ups and Downs of Immuni

A Brief History of the Italian Contact Tracing App

Author: Alessio Ferrari (@alessferra)

In collaboration with: Muneera Bano (@DrMuneeraBano), Chetan Arora (@chetan_arora), Didar Zowghi (@DidarZowghi)

Italy was the first nation in Europe to be heavily hit by the COVID-19 wave. In mid February, the virus was silently circulating in the northern regions, and the superspreading event of the Atalanta-Valencia Champions League football match, now known as “Game Zero”, accelerated its diffusion in the nation and later in the rest of the continent. 

Since the early days of the pandemic, researchers, technicians, journalists, and common people started flooding the media with discussions on the possibility of an app to support contact tracing and contain the virus. The solution, called Immuni, was announced on April 16, and was released on June 15, without any financial cost for the Italian government. In mid October, it has been downloaded more than 9 million times (24% percent of Italian smartphones). In almost 5 months from its release, about 1.000 positive users shared their status, out of almost 500.000 positive cases identified. It is not clear how many new cases have been discovered through Immuni at today---the number of notifications is increasing rapidly---but at the beginning of September the total number of people who discovered to be positive thanks to the app was only 7 and one month later increased to 13.

A free, privacy-preserving, transparent app

Immuni only uses public infrastructures located within the national borders, and it is managed by a public company, Sogei S.p.A. The source code has been developed by a private company, Bending Spoons S.p.A. without any financial cost for the government, it is released under a GNU Affero General Public License version 3, and can be downloaded from GitHub. 

It was  developed to fulfill utility, accessibility, accuracy, privacy, scalability and transparency principles, as stated in its documentation. On October 22, it had 4.1 stars in the App Store, and 3.5 in the Play Store, with highly polarised reviews. One of the most common titles of its negative reviews is “inutile”, which means useless. But let’s start from the beginning. 

Immuni in the Play Store has highly polarized ratings.

How does it work? Immuni uses the distributed DP-3T protocol, and the Google and Apple Exposure Notification framework, which is also adopted by the German Corona Warn-App. The app stores the identifiers of the smartphones of other Immuni users who come into close contact with you for more than 15 minutes, by exchanging information through Bluetooth Low Energy. The identifier of each user is generated locally, and changes frequently. If you are tested positive, a healthcare operator asks you if you want to report your case by sharing your recent list of generated identifiers. If you decide to share, the healthcare operator enables your app to load your list of identifiers on a central server. Each day, Immuni downloads the lists of identifiers of positive cases from the server, and looks for a match in the list of recently met people. This way, the users who came into contact with a positive case can be warned by their app and can go get tested. 

Debated Incubation Italy started its nationwide lockdown on March 9, and less than ten days after, the contact tracing app was announced as “already developed”. The app was expected to use the GPS to track movements, and also to identify crowds and dangerous behaviours. This was of course regarded as a privacy threat by the population and a long debate followed in the news and social media. On 16 April 2020 the solution from Bending Spoon S.p.A. was selected, but many aspects of the working principles of the app (e.g., centralised vs decentralised model), were still unknown, and on April 20 an open letter was issued by Italian researchers encouraging the government to go for a transparent solution and a decentralised model. The public debate led to abandoning the initial option of the GPS, and led to an open-source, privacy preserving, and decentralised app based on Bluetooth. Somehow, the public opinion acted as a (self-involved) user that ended up changing the system specification. 

Late-spring birth After a pilot study in four specific regions that led to the discovery of three cases, Immuni was released on June 16. Italians saluted with excitement, with half a million downloads in the first 24 hours, and two millions in the first week.

Summer hibernation  The initial buzz ended quite early, and the final number of downloads stagnated between 4 and 5 millions for the whole summer, with around 100-200.000 downloads per week. The virus felt less dangerous as the cases were decreasing, and people almost forgot that an app existed for contact tracing. Although the key idea of a contact tracing app is to identify cases of casual contacts, as the family contacts can be traced more easily, the app was not used in the period in which casual contacts were more frequent: summer time, which in Iitaly means seaside crowds, discos, and loud aperitivos in small roads. In Sardinia---a typical holiday region, almost untouched by the virus until June---the number of local outbreaks passed from 3 to 80 during summer.

Autumn awakening With the increasing number of cases that followed the summer (2.000 at the beginning of October and 15.000 today) people started downloading Immuni more and more, with one million downloads per week, somehow looking at it as a vaccine, as also its name suggests. More users means also that more problems are emerging, especially concerning the integration of the app with the regional healthcare system, and, also to avoid the refusal of the app by some of the regions, the Government Decree of October 18 obliges the healthcare operators to enable the loading of the codes for the users who are tested positive. 

Problems and Users’ Opinions

• I need it! When will they release it? During the incubation of Immuni, from March to June 2020, news and social media were filled with discussions about the imminent release of the digital medicine for COVID-19. Both enthusiasts and skeptics seemed to think that people are as fast in developing apps as the computers are quick in running them, and thought that a novel, privacy-aware software with national impact, and (nice-to-have) able to interact with apps from other nations, could be developed in a weekend of mad programming. This was of course a false hope, but it is true that when the app came out it was almost summer, the pandemic appeared less dangerous (around 100 daily cases in mid-July), and you don’t carry your phone when you’re playing volleyball on the shore. Therefore, Immuni was released too late to be considered useful as initially planned.

• Wait, is it working? The majority of the app reviews state that the Immuni is useless and they wonder if it’s working. Similarly to the australian app, Immuni was conceived to be as discreet and privacy-preserving as possible, but this implies that it does not clearly report the contacts, and their duration, so that you can have an idea if it is correctly doing what it’s supposed to do. The only moment in which you know that Immuni is working is when you get notified that you had a contact with a positive case. For some users, the possibility of this notification is too scary to keep the app installed. In conclusion, although the app was developed around the pillar of transparency, with open source code, it should also be somewhat transparent at runtime, with clear information about what it has been doing. 

• Shall I buy a new phone? The app was designed to be accessible, so that everyone could use it. However, some older versions of Android and iPhone could not run it. Furthermore, with some iPhone versions users complained of having problems when moving to certain provinces, as the app notifies them that their area is not covered. This is also a weird notification, as the app should not be aware of the position of the user.  

• Oh, I am COVID-19 positive, what to do now? One of the problems noticed by some users is the lack of information on what to do when you are tested positive and you are an Immuni user. You receive the test results at home, and the healthcare operators should provide you information, but this did not happen and a special Government Decree was needed in October to oblige the healthcare operators to take Immuni into account. Still, some users complain that after receiving the results, they did not get any call from the healthcare operators, did not know what to do, and their casual contacts remained unaware of the possibility of being positive. 

• Oh, I had a contact with a COVID-19 positive, what to do now? Users complained that, after receiving the scary notification that they had a contact with a positive case, they did not know what to do. Were they supposed to self-isolate? Go for a test at their own cost? Many healthcare operators who were called in these cases did not have an answer for that. 

Overall, Immuni in itself appears to do its job, but it is still not well connected with the socio-technical ecosystem for contact tracing, and the healthcare practitioners involved in the contact tracing loop are not sufficiently trained on their role in the usage of the app. 

All is lost?

No, and we do not have to thank only the doctors this time, but also the software practitioners. I will report a personal fact. In my gym there was a positive case. As soon as my coach knew that, he notified us athletes through our Whatsapp group, and told us that we would be called by the healthcare operator to fix a test. The coach also sent the phone number of all the people present during the last training to the healthcare operator. How could he know who was present? Everytime we go, we register through a specific app, so we can be traced. The day after I received a call, I fixed the test, and I received an official email stating that I should self-isolate. After the test, I could check if I was positive from a very simple web portal, set-up just to release COVID-19 analysis (I was negative). This whole loop did not involve Immuni, but was supported by apps and software, that, together with the platforms entertaining me during the self-isolation, made me clear how much software practitioners have done to set-up the infrastructure to fight the pandemic, even before knowing there would be one. 

-------------------------------------------------------------------------------------------------------------------

Previous Articles on Contact Tracing Apps

1- Australia's CovidSafe App

2. German's Corona-Warn-App

What went wrong with Covid-19 Contact Tracing Apps - PART II

 

Germany’s Corona-Warn-App

Chetan Arora (@chetan_arora), Muneera Bano (@DrMuneeraBano), Didar Zowghi (@DidarZowghi)

Previous Part I: Australia's CovidSafe App

Over the past eight months, the novel coronavirus has impacted each and every aspect of our lives. What would’ve seemed like a narration of a sci-fi movie until 2019 is our bitter reality today. An integral part of this new reality are contact-tracing apps. We covered the factor leading to the widespread rejection of Australia’s CovidSafe app and how it failed to deliver on its promises in the first blog of this series. This entry explores the factors surrounding Germany’s national contract tracing application -- Corona-Warn-App. Corona-Warn-App notifies the users if they’ve been in contact with someone who tested positive for COVID-19. The app further allows users to receive their test results from the testing labs on the app. 

The “German” Engineering and Transparency

Engineering Practice and Implementation. Corona-Warn-App first launched on June 16, 2020 and has been updated several times after the fact. The app has been developed using Google and Apple’s Exposure Notification framework and works with Bluetooth low energy, and stores locally the randomized identifiers of the other phones it comes in contact with. The unique identifiers of each phone are valid only for 10-20 minutes, and are updated thereafter. Among the global contact tracing apps split over centralized vs decentralized mode, the german app took the more secure decentralized path. The randomized identifiers of a user’s contacts are stored locally on the phone, and only voluntarily stored on a central server, when the users test positive. No personal data is stored by the app. The app was launched in two languages -- German and English, and has since been updated to include Turkish, Bulgarian, Polish and Romanian, with plans to further extend it to Arabic and Russian.

Democratic Practices. Germany’s Corona-Warn-App, vis-à-vis numerous contact tracing apps from other countries,  stood out on several levels. The app development was based on democratic principles, wherein the federal health institute - Robert Koch Institute (RKI) (i) involved reputed public and private partners for developing and supporting this app -- such as SAP, Fraunhofer research institute, Deutsche Telekom, and Centre for Information Security, (ii) maintained transparency in the source code management by making it open source from the very outset of its launch, and (iii) designing the app based on GDPR principles to alleviate any issues related to data privacy. The app was fully backed by the government as support to the manual contact tracing system and for spreading awareness in public on coronavirus, and even termed it as the “best app” globally for this purpose.

Engagement with the Users. As we noted in our first blog, our previous research established that the dissatisfaction of users can result in software project failures even if it is delivered on time, on budget and is of high quality. In particular, in countries like Germany with a low-trust index of the citizens in their government, an indication of non-democratic or non-inclusive practice could lead to dissatisfied users. Given the low trust in government, the app had slim chances of success and adoption, especially given the exorbitant 68 million Euros estimated cost of the app by the end of 2021. RKI’s democratic practices outlined above improved these chances. The developers gained and maintained the trust of the users by staying in contact with the users. The developer-user communication contributed significantly to the adoption of the app.

As per the statistical forecasting by Oxford University’s Nuffield Department of Medicine and Alphabet Inc (Google), a contact tracing app, such as, Corona-Warn-App if regularly used by 15% of the population alongside a well-staffed contact-tracing workforce can potentially lead to a 15% drop in overall infection rates and an approximate 11% drop in COVID-19 deaths. With 18.4 million downloads recorded as of 23rd September 2020, more than 20% of the population in Germany has already downloaded the contact-tracing app. 

While the number of downloads, in proportion to the population, is encouraging to compare with other countries such as Australia, the number is still low. In a country of more than 83 million inhabitants with approximately 57 million of them owning smartphones, the contact tracing app can not be simply termed as “success” or the “best” contact-tracing app. While the German app, against some odds, seems to have more than 18 million downloads, the number of downloads is still less than one-third of the maximum number. The app completed 100 days in operation on September 23rd 2020. The officials reported only 5000 of 277376 people (that tested positive for COVID-19 in Germany) voluntarily shared their result as positive and their contact results on the app. Despite such results, the app was termed as the “best app”. In the next section, we discuss the possible grounds for low adoption rates and what went wrong for the German app. 

What went “schief” for the Corona-Warn-App

We collected all the user reviews from both the Apple store and the Google store for the iOS and Android versions of the app, respectively, from the onset of the app to September 15th, 2020. We base our analysis on the reviews collected.

Technical Issues. Based on the user reviews of the app on Google Play store and the App store from Apple, more than a quarter of the reviews complained about bugs in different versions of the app. While developers of the app, to their credit, maintained a chain of communication with the reviewers on both the App stores, the app and its adaptability has been definitely affected by glitches and bugs throughout its lifespan so far. 

The second most prevalent issue in the app (according to the user reviews) is related to the platform and devices. Corona-warn-app has major issues with iPhones and did not work with Android phones either for up to five weeks due to issues in the operating systems. The app further does not work with older OS versions in both iOS and Android phones. For instance, for iOS devices one would require a phone with iOS version 13.5 or later, and similar technical restrictions exist for Android devices. According to an estimate by experts, Corona-Warn-App wouldn’t work for 10-20% of devices in Germany, i.e., this section of the population is divested of the benefits offered by digital contact tracing.

Lack of Cultural Considerations. While the app was available in English and German from the beginning, and Turkish (given a large fraction of Turkish speakers in Germany), the app does not yet include Arabic and Italian languages. As inferred by the following review on the app in the Google store - “Die App muss mindestens auf italienisch, spanisch und arabisch verfügbar sein :) halbe Million Italiener, 2 Millionen arabisch sprechende wohnen in Deutschland.” translated as “The app must be at the very least available in Italian, Spanish and Arabic. There are 0.5 million  Italian and 2 million Arabic speakers living in Germany”. With the influx of Arabic (and Kurdish)-speaking asylum seekers and migrants in Germany in the past few years, it was naturally expected for the Corona-Warn-App to be available in Arabic, which unfortunately has not been the case. In fact, in a recent turn of events, the government claimed that translation of the app in Kurdish will cost a sum of 250,000 Euros. 

Privacy and Effectiveness Concerns. Many users, from the 18 million that downloaded the app, claimed that they would uninstall the app or will discontinue using it. As per the experts, only 15 million are active users, which is a major blow to the effectiveness, in addition to and not discounting the fact that a large fraction of potential users did not install the app in the first place. Germans still do not fully trust the privacy and data protection of the app, and the concerns remain over the period as suggested by a survey conducted by TU München. The survey reflects that around 9% more people (from July’2020 to August’2020) feel that they will not use the app in any case. More than half of the respondents also mentioned that they do not believe that the app can be effective or can have any impact.

In conclusion, the German app trumps several contact tracing apps across the world in terms of democratic practices, number of downloads, and user-development engagement. However, the overall effectiveness of the contact tracing app is yet to be determined. With the rise in the cases, in what is deemed as the second wave in Germany (illustrated in the chart above), the doubts around the effectiveness of the app are increasing.

In addition, the voluntary model of downloading the app (in countries with the low-trust index on government like Germany), and voluntary sharing of the contact list by the infected people has a huge impact on the adoption and effectiveness of the app. User dissatisfaction and lack of confidence in the app are apparent. In our opinion, the app needs to be further improved, and the developers and the government have an uphill task of (re-)gaining the trust of citizens before reiterating their rhetoric of Corona-Warn-App being the “best” app in the world. 

What went wrong with Covid-19 Contact Tracing Apps - PART I

Australia's CovidSafe App

Muneera Bano (@DrMuneeraBano), Chetan Arora (@chetan_arora), Didar Zowghi (@DidarZowghi)

The new world of Covid-19

Last year this time, the majority of us were not even familiar with the term Covid-19. Everything that seemed ordinary back then is extraordinary today - walking outside without a mask, standing next to strangers on public transport, teaching a class of over 200 students in a seminar room, joining crowds in sporting and entertainment events. All these instances seem in another world and ages ago! Covid-19 has shaken every fabric of the society in every country around the world.In our current reality, social distancing is the new expression of love and care, masks have become a controversial sign of obedience or defiance, Zoom has taken over the new forms of social connections whether its education, meeting family and friends or online dating, and contact tracing apps seem to be the new global trend.

The Covid-19 has hit the world harder and faster than anticipated and not many countries were prepared for such level of global pandemic. To handle the situation, we have seen the measures introduced in different countries related to: social distancing, lock-downs, and contact tracing. The speed of the spread of this infection highlighted the limitations of manual contact tracing. Contact tracing is a critical part of dealing with this pandemic in order to quarantine the infected and slow down or curb the spread of the virus. 

There is always an app!

The numerous challenges presented by Covid-19, such as the incubation period and the asymptomatic super spreaders, have exacerbated the challenges of manual contact tracing. This pushed the governments around the world to look for alternative solutions to aid and expedite the process with technology and we saw a flood of corona-virus related apps launched in the first half of 2020.

 

The idea is that an app will make use of Bluetooth and/or GPS technologies on mobile phones and will provide a low cost, low energy solution by interacting with other mobile phones in order to record community interactions for a period of time that may pose the danger of infection transmission. This digital solution offers great benefits in terms of instant recording of interactions and alleviates the limitations and biases of patients’ memory, and in theory it has been considered a very effective solution for the scale and level of Covid-19 pandemic.

However, the promised benefits of these contact tracing apps failed to materialise, to a large extent. No matter how technologically sound, these apps have not been able to provide successful outcomes. While there are several confounding causes for the failure of the contact-tracing apps, one important factor is related to the involvement of the users and whether the users’ requirements are considered and addressed by these apps.

Developed by the People, for the People?

Inspired by our previous research on the correlations between user involvement and system success, we were interested to explore the factors that limited the success and adoption of the Covid-19 apps. Among technical issues with the apps, these factors, include, the distrust of users, as well as disregard for the technological demographics of citizens during inception, design, and launch of the apps.

Our previous research established that the dissatisfaction of users can result in software project failures even if it is delivered on time, on budget and is of high quality. For societies with democratic values, the non-democratic and non-inclusive approaches adopted for any software development project could lead to dissatisfied users who would eventually distrust and reject the final product. The user dissatisfaction in many cases with Covid-19 apps is in line with our previous findings.

In Australia, the government launched a contact tracing app called CovidSafe on 26th April 2020, and there was a significant level of mistrust on the app since its launch due to several reasons, including among others, privacy concerns, technical issues, lack of transparency and no end-user involvement. As of 28th June after 6 million downloads, it was reported that the app has not  been able to detect any new cases. Furthermore, publicly available information on the app’s contribution to pandemic control and the usage of the app is scarce.

Our Australian case motivated us to study other contact-tracing apps launched around the world and explore their acceptance by the target community. We were interested to see the responses of the users of these apps to understand what social, economical, cultural, political and technological factors contributed to the successes or failures of contact tracing apps.

We have studied 21 countries in our research and downloaded the user reviews from both Apple and Google app stores in order to analyse the voices of the users for these apps. In this blog series we will be presenting our analysis of the various social dimensions around user satisfaction/dissatisfaction, staring with the case of Australia.

We collected all the reviews for the apps listed below in the table (for the duration of  1st January 2020 – 30th July 2020) from Google Play and Apple’s App Store.

Looking at the comments of users for CovidSafe app, and comparing them to the reviews coming from apps launched by others countries, we observed the following issues:

Lack of Citizen Involvement

• CovidSafe app was inspired by the Singapore model of TraceTogether app. Replicating the same model in Australia without considering the social context would not yield the same results. In Singapore the people have a high level of trust in their government whereas Australia scores low on the trust index. For a democratic society, that leans highly towards individualism, lack of citizens involvement in projects that involve data privacy issues would have lower chances of success.

• In Germany, the trust index of people in their government is also low, however, their app, Corona-Warn-App is positively received by the people, due to the democratic approaches in the development process. The app was open source from the beginning, and the development involved 13 reputable non-government and research organisations together with the involvement of the global open source community.

(https://www.coronawarn.app/en/)

Non- Responsiveness to concerns   

• There were numerous issues reported on CovidSafe app related to Bluetooth, Battery and Data Consumption, however there was no engagement shown by the development team to respond to the authors of the reviews. Whereas there was effective developer-user communication seen in app reviews from New Zealand, Germany and India that helped in building people’s trust for the app. 

Lack of understanding technological context of Australian people

• The Australian society’s technology orientation was not considered properly during the planning phase of the app development. A significant number of Australians are iOS users, whereas in general contact tracing apps seem to work better on Android platforms. CovidSafe app was reported in early stages after launch to have issues with iOS platforms. Similar observation was made in the case of India’s national app ‘Aarogya Setu’ and its popularity on Google store, as the majority of the population use Android platform. Almost all the contact tracing apps around the world had dominant negative sentiments towards iOS versions.

Ambitious technical assumptions without cultural considerations

• The success of CovidSafe app was dependent heavily on the significant proportion of the population downloading and keeping the app active while in public spaces. This worked in countries like China, Singapore and India where people seem to have high levels of trust in governments and the national culture is heavily leaning towards collectivism. The voluntary download model in a democratic society such as Australia with much lower trust index in government, lack of transparency in the process, concerns of data privacy and the national culture leaning towards individualism is not ideal. The success of this app was based on a great deal of ambitious assumptions.

No End-User Experience

• CovidSafe app works in the background. The users do not see any immediate results and/or any functionality and hence there is no user experience. All they see is a simple green screen to tell them the app is active and no further actions required. What's happening in the background is invisible to the users. In contrast, the Indian app Aarogya Setu shows updates to the people on the number of positive cases in their vicinity. Similarly, New Zealand offered a different solution, wherein people can create a digital diary of the places they have visited by scanning the official QR codes. Such features in Indian and New Zealand apps, unlike in the Australian app, gives an impression to the users that they are in control and hence an integral part of the process. For CovidSafe app, once the app is installed the only possible way to control it is to turn-off the Bluetooth which will stop the functions of the app.

 Interface of CovidSafe App

 Interface of NZ Covid Tracer App

 India's Covid-19 Tracker App Aarogya Setu

Waiting for successful outcomes

• Australians are still waiting to hear any successful outcome from this multi-million Dollar app. This has further depreciated any motivation or trust that 6 million downloads showed since its launch. In Germany, it is reported that 660 people who tested positive for Covid-19 had the opportunity to warn other people via Corona-Warn-App. A significant number of Australian users commented that they uninstalled the CovidSafe app after it caused Bluetooth and battery issues and they do not see any positive contribution of the app in helping to stop the spread of the virus. 

• The inability to fulfil the purpose for which the app was launched has also been observed in the UK Government’s NHS-Covid19 that was taken off the app stores (until recently when the UK government announced its second phase with Google/Apple Framework).

• While currently Melbourne is going through the second wave of Covid-19, the state government in Victoria has accessed the data collected through the app around 400 times, but it is reported to not have revealed any single case that wasn’t known through manual tracing. Hence, Australian citizens are yet to witness any tangible benefits of the multi-million dollar CovidSafe application, at the time of writing this article.

In conclusion: 

Every country has their own unique struggle with the COVID-19 virus. For a successful technological solution in form of contact tracing app, the political and cultural context of the country cannot be ignored. Following or replicating the app framework of Singapore seems to have not worked in Australia. According to the Cultural Compass of Hofstede, Australia resembles closely to Germany when it comes to power structures and individualism in citizens, and hence could have taken similar steps to build public trust. In conclusion, and as established in our previous research, lack of user involvement and user dissatisfaction can lead to software project failure, despite meeting the time and budget goals. This was the case for CovidSafe, and in our opinion if the CovidSafe app was meant for Australians, then Australians should have been involved in the design, implementation and launch of it.