Saturday, October 24, 2020

What went wrong with Covid-19 Contact Tracing Apps - PART III

The Ups and Downs of Immuni

A Brief History of the Italian Contact Tracing App




Italy was the first nation in Europe to be heavily hit by the COVID-19 wave. In mid February, the virus was silently circulating in the northern regions, and the superspreading event of the Atalanta-Valencia Champions League football match, now known as “Game Zero”, accelerated its diffusion in the nation and later in the rest of the continent. 

Since the early days of the pandemic, researchers, technicians, journalists, and common people started flooding the media with discussions on the possibility of an app to support contact tracing and contain the virus. The solution, called Immuni, was announced on April 16, and was released on June 15, without any financial cost for the Italian government. In mid October, it has been downloaded more than 9 million times (24% percent of Italian smartphones). In almost 5 months from its release, about 1.000 positive users shared their status, out of almost 500.000 positive cases identified. It is not clear how many new cases have been discovered through Immuni at today---the number of notifications is increasing rapidly---but at the beginning of September the total number of people who discovered to be positive thanks to the app was only 7 and one month later increased to 13.

A free, privacy-preserving, transparent app

Immuni only uses public infrastructures located within the national borders, and it is managed by a public company, Sogei S.p.A. The source code has been developed by a private company, Bending Spoons S.p.A. without any financial cost for the government, it is released under a GNU Affero General Public License version 3, and can be downloaded from GitHub


It was  developed to fulfill utility, accessibility, accuracy, privacy, scalability and transparency principles, as stated in its documentation. On October 22, it had 4.1 stars in the App Store, and 3.5 in the Play Store, with highly polarised reviews. One of the most common titles of its negative reviews is “inutile”, which means useless. But let’s start from the beginning. 


Immuni in the Play Store has highly polarized ratings.


How does it work? Immuni uses the distributed DP-3T protocol, and the Google and Apple Exposure Notification framework, which is also adopted by the German Corona Warn-App. The app stores the identifiers of the smartphones of other Immuni users who come into close contact with you for more than 15 minutes, by exchanging information through Bluetooth Low Energy. The identifier of each user is generated locally, and changes frequently. If you are tested positive, a healthcare operator asks you if you want to report your case by sharing your recent list of generated identifiers. If you decide to share, the healthcare operator enables your app to load your list of identifiers on a central server. Each day, Immuni downloads the lists of identifiers of positive cases from the server, and looks for a match in the list of recently met people. This way, the users who came into contact with a positive case can be warned by their app and can go get tested. 


Debated Incubation Italy started its nationwide lockdown on March 9, and less than ten days after, the contact tracing app was announced as “already developed”. The app was expected to use the GPS to track movements, and also to identify crowds and dangerous behaviours. This was of course regarded as a privacy threat by the population and a long debate followed in the news and social media. On 16 April 2020 the solution from Bending Spoon S.p.A. was selected, but many aspects of the working principles of the app (e.g., centralised vs decentralised model), were still unknown, and on April 20 an open letter was issued by Italian researchers encouraging the government to go for a transparent solution and a decentralised model. The public debate led to abandoning the initial option of the GPS, and led to an open-source, privacy preserving, and decentralised app based on Bluetooth. Somehow, the public opinion acted as a (self-involved) user that ended up changing the system specification. 


Late-spring birth After a pilot study in four specific regions that led to the discovery of three cases, Immuni was released on June 16. Italians saluted with excitement, with half a million downloads in the first 24 hours, and two millions in the first week.


Summer hibernation  The initial buzz ended quite early, and the final number of downloads stagnated between 4 and 5 millions for the whole summer, with around 100-200.000 downloads per week. The virus felt less dangerous as the cases were decreasing, and people almost forgot that an app existed for contact tracing. Although the key idea of a contact tracing app is to identify cases of casual contacts, as the family contacts can be traced more easily, the app was not used in the period in which casual contacts were more frequent: summer time, which in Iitaly means seaside crowds, discos, and loud aperitivos in small roads. In Sardinia---a typical holiday region, almost untouched by the virus until June---the number of local outbreaks passed from 3 to 80 during summer.


Autumn awakening With the increasing number of cases that followed the summer (2.000 at the beginning of October and 15.000 today) people started downloading Immuni more and more, with one million downloads per week, somehow looking at it as a vaccine, as also its name suggests. More users means also that more problems are emerging, especially concerning the integration of the app with the regional healthcare system, and, also to avoid the refusal of the app by some of the regions, the Government Decree of October 18 obliges the healthcare operators to enable the loading of the codes for the users who are tested positive. 


Problems and Users’ Opinions


  • I need it! When will they release it? During the incubation of Immuni, from March to June 2020, news and social media were filled with discussions about the imminent release of the digital medicine for COVID-19. Both enthusiasts and skeptics seemed to think that people are as fast in developing apps as the computers are quick in running them, and thought that a novel, privacy-aware software with national impact, and (nice-to-have) able to interact with apps from other nations, could be developed in a weekend of mad programming. This was of course a false hope, but it is true that when the app came out it was almost summer, the pandemic appeared less dangerous (around 100 daily cases in mid-July), and you don’t carry your phone when you’re playing volleyball on the shore. Therefore, Immuni was released too late to be considered useful as initially planned.


  • Wait, is it working? The majority of the app reviews state that the Immuni is useless and they wonder if it’s working. Similarly to the australian app, Immuni was conceived to be as discreet and privacy-preserving as possible, but this implies that it does not clearly report the contacts, and their duration, so that you can have an idea if it is correctly doing what it’s supposed to do. The only moment in which you know that Immuni is working is when you get notified that you had a contact with a positive case. For some users, the possibility of this notification is too scary to keep the app installed. In conclusion, although the app was developed around the pillar of transparency, with open source code, it should also be somewhat transparent at runtime, with clear information about what it has been doing


  • Shall I buy a new phone? The app was designed to be accessible, so that everyone could use it. However, some older versions of Android and iPhone could not run it. Furthermore, with some iPhone versions users complained of having problems when moving to certain provinces, as the app notifies them that their area is not covered. This is also a weird notification, as the app should not be aware of the position of the user.  


  • Oh, I am COVID-19 positive, what to do now? One of the problems noticed by some users is the lack of information on what to do when you are tested positive and you are an Immuni user. You receive the test results at home, and the healthcare operators should provide you information, but this did not happen and a special Government Decree was needed in October to oblige the healthcare operators to take Immuni into account. Still, some users complain that after receiving the results, they did not get any call from the healthcare operators, did not know what to do, and their casual contacts remained unaware of the possibility of being positive. 


  • Oh, I had a contact with a COVID-19 positive, what to do now? Users complained that, after receiving the scary notification that they had a contact with a positive case, they did not know what to do. Were they supposed to self-isolate? Go for a test at their own cost? Many healthcare operators who were called in these cases did not have an answer for that. 


Overall, Immuni in itself appears to do its job, but it is still not well connected with the socio-technical ecosystem for contact tracing, and the healthcare practitioners involved in the contact tracing loop are not sufficiently trained on their role in the usage of the app. 



All is lost?


No, and we do not have to thank only the doctors this time, but also the software practitioners. I will report a personal fact. In my gym there was a positive case. As soon as my coach knew that, he notified us athletes through our Whatsapp group, and told us that we would be called by the healthcare operator to fix a test. The coach also sent the phone number of all the people present during the last training to the healthcare operator. How could he know who was present? Everytime we go, we register through a specific app, so we can be traced. The day after I received a call, I fixed the test, and I received an official email stating that I should self-isolate. After the test, I could check if I was positive from a very simple web portal, set-up just to release COVID-19 analysis (I was negative). This whole loop did not involve Immuni, but was supported by apps and software, that, together with the platforms entertaining me during the self-isolation, made me clear how much software practitioners have done to set-up the infrastructure to fight the pandemic, even before knowing there would be one. 


-------------------------------------------------------------------------------------------------------------------


Previous Articles on Contact Tracing Apps



Wednesday, September 23, 2020

What went wrong with Covid-19 Contact Tracing Apps - PART II

 

Germany’s Corona-Warn-App


Chetan Arora (@chetan_arora), Muneera Bano (@DrMuneeraBano), Didar Zowghi (@DidarZowghi)


Previous Part I: Australia's CovidSafe App



Over the past eight months, the novel coronavirus has impacted each and every aspect of our lives. What would’ve seemed like a narration of a sci-fi movie until 2019 is our bitter reality today. An integral part of this new reality are contact-tracing apps. We covered the factor leading to the widespread rejection of Australia’s CovidSafe app and how it failed to deliver on its promises in the first blog of this series. This entry explores the factors surrounding Germany’s national contract tracing application -- Corona-Warn-App. Corona-Warn-App notifies the users if they’ve been in contact with someone who tested positive for COVID-19. The app further allows users to receive their test results from the testing labs on the app. 

The “German” Engineering and Transparency

Engineering Practice and Implementation. Corona-Warn-App first launched on June 16, 2020 and has been updated several times after the fact. The app has been developed using Google and Apple’s Exposure Notification framework and works with Bluetooth low energy, and stores locally the randomized identifiers of the other phones it comes in contact with. The unique identifiers of each phone are valid only for 10-20 minutes, and are updated thereafter. Among the global contact tracing apps split over centralized vs decentralized mode, the german app took the more secure decentralized path. The randomized identifiers of a user’s contacts are stored locally on the phone, and only voluntarily stored on a central server, when the users test positive. No personal data is stored by the app. The app was launched in two languages -- German and English, and has since been updated to include Turkish, Bulgarian, Polish and Romanian, with plans to further extend it to Arabic and Russian.


Democratic Practices. Germany’s Corona-Warn-App, vis-à-vis numerous contact tracing apps from other countries,  stood out on several levels. The app development was based on democratic principles, wherein the federal health institute - Robert Koch Institute (RKI) (i) involved reputed public and private partners for developing and supporting this app -- such as SAP, Fraunhofer research institute, Deutsche Telekom, and Centre for Information Security, (ii) maintained transparency in the source code management by making it open source from the very outset of its launch, and (iii) designing the app based on GDPR principles to alleviate any issues related to data privacy. The app was fully backed by the government as support to the manual contact tracing system and for spreading awareness in public on coronavirus, and even termed it as the “best app” globally for this purpose.



Engagement with the Users. As we noted in our first blog, our previous research established that the dissatisfaction of users can result in software project failures even if it is delivered on time, on budget and is of high quality. In particular, in countries like Germany with a low-trust index of the citizens in their government, an indication of non-democratic or non-inclusive practice could lead to dissatisfied users. Given the low trust in government, the app had slim chances of success and adoption, especially given the exorbitant 68 million Euros estimated cost of the app by the end of 2021. RKI’s democratic practices outlined above improved these chances. The developers gained and maintained the trust of the users by staying in contact with the users. The developer-user communication contributed significantly to the adoption of the app.



As per the statistical forecasting by Oxford University’s Nuffield Department of Medicine and Alphabet Inc (Google), a contact tracing app, such as, Corona-Warn-App if regularly used by 15% of the population alongside a well-staffed contact-tracing workforce can potentially lead to a 15% drop in overall infection rates and an approximate 11% drop in COVID-19 deaths. With 18.4 million downloads recorded as of 23rd September 2020, more than 20% of the population in Germany has already downloaded the contact-tracing app. 


While the number of downloads, in proportion to the population, is encouraging to compare with other countries such as Australia, the number is still low. In a country of more than 83 million inhabitants with approximately 57 million of them owning smartphones, the contact tracing app can not be simply termed as “success” or the “best” contact-tracing app. While the German app, against some odds, seems to have more than 18 million downloads, the number of downloads is still less than one-third of the maximum number. The app completed 100 days in operation on September 23rd 2020. The officials reported only 5000 of 277376 people (that tested positive for COVID-19 in Germany) voluntarily shared their result as positive and their contact results on the app. Despite such results, the app was termed as the “best app”. In the next section, we discuss the possible grounds for low adoption rates and what went wrong for the German app. 

What went “schief” for the Corona-Warn-App

We collected all the user reviews from both the Apple store and the Google store for the iOS and Android versions of the app, respectively, from the onset of the app to September 15th, 2020. We base our analysis on the reviews collected.


Technical Issues. Based on the user reviews of the app on Google Play store and the App store from Apple, more than a quarter of the reviews complained about bugs in different versions of the app. While developers of the app, to their credit, maintained a chain of communication with the reviewers on both the App stores, the app and its adaptability has been definitely affected by glitches and bugs throughout its lifespan so far. 


The second most prevalent issue in the app (according to the user reviews) is related to the platform and devices. Corona-warn-app has major issues with iPhones and did not work with Android phones either for up to five weeks due to issues in the operating systems. The app further does not work with older OS versions in both iOS and Android phones. For instance, for iOS devices one would require a phone with iOS version 13.5 or later, and similar technical restrictions exist for Android devices. According to an estimate by experts, Corona-Warn-App wouldn’t work for 10-20% of devices in Germany, i.e., this section of the population is divested of the benefits offered by digital contact tracing.


Lack of Cultural Considerations. While the app was available in English and German from the beginning, and Turkish (given a large fraction of Turkish speakers in Germany), the app does not yet include Arabic and Italian languages. As inferred by the following review on the app in the Google store - “Die App muss mindestens auf italienisch, spanisch und arabisch verfügbar sein :) halbe Million Italiener, 2 Millionen arabisch sprechende wohnen in Deutschland.” translated as “The app must be at the very least available in Italian, Spanish and Arabic. There are 0.5 million  Italian and 2 million Arabic speakers living in Germany”. With the influx of Arabic (and Kurdish)-speaking asylum seekers and migrants in Germany in the past few years, it was naturally expected for the Corona-Warn-App to be available in Arabic, which unfortunately has not been the case. In fact, in a recent turn of events, the government claimed that translation of the app in Kurdish will cost a sum of 250,000 Euros


Privacy and Effectiveness Concerns. Many users, from the 18 million that downloaded the app, claimed that they would uninstall the app or will discontinue using it. As per the experts, only 15 million are active users, which is a major blow to the effectiveness, in addition to and not discounting the fact that a large fraction of potential users did not install the app in the first place. Germans still do not fully trust the privacy and data protection of the app, and the concerns remain over the period as suggested by a survey conducted by TU München. The survey reflects that around 9% more people (from July’2020 to August’2020) feel that they will not use the app in any case. More than half of the respondents also mentioned that they do not believe that the app can be effective or can have any impact.



In conclusion, the German app trumps several contact tracing apps across the world in terms of democratic practices, number of downloads, and user-development engagement. However, the overall effectiveness of the contact tracing app is yet to be determined. With the rise in the cases, in what is deemed as the second wave in Germany (illustrated in the chart above), the doubts around the effectiveness of the app are increasing.






In addition, the voluntary model of downloading the app (in countries with the low-trust index on government like Germany), and voluntary sharing of the contact list by the infected people has a huge impact on the adoption and effectiveness of the app. User dissatisfaction and lack of confidence in the app are apparent. In our opinion, the app needs to be further improved, and the developers and the government have an uphill task of (re-)gaining the trust of citizens before reiterating their rhetoric of Corona-Warn-App being the “best” app in the world. 

Friday, August 14, 2020

What went wrong with Covid-19 Contact Tracing Apps - PART I

Australia's CovidSafe App

ACS members back COVIDSafe app | Information Age | ACS

The new world of Covid-19

Last year this time, the majority of us were not even familiar with the term Covid-19. Everything that seemed ordinary back then is extraordinary today - walking outside without a mask, standing next to strangers on public transport, teaching a class of over 200 students in a seminar room, joining crowds in sporting and entertainment events. All these instances seem in another world and ages ago! Covid-19 has shaken every fabric of the society in every country around the world.In our current reality, social distancing is the new expression of love and care, masks have become a controversial sign of obedience or defiance, Zoom has taken over the new forms of social connections whether its education, meeting family and friends or online dating, and contact tracing apps seem to be the new global trend.

The Covid-19 has hit the world harder and faster than anticipated and not many countries were prepared for such level of global pandemic. To handle the situation, we have seen the measures introduced in different countries related to: social distancing, lock-downs, and contact tracing. The speed of the spread of this infection highlighted the limitations of manual contact tracing. Contact tracing is a critical part of dealing with this pandemic in order to quarantine the infected and slow down or curb the spread of the virus. 

There is always an app!

The numerous challenges presented by Covid-19, such as the incubation period and the asymptomatic super spreaders, have exacerbated the challenges of manual contact tracing. This pushed the governments around the world to look for alternative solutions to aid and expedite the process with technology and we saw a flood of corona-virus related apps launched in the first half of 2020.

Corona, Covid-19, Covid, Corona App, App, Program 

The idea is that an app will make use of Bluetooth and/or GPS technologies on mobile phones and will provide a low cost, low energy solution by interacting with other mobile phones in order to record community interactions for a period of time that may pose the danger of infection transmission. This digital solution offers great benefits in terms of instant recording of interactions and alleviates the limitations and biases of patients’ memory, and in theory it has been considered a very effective solution for the scale and level of Covid-19 pandemic.

However, the promised benefits of these contact tracing apps failed to materialise, to a large extent. No matter how technologically sound, these apps have not been able to provide successful outcomes. While there are several confounding causes for the failure of the contact-tracing apps, one important factor is related to the involvement of the users and whether the users’ requirements are considered and addressed by these apps.

Developed by the People, for the People?

Inspired by our previous research on the correlations between user involvement and system success, we were interested to explore the factors that limited the success and adoption of the Covid-19 apps. Among technical issues with the apps, these factors, include, the distrust of users, as well as disregard for the technological demographics of citizens during inception, design, and launch of the apps.

Our previous research established that the dissatisfaction of users can result in software project failures even if it is delivered on time, on budget and is of high quality. For societies with democratic values, the non-democratic and non-inclusive approaches adopted for any software development project could lead to dissatisfied users who would eventually distrust and reject the final product. The user dissatisfaction in many cases with Covid-19 apps is in line with our previous findings.

In Australia, the government launched a contact tracing app called CovidSafe on 26th April 2020, and there was a significant level of mistrust on the app since its launch due to several reasons, including among others, privacy concerns, technical issues, lack of transparency and no end-user involvement. As of 28th June after 6 million downloads, it was reported that the app has not  been able to detect any new cases. Furthermore, publicly available information on the app’s contribution to pandemic control and the usage of the app is scarce.

Our Australian case motivated us to study other contact-tracing apps launched around the world and explore their acceptance by the target community. We were interested to see the responses of the users of these apps to understand what social, economical, cultural, political and technological factors contributed to the successes or failures of contact tracing apps.

We have studied 21 countries in our research and downloaded the user reviews from both Apple and Google app stores in order to analyse the voices of the users for these apps. In this blog series we will be presenting our analysis of the various social dimensions around user satisfaction/dissatisfaction, staring with the case of Australia.

We collected all the reviews for the apps listed below in the table (for the duration of  1st January 2020 – 30th July 2020) from Google Play and Apple’s App Store.

Looking at the comments of users for CovidSafe app, and comparing them to the reviews coming from apps launched by others countries, we observed the following issues:

Lack of Citizen Involvement

Non- Responsiveness to concerns   

  • There were numerous issues reported on CovidSafe app related to Bluetooth, Battery and Data Consumption, however there was no engagement shown by the development team to respond to the authors of the reviews. Whereas there was effective developer-user communication seen in app reviews from New Zealand, Germany and India that helped in building people’s trust for the app. 

Lack of understanding technological context of Australian people

  • The Australian society’s technology orientation was not considered properly during the planning phase of the app development. A significant number of Australians are iOS users, whereas in general contact tracing apps seem to work better on Android platforms. CovidSafe app was reported in early stages after launch to have issues with iOS platforms. Similar observation was made in the case of India’s national app ‘Aarogya Setu’ and its popularity on Google store, as the majority of the population use Android platform. Almost all the contact tracing apps around the world had dominant negative sentiments towards iOS versions.

Ambitious technical assumptions without cultural considerations

  • The success of CovidSafe app was dependent heavily on the significant proportion of the population downloading and keeping the app active while in public spaces. This worked in countries like China, Singapore and India where people seem to have high levels of trust in governments and the national culture is heavily leaning towards collectivism. The voluntary download model in a democratic society such as Australia with much lower trust index in government, lack of transparency in the process, concerns of data privacy and the national culture leaning towards individualism is not ideal. The success of this app was based on a great deal of ambitious assumptions.

No End-User Experience

  • CovidSafe app works in the background. The users do not see any immediate results and/or any functionality and hence there is no user experience. All they see is a simple green screen to tell them the app is active and no further actions required. What's happening in the background is invisible to the users. In contrast, the Indian app Aarogya Setu shows updates to the people on the number of positive cases in their vicinity. Similarly, New Zealand offered a different solution, wherein people can create a digital diary of the places they have visited by scanning the official QR codes. Such features in Indian and New Zealand apps, unlike in the Australian app, gives an impression to the users that they are in control and hence an integral part of the process. For CovidSafe app, once the app is installed the only possible way to control it is to turn-off the Bluetooth which will stop the functions of the app.

 Interface of CovidSafe App


 Interface of NZ Covid Tracer App

Aarogya Setu App | NPCI - National Payments Corporation of India ... India's Covid-19 Tracker App Aarogya Setu


Waiting for successful outcomes

  • Australians are still waiting to hear any successful outcome from this multi-million Dollar app. This has further depreciated any motivation or trust that 6 million downloads showed since its launch. In Germany, it is reported that 660 people who tested positive for Covid-19 had the opportunity to warn other people via Corona-Warn-App. A significant number of Australian users commented that they uninstalled the CovidSafe app after it caused Bluetooth and battery issues and they do not see any positive contribution of the app in helping to stop the spread of the virus. 

  • The inability to fulfil the purpose for which the app was launched has also been observed in the UK Government’s NHS-Covid19 that was taken off the app stores (until recently when the UK government announced its second phase with Google/Apple Framework).

  • While currently Melbourne is going through the second wave of Covid-19, the state government in Victoria has accessed the data collected through the app around 400 times, but it is reported to not have revealed any single case that wasn’t known through manual tracing. Hence, Australian citizens are yet to witness any tangible benefits of the multi-million dollar CovidSafe application, at the time of writing this article.

In conclusion: 

Every country has their own unique struggle with the COVID-19 virus. For a successful technological solution in form of contact tracing app, the political and cultural context of the country cannot be ignored. Following or replicating the app framework of Singapore seems to have not worked in Australia. According to the Cultural Compass of Hofstede, Australia resembles closely to Germany when it comes to power structures and individualism in citizens, and hence could have taken similar steps to build public trust. In conclusion, and as established in our previous research, lack of user involvement and user dissatisfaction can lead to software project failure, despite meeting the time and budget goals. This was the case for CovidSafe, and in our opinion if the CovidSafe app was meant for Australians, then Australians should have been involved in the design, implementation and launch of it.