We currently live in the era of mobile computing. It is conceivable, as smartphones and tablets gain worldwide popularity, that many individuals’ primary computing device will soon be a mobile “smart” device. A large contributor to this phenomenon is the continued rise of an “App Store” economy where users turn to increasingly complex applications to accomplish mainstream computing tasks that were previously accomplished primarily in a desktop computing environment.
This has created a fiercely competitive software marketplace where, if an application is not performing as expected due to bugs, crashes, or a lack of customer desired features, nearly half of users will simply abandon it for a similar application [1]. Therefore, for an application to be successful in a modern mobile ecosystem, development teams must be extremely efficient and effective during the software maintenance cycle, pushing regular updates to consumers. Unfortunately, several mobile-specific confounding factors such as hardware and platform fragmentation, API-instability and fault-proneness [2], and difficult-to-test features (e.g., sensors) make for a particularly challenging maintenance process.
This has created a fiercely competitive software marketplace where, if an application is not performing as expected due to bugs, crashes, or a lack of customer desired features, nearly half of users will simply abandon it for a similar application [1]. Therefore, for an application to be successful in a modern mobile ecosystem, development teams must be extremely efficient and effective during the software maintenance cycle, pushing regular updates to consumers. Unfortunately, several mobile-specific confounding factors such as hardware and platform fragmentation, API-instability and fault-proneness [2], and difficult-to-test features (e.g., sensors) make for a particularly challenging maintenance process.
Perhaps the most important software maintenance activity is bug report resolution. However, the manner in which bugs are currently reported to developers is far from ideal. For apps released in mobile marketplaces, most developers and teams must sort through potentially thousands of user-reviews where users may or may not describe bugs in a manner sufficient to allow bug reproduction and resolution. Additionally, a recent study has shown that only a reduced set of user reviews can be considered useful or informative to developers [3]. Open source applications or commercial applications undergoing internal or beta testing will often make use of existing issue tracking systems (e.g., Bugzilla, Mantis, JIRA, GitHub Issue Tracker) to report information related to mobile bugs and crashes. However, these bug reporting systems rely mostly on unstructured natural language bug descriptions and lack key features that would better support and facilitate accurate bug reporting for mobile applications.
A previous study has shown that there are typically three key types of information that developers consider most helpful when comprehending and fixing issue reports [4]: 1) detailed reproduction steps, 2) stack traces, and 3) re-playable test cases or scenarios. Unfortunately, this information is also typically the most difficult for reporters to provide, and in the context of mobile applications, is further complicated by a highly GUI-event driven nature. This highlights the typical lexical gap that exists between reporters of bugs and the developers that are attempting to read and fix the reports. Reporters, particularly end users and beta testers, typically have only a working functional knowledge of an application, whereas developers have intimate code-level knowledge; thus the primary task that a bug reporting system must accomplish is the bridging of this gap by providing the types of information listed above. The whole of these current issues highlight the need for an improved reporting mechanism for mobile applications.
The SEMERU research team has taken the first step towards addressing these problems by developing a novel bug reporting mechanism called Fusion [5] that operates under the following key insight: automated program analysis techniques can be used to bridge the lexical knowledge gap between reporters and developers. Currently, issue tracking systems operate with essentially no prior knowledge of the application that they support, and given the information that can be gleaned from automated program analysis techniques, we saw a natural opportunity to create a “smart” bug reporting mechanism for mobile apps. Fusion uses fully-automated static and dynamic analysis techniques to gather screenshots, GUI-component information, and app event-flow information, linking dynamically extracted information back to program source code to help facilitate users in reporting issues and aid developers in resolving bugs. During the reporting process, users leverage a smart web-form to fill out the reproduction steps for a bug using an <action, component> formula where the action is the touch event (e.g. tap, long-tap) and the component is the GUI component on the screen (e.g. button, checkbox). The form tracks the location of the user in the high level event-flow of the application, making auto completion suggestions to guide the reporting (e.g. if the user reports that they clicked on a button that leads to a new app screen, the system will only suggest components from the new screen). In the end, the developer is presented with a report that includes detailed reproduction steps, screenshots, and traceability links back to the source code.
A previous study has shown that there are typically three key types of information that developers consider most helpful when comprehending and fixing issue reports [4]: 1) detailed reproduction steps, 2) stack traces, and 3) re-playable test cases or scenarios. Unfortunately, this information is also typically the most difficult for reporters to provide, and in the context of mobile applications, is further complicated by a highly GUI-event driven nature. This highlights the typical lexical gap that exists between reporters of bugs and the developers that are attempting to read and fix the reports. Reporters, particularly end users and beta testers, typically have only a working functional knowledge of an application, whereas developers have intimate code-level knowledge; thus the primary task that a bug reporting system must accomplish is the bridging of this gap by providing the types of information listed above. The whole of these current issues highlight the need for an improved reporting mechanism for mobile applications.
The SEMERU research team has taken the first step towards addressing these problems by developing a novel bug reporting mechanism called Fusion [5] that operates under the following key insight: automated program analysis techniques can be used to bridge the lexical knowledge gap between reporters and developers. Currently, issue tracking systems operate with essentially no prior knowledge of the application that they support, and given the information that can be gleaned from automated program analysis techniques, we saw a natural opportunity to create a “smart” bug reporting mechanism for mobile apps. Fusion uses fully-automated static and dynamic analysis techniques to gather screenshots, GUI-component information, and app event-flow information, linking dynamically extracted information back to program source code to help facilitate users in reporting issues and aid developers in resolving bugs. During the reporting process, users leverage a smart web-form to fill out the reproduction steps for a bug using an <action, component> formula where the action is the touch event (e.g. tap, long-tap) and the component is the GUI component on the screen (e.g. button, checkbox). The form tracks the location of the user in the high level event-flow of the application, making auto completion suggestions to guide the reporting (e.g. if the user reports that they clicked on a button that leads to a new app screen, the system will only suggest components from the new screen). In the end, the developer is presented with a report that includes detailed reproduction steps, screenshots, and traceability links back to the source code.
 |
| Figure 1: An Example Fusion Bug Report |
In a user study with over 20 participants conducted by our lab, we found that users of Fusion system generated more reproducible bug reports for 14 real-world open source Android applications than a comparable system, the Google Code Issue Tracker. Particularly encouraging is that a subset of users in our study had no background in coding or computer science, but were still able to construct detailed, actionable reports. We believe that this is the first step towards creating more effective bug reporting systems for mobile and GUI-based applications. However, these reports could enable much more. The rich information contained within the reports could enable more effective methods of supporting other typically expensive or difficult maintenance tasks such as developer triaging or duplicate bug report detection.
You can access the Fusion tool, documentation, and demonstration videos at www.fusion-android.com.
References
[1] Mobile apps: What consumers really need and want https://info.dynatrace.com/rs/compuware/images/Mobile_App_Survey_Report.pdf
[2] Linares-Vásquez, M., Bavota, G., Bernal-Cárdenas, C., Di Penta, M., Oliveto, R., and Poshyvanyk, D., "API Change and Fault Proneness: A Threat to the Success of Android Apps", in Proceedings of 9th Joint Meeting of the European Software Engineering Conference and the 21st ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE’13)
[3] N. Chen, J. Lin, S. Hoi, X. Xiao, and B. Zhang. AR-Miner: Mining informative reviews for developers from mobile app marketplace. In 36th International Conference on Software Engineering (ICSE’14)
[4] N. Bettenburg, S. Just, A. Schro ̈ter, C. Weiss,
R. Premraj, and T. Zimmermann. What makes a good bug report? In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering, (SIGSOFT ’08/FSE-16)
[5]
K. Moran, M. Linares-Vásquez, C. Bernal-Cárdenas, and D. Poshyvanyk. Auto-completing bug reports for android applications. 10th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE’15)
This comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
ReplyDelete