Monday, February 5, 2018

IEEE November/December Issue, Blog, and SE Radio Summary

The November/December Issue of IEEE Software, voted unanimously as best software magazine ever, is chock full of healthy goodness after a cheerful, yet gluttonous, holiday! This issue features numerous articles on building smart, context-aware healthcare systems. Along with articles on healthcare, this issue also features articles on other hot software topics including requirements engineering, agile development, and blockchain-based systems. Special in this issue is an article providing highlights from ICSE 2017.

As I mentioned, the feature topic of this issue of IEEE Software is smart and context-aware systems, with a focus on healthcare systems. This issue features the following articles on this topic:

In "The Elusiveness of Smart Healthcare," IEEE Software Editor in Chief Diomidis Spinellis speaks on his experiences with healthcare technology. Based on his experiences, he outlines challenges that lie ahead and should be considered when building smart healthcare systems.  Potential challenges aside,  a subset of article related to the feature topic showcase recent advances in building smart, context-aware healthcare software solutions. 

The authors of "In the Pursuit of Hygge Software" discuss how we can, and should, improve hygge in pervasive technologies, such as the ones often used in healthcare. For those who don't know (like me), hygge is a Danish and Norwegian word that essentially means feeling connected to others. The authors suggest that we can improve hygge by helping people find other people, places, and information that could help through their situation and propose a hygge-enabled software architecture. Similarly, in "Crowd-Based Ambient Assisted Living to Monitor the Elderly's Health Outdoors," the authors discuss how we can build healthcare systems that can help monitor the elderly's health when they're outdoors. Unlike existing technologies, however, their SafeNeighborhood approach uses crowdsourced information to improve inferences made by contextual and sensor data.

Some articles focused on the general idea of building context-aware systems, particularly on handling variability in context-aware systems.
Common amongst these articles is the idea of modeling context-variability and using these models to identify and deal with problems at run-time, without human intervention.
If you have any interest in understanding or improving smart healthcare systems, this issue is especially for you!

IEEE Software Blog

As usual, one feature item on the IEEE Software Blog is a summary of the IEEE Software September/October issue. If you're behind, make sure you check it out!

The past couple of months haven't seen too many new blog posts, but featured a mix of articles showcasing small and large scale research efforts. One article discusses an initiative called Naming the Pain in Requirements Engineering (NaPiRE). This initiative came together to conduct a global study on requirements engineering in industry with the goal of building a holistic theory on industry practices, trends, and problems.

The other articles in this issue discuss technical research. If you're conducting or interested in research related to StackOverflow, there's an article on asking good technical questions that might catch your attention. Also of interest to those who take interest in eye tracking studies (which are becoming more frequent in the SE research community), one article explores using eye tracking to automate traceability link recovery. 

SE Radio

Same hosts, new (hot) topics. 
Got a secret you need to keep? Want to learn how to manage your secret? There's an episode for that.
There are also episodes that can help with understanding and managing people, more specifically how IT architectures transform and adapt and how to hire and retain DevOps engineers.
Other episodes focused on understanding various aspects of the software we build. IEEE Software editor in chief Diomidis Spinellis step in to talk tools, practices and other topics relevant to performance optimization. Other topics discussed included how we as software engineers can improve our "security stature" and how Internet of Things (IoT) applications are built and used.

Monday, January 22, 2018

SEABED: An Open-Source Repository of Software Engineering Case Studies

by Veena Saini (NIT Jalandhar, India), Deepti Ameta (DAIICT, India), Ashish Sureka (Ashoka University, India), Paramvir Singh (NIT Jalandhar, India) and Saurabh Tiwari (DAIICT, India)

Associate Editor: Sridhar Chimalakonda (@ChimalakondaSri)

Case-Based Learning (CBL) is a non-traditional teaching methodology which helps in enhancing the learning skills. It helps in the deeper understanding of the concepts; involves students in active learning, enhances collaboration among team members, helps in nurturing analytic and interpersonal skills through well-defined cases. We looked for most prevalent teaching pedagogies being used in different domains like Medical, Law, and Business education that require brainstorming and we found CBL to be an effective approach in enhancing students learning levels by helping them to think in different perspectives for the same problem.

Why CBL for Software Engineering?

Software Engineering is a highly practice oriented field, concerned with the development, operation, and maintenance of software, hence providing practical knowledge for its applicability in real world becomes equally important. Software development process is primarily based on a set of well-defined requirements in which understanding and prioritizing them is important for the success of any product. It is very important to bridge the gap between the theoretical concepts at university level and practical implementation of the concepts at industry level, hence we found CBL to be an effective approach in teaching some of the SE concepts as it is a student-centric teaching methodology which focuses on learning objectives like critical thinking, brainstorming, discussions, and understanding real life scenarios with multiple perspective solutions.

Our progress to bring CBL into practice through SEABED

The backbone of SEABED [1] is the variety of cases and vibrant SEABED community. We are building up our SEABED community by interacting with SE practitioners, enthusiasts and researchers all over the globe. Gradually SEABED community is growing up with potential users. Along with all the researchers and SE enthusiasts, we also welcome thesis students to contribute and take this learning platform to a new level. Our idea was to create a web-based platform as a hub for SE CBL practitioners. Teachers, students, researchers, instructors, or anyone can be benefited from CBL and can find relevant cases from our open-source case repository in Case Collection section. Cases are categorized according to the various phases of Software Development Life Cycle (SDLC).
It allows the interested users to contribute with their own case problems as PDF document (Case Submission).  They can also upload the experience report about the CBL session, its empirical analysis and students’ feedback after conducting some CBL sessions in the University. They can revise any existing case available at SEABED, by providing proper justification document that tells the rationale behind the amendments (Case Evolution). 

Case Example: Twitter started as a side project of Odeon Inc. in 2006. It had immense growth nearly 1000% growth/year and soon became the micro blogging platform of choice for majority of Internet users. It had 400,000 tweets per quarter in 2007. This grew to 100 million tweets per quarter in 2008. By 2010 there were around 175 million users, 90 million tweets per day and around 500 million searches per day. Twitter was initially built with time to market in mind. So the architecture and technology to build twitter was chosen such that they can build the site in a very short time. Twitter was not designed with this kind of growth in mind. Your team has to come up with a new architecture that addresses the scalability problems of twitter.
Challenges 1: What are the architectural drivers, assumptions and major constraints? Give details of at least 5 decisions related to major architectural strategies. • 2: Give the architecture in terms of system decomposition (as a diagram and text), structure, and connector and component responsibilities.

This is a sample case [2] for Software Engineering (SE) case-based learning (CBL) practice with challenging questions.

CBL Implementation at different universities

Experiment 1 - IIIT Hyderabad, India: We begin our journey when CBL was first implemented at IIIT Hyderabad for practicing SE concepts. They proposed a case based learning environment - Case Oriented Software Engineering Education model (COSEEd). It embeds problem solving as a core skill with cases as the primary learning objects. We implemented 4 course offerings with COSEEd considering UG, PG, and PGSSP (Industry participants). In all the offerings, students felt that CBL helped them in gaining learning objects along with an improvement in their communications skills. This model was successful in covering cognitive goals in SE education. The research work can be reached at [3].

Experiment 2 - NIT Jalandhar, India:
Here CBL session was conducted for 89, 3rd year UG students. CBL is found to be effective with an agreement of 74.53% students, who were able to understand all five learning principles of learning, critical thinking, engagement, communication skills, and team work. Experience report [4] of this implementation is available at SEABED. After this exercise, SEABED platform was created and we proposed a case writing template. The research work can be reached at [5].

The architecture of SEABED is given below:

Experiment 3 - DAIICT Gandhinagar, India:  
The CBL was introduced in two different SE courses, Software Testing and Requirements Engineering, with a number of CBL sessions. Firstly, we applied CBL for Software Testing Discipline and got appreciable responses at DAIICT, Gandhinagar, India. This was the first time when students were introduced with the concepts of CBL. This drove us to conduct more CBL sessions at DAIICT and the second time we applied CBL for practicing Requirements Engineering Concepts by providing two different RE cases [6][7] to student teams and asked them to bring solutions (PowerPoint presentations) to check the effectiveness of CBL by evaluating their results on different parameters. We also intend to perform several experiments on the same. After every CBL session we asked student teams and Teaching Assistants (TAs) to give feedback and we got positive responses this time too. We further aim to expand this by considering new research techniques on CBL implementation.

So the overall analysis of the CBL experiments conducted at different Universities reveals that CBL teaching methodology can be used in teaching some of the Software Engineering concepts efficiently, supported by well-designed cases. It helps students’ in in solving real world problems by engaging themselves in thinking and discussing activities.

What’s next (future opportunities): Case-based learning in Software Engineering domain is effective and there are several sub-areas that are still unexplored. Our platform does not only help students to assimilate the concepts well, but also provides a promising research area for SE practitioners and enthusiasts to submit and discuss efficient cases. We further aim to expand this work by conducting more useful sessions with other universities and experimenting CBL with more detailed observations, analysis and results. We also invite MS, M.Tech and PhD scholars to collaborate with us on CBL project.

[3] Kirti Garg, Ashish Sureka and Vasudeva Verma. 2015. A Case Study on Teaching Software     Engineering Concepts using a Case-Based Learning Environment.
[5]V. Saini, P. Singh and A. Sureka, "SEABED: An Open-Source Software Engineering Case-Based       Learning Database," 2017. IEEE 41st Annual Computer Software and Applications Conference           (COMPSAC), Turin, 2017, pp. 426-431. doi: 10.1109/COMPSAC.2017.204
[8] D. Kundra and A. Sureka, “An Experience Report on Teaching Compiler Design Concepts Using       Case-Based and Project-Based Learning Approaches,” 2016 IEEE Eighth International                         Conference on Technology for Education (T4E), Mumbai, 2016, pp. 216-219. doi:                               10.1109/T4E.2016.052.

Monday, December 4, 2017

Gazes to Traces: Tracing Developers’ Eyes to Automate Traceability Link Recovery

by Bonita Sharif, Youngstown State University (@shbonita), and Huzefa Kagdi, Wichita State University
Associate Editor: Christoph Treude (@ctreude)

There is no denying that large-scale software systems evolve.  Software developers are routinely faced with new features and bugs that drive essential changes.  Oftentimes, they are not even the original authors of the code they need to change. They need to piece together many aspects to realize these changes whilst juggling various project parameters such as quality assurance and deadlines. The knowledge space for this existential activity typically spans several artifacts from the problem (e.g., requirements) and solution (design and implementation) domains.  The explicit connectivity among these artifacts is often missing in practice [1, 2], which further adds to the developer’s plea. Questions that are left to be answered include “Where is the relevant code to this feature/bug?” or “What would be the design impact due to a change request?”.  The promise of a software traceability tool is to establish (recover) and maintain (evolve) links among artifacts as software evolves [3]. Traceability benefits are projected in several key software development tasks such as program comprehension [4, 5] and impact analysis [6, 7], which address the aforementioned questions.

Over the years, software engineering researchers have proposed techniques for automatically recovering and maintaining (explicit) traceability links among software artifacts.  The application of Information Retrieval (IR) techniques is a popular and heavily experimented choice. It is an artifact-centric approach that has an underlying model of textual similarities in software artifacts.   The expressiveness, effectiveness, and usefulness rely on the artifacts, and their state of availability and quality during the software’s lifecycle. 

What We Did

Our research direction was to investigate a human-centric approach to traceability.  The underlying premise of our approach is based on what humans look at while they are performing software-engineering tasks, including those of bug fixes and implementing new features.   The prerequisite to such an approach is that it should be unobtrusive to developers and blend into the background.  We use an eye tracker to collect developers' gazes on software artifacts while they work on their tasks within the IDE (we use Eclipse; however, the same concept applies to others). Our eye tracking infrastructure is called iTrace [8, 9]. It seamlessly works within the IDE to map eye gazes to source code elements on large files in the presence of scrolling and context switching between files. A preliminary version of iTrace is available at An enhanced release with additional support for IDEs and Web browsers is planned for the near future.

We first did a pilot study [10] to determine the feasibility of using eye gaze for traceability link recovery. After seeing promising results, we conducted a larger, realistic study [11] with thirteen software developers who were asked to perform bug-localization tasks for eight submitted bug reports in Jabref (an open-source reference management system). The gaze-link algorithm uses gaze data of developers during the session using certain heuristics and weights. In a bug fixing task for example, the main premise is that as you get closer to fixing a bug you will focus on the part(s) of the code that is most related to the bug report, thus having more weight. We compared trace link results of our gaze-link algorithm with IR methods such as Latent Semantic Indexing (LSI) and Vector Space Model (VSM). 

What We Found

The gaze-link algorithm outperforms both LSI and VSM in terms of precision and recall with respect to the commit oracle (how the JabRef developers fixed the bug). We recorded an average precision of 55% and recall of 67% for all tasks. The gaze-link algorithm outperformed in 6 out of the 8 tasks.  Another set of developers found the links generated with iTrace to be significantly more useful than the IR links in a majority of the tasks. The gaze-link algorithm underperforms when the developer prematurely attempts to fix the bug without the adequate understanding of the bug or its solution. Interestingly and perhaps surprisingly, the gazes captured from a developer in cases where they did not quite fix the bug, but was close enough, ended up being helpful to another developer as a starting point towards eventually fixing the bug. 

See Table 1 for an example of links generated for a bug report across the different approaches at both class and method level granularity. The gaze algorithm is crucial to weeding out irrelevant and stray glances (shown in ETraw). Results from the gaze link algorithm (ETweighted) are more specific than the rankings from current IR methods (LSI and VSM). 

Table 1. An example of class and method level source code elements found linked to Bug ID: 1489454 titled “Acrobat Launch fails on Win 98”. Variables are shown in square brackets. ETraw represents raw eye tracking data before the algorithm is run, ETweighted is the result of running gaze-link on the gazes. The Commit is our oracle representing the fix by the JabRef developer.
Concluding Remarks

The eye tracker captures the gaze data without any additional effort on the part of the developers. This property allows it to be as effortless as we can get to provide traceability under the hood while developers work.  An added benefit of using the eye tracker is that we also learn about those hidden links that information retrieval methods have a hard time finding as they are related to tacit developer knowledge in various related code entities. Imagine a software development world where your gazes could inform things you do. The transparency and minimal effort required by developers makes gaze tracking an attractive possibility. It is reasonable to imagine a future in which eye trackers capture information while we work to help us with many more tasks than just software traceability. 

We refer interested readers to the research article titled “Eye Movements in Software Traceability Link Recovery” [11].


[1] B. Ramesh, "Factors Influencing Requirements Traceability Practice," Communications of the ACM, vol. 41, pp. 37-44, December 1998.
[2] P. Mader, O. Gotel, and I. Philippow, "Motivation matters in the traceability trenches," in RE, 2009, pp. 143-148.
[3] D. J. Palmer, "Traceability," in Software Engineering, M. Dorfman and R. H. Thayer, Eds., ed: Wiley-IEEE Computer Society Press, Los Alamitos, California, 1996, pp. 266-276.
[4] A. De Lucia, D. P. Massimiliano, and R. Oliveto, "Improving Source Code Lexicon via Traceability and Information Retrieval," IEEE Transactions on Software Engineering pp. 205-227, 2011.
[5] A. De Lucia, M. D. Penta, R. Oliveto, and F. Zurolo, "Improving Comprehensibility of Source Code via Traceability Information: a Controlled Experiment," in 14th IEEE International Conference on Program Comprehension (ICPC'06), Athens, Greece, 2006, pp. 317-326.
[6] G. Antoniol, G. Canfora, G. Casazza, and A. De Lucia, "Identifying the starting impact set of a maintenance request," in 4th CSMR 2000, pp. 227–230.
[7] A. von Knethen, "Change-Oriented Requirements Traceability. Support for Evolution of Embedded Systems," in International Conference on Software Maintenance (ICSM 02), 2002.
[8] B. Sharif and J. I. Maletic, "iTrace: Overcoming the Limitations of Short Code Examples in Eye Tracking Experiments," in 32nd IEEE International Conference on Software Maintenance and Evolution (ICSME), Technical Briefing, Raleigh, NC, USA, 2016, p. 647.
[9] T. Shaffer, J. Wise, B. Walters, S. Müller, M. Falcone, and B. Sharif, "iTrace: Enabling Eye Tracking on Software Artifacts Within the IDE to Support Software Engineering Tasks," in ESEC/FSE 2015, Bergamo, Italy, 2015, pp. 954-957.
[10] B. Walters, T. Shaffer, B. Sharif, and H. Kagdi, "Capturing Software Traceability Links from Developers' Eye Gazes," in 22nd International Conference on Program Comprehension (ICPC), Hyderabad, India, 2014, pp. 201-204.
[11] B. Sharif, J. Meinken, T. Shaffer, and H. Kagdi, "Eye Movements in Software Traceability Link Recovery," Empirical Software Engineering Journal, vol. 22, pp. 1063-1102, 2017.

Monday, November 20, 2017

Can We Trust the Stack Overflow Netiquette? Evidence-based Guidelines for Asking Good Technical Questions

by Fabio Calefato, University of Bari, ITALY (@fcalefato), Filippo Lanubile, University of Bari, ITALY (@lanubile), and Nicole Novielli, University of Bari, ITALY (@NicoleNovielli)
Associate Editor: Christoph Treude (@ctreude)

Several thousands of developers daily head to Stack Overflow (SO) for asking technical questions, hoping to receive swift help and fix the issues that they have been facing. To increase the chances of getting help from others, the SO community provides members with detailed guidelines on how to write more effective questions (e.g., see [9]). These official recommendations also include those provided by Jon Skeet, the highest reputation member, whose guidelines have become over time a de facto standard for the community [10].

For example, SO states that the site is “all about getting answers. It's not a discussion forum. There's no chit-chat.” Thus, askers are recommended to avoid “greetings and sign-offs […], as they’re basically a distraction,” which are also supposed to be edited out by other users [10]. Still, many askers finish their questions showing gratitude in advance towards potential helpers. Why do they go against this explicit recommendation? Are they just unaware of it or do they feel that having a positive attitude may attract more potential solutions?

In our work [5], we provide an evidence-based netiquette for writing effective questions by empirically validating several SO guidelines, retrieved from both the community and previous empirical studies on Q&A sites. Specifically, we analyzed a dataset of 87K questions by combining a logistic regression analysis with a user survey, first, to estimate the effect of these guidelines on the probability of receiving a successful answer and, then, to compare their actual effectiveness to that perceived by SO users.

Actionable Factors for Good Questions

Figure 1. Our conceptual framework of success factors for writing good questions in Stack Overflow. The success of a question is defined as the probability of receiving an answer that is accepted as solution. Affect, Presentation Quality, and Time are the three actionable success factors of interest, while Reputation (non-actionable) serves as a control factor. Each success factor is related to several metrics used in turn to inform guidelines.
We developed a conceptual framework (Figure 1) for the analysis of the aspects that may influence the success of a question. We focused on 9 actionable metrics that can be acted upon by developers when writing a question and, therefore, are useful to inform guidelines. These metrics are grouped into 3 success factors, concerning affect (i.e., the positive/negative tone conveyed by a question), presentation quality (i.e., the readability and comprehensibility of its text), and time (i.e., when to post it). As per the asker’s reputation, it is included in our model, but only as a non-actionable control factor. While reputation has been already found to help people receive more and fasters answers in other communities such as Reddit [1], one can’t really do anything to increase their score just when posting a question.

Findings: Evidence vs. User Perception

Table 1. The evidence-based netiquette for effective question writing on Stack Overflow. Guidelines are shown in bold when supported by evidence.
# Guideline Success factor Empirical support User perception Source
1 Write questions using a neutral emotional style Affect Yes Effective Skeet [10], SO Help Center [9], Kucuktunc et al. [7], Bazelli et al. [3]
2 Provide sample code and data Presentation quality Yes Effective Skeet [10], Asaduzzaman et al. [2], Duijn et al. [6], Treude et al. [11]
3 Use capital letters where appropriate Presentation quality Yes Effective Skeet [10]
4 Be concise Presentation quality Yes Ineffective Skeet [10]
5 Use short, descriptive question titles Presentation quality No Ineffective Skeet [10]
6 Provide context through tags Presentation quality No Effective Skeet [10]
7 Provide context through URLs Presentation quality No Effective Ponzanelli et al. (partially) [8]
8 Be aware of low-efficiency hours Time Yes Ineffective Bosu et al. [4]

Table 1 reports the main findings of our study, from which some interesting observations emerge. For 4 out of the 8 guidelines studied, we found that the perceived and actual effectiveness match. In particular, SO users are correct to think that minding their tone (#1), providing snippets with examples (#2), and avoiding the inappropriate use of capital letters (#3) all increase the probability to receive a successful answer to their questions. Instead, the use of short titles is neither perceived nor found to be an effective guideline (#5).

Perhaps even more interesting considerations arise from the remaining cases of mismatch, though. First, SO users seem to be unaware that writing questions concisely (#4) and posting them during GMT evening hours (#8) increase the chances of a question to be resolved. Regarding efficiency times, Bosu et al. [4] have speculated that these time slices are the most successful because they correspond to the working time in the USA. Second, contrary to users’ perception, we found that providing context for questions by adding extra tags (#6) and including in the text URLs to external resources (#7) have no positive effects on the chance of getting a successful answer.

Final Remarks

One of the greatest issues with Stack Overflow is the sheer number of unresolved questions. Currently, the site hosts almost 15 million questions of which about 7 million are still unresolved. Helping users ask “better” questions can increase the number of those resolved. One way to do so is increasing the awareness among community members about the existence of effective question-writing guidelines while also trimming down the list to only those supported by evidence. 

For more information about our study, please refer to our paper “How to Ask for Technical Help? Evidence-based Guidelines for Writing Questions on Stack Overflow” [5].


[1] T. Althoff, T., Danescu-Niculescu-Mizil, C., and Jurafsky, D. How to Ask for a Favor: A Case Study on the Success of Altruistic Requests. In Proc. of the 8th Int’l Conf. on Weblogs and Social Media (ICWSM’14), 2014.
[2] M. Asaduzzaman, A.S Mashiyat, C.K. Roy, K.A. Schneider. 2013. Answering questions about unanswered questions of Stack Overflow. In Proc. of the 10th IEEE Working Conf. on Mining Software Repositories (MSR 2013), pp. 97-100.
[3] B. Bazelli, A. Hindle, and E. Stroulia. On the Personality Traits of StackOverflow Users. In Proc. of Int’l Conference on Software Maintenance (ICSM '13), 2013, pp. 460-463.
[4] A. Bosu, C.S. Corley, D. Heaton, D. Chatterji, D.J. Carver, and N.A. Kraft. Building Reputation in StackOverflow: An Empirical Investigation. In Proc. of 10th IEEE Working Conf. on Mining Software Repositories (MSR’13), 2013, pp. 89-92.
[5] F. Calefato, F. Lanubile, N. Novielli. How to Ask for Technical Help? Evidence-based Guidelines for Writing Questions on Stack Overflow. Information and Software Technology, 2017.
[6] M. Duijn, A. Kučera, and A. Bacchelli. Quality questions need quality code: classifying code fragments on stack overflow. In Proc. of 12th Working Conf. on Mining Software Repositories (MSR '15), 2015, pp. 410-413.
[7] O. Kucuktunc, B.B. Cambazoglu, I. Weber, and H. Ferhatosmanoglu. A large-scale sentiment analysis for Yahoo! answers. In Proc. of 5th Int’l Conf. on Web Search and Data Mining (WSDM '12), 2012, pp. 633-642.
[8] L. Ponzanelli, A. Mocci, A. Bacchelli, and M. Lanza. Understanding and Classifying the Quality of Technical Forum Questions. In Proc. of 14th Int’l Conf. on Quality Software (QSIC’14), 2014, pp. 343-352.
[9] Stack Overflow Help Center, Be nice (last accessed: Nov. 2017).
[10] J. Skeet, 2010. Writing the Perfect Question (last accessed: Nov. 2017).
[11] C. Treude, O. Barzilay, and M-A. Storey. How do programmers ask and answer questions on the web? (NIER track). In Proc. of the 33rd Int’l Conf. on Software Engineering (ICSE '11), 2011, pp. 804-80.

Monday, November 13, 2017

IEEE September/October Issue, Blog, and SE Radio Summary

Associate Editor: Brittany Johnson (@drbrittjay)

The September/October Issue of IEEE Software, top magazine for all things software, again delivers a range of interesting topics for thought and discussion in the SE community. The topics discussed in this issue included software requirements and testing, DevOps, gamification, and software architecture.

The feature topic in this issue of IEEE Software was software testing. This issue featured the following articles related to software testing:

For those interested in getting a quick overview of how testing is used and viewed in the software community, "Software Testing: The State of the Practice" and "Worlds Apart: Industrial and Academic Focus Areas in Software Testing" are great articles to read.
In "Software Testing: The State of the Practice", Kassab and colleagues conducted a comprehensive survey of software practitioners to gain a much needed understanding of the state-of-the-art in software testing and quality assurance practices.
Their ultimate goal was to gather and be able to disseminate best practices to community based on their findings, such as practicing test-driven development.

In "Worlds Apart: Industrial and Academic Focus Areas in Software Testing", Garousi and Felderer discusses the problem of the often disparate efforts between industry and academy when it comes to academia doing research that matters and industry applying this research to their work.
Their focus was on software testing and how we can improve industry-academic collaborations.
They observed titles of presentations given at industrial and academic conferences and, using Wordle to create word clouds, found that these communities often focus on different aspects of testing. 
For example, industry conferences most often talk about automation while academic conferences most often talk about models. The authors make suggestions, such as inviting practitioners to research-intensive SE conferences to gain their perspective on the research we're doing.

Some of the articles in this issue focus on a specific subset of testing, such as GUI testing. 
In "Adaptive Virtual Gestures for GUI Testing on Smartphones", Hsu and colleagues propose an approach to testing mobile software called adaptive GUI testing (AGT). AGT allows for faster cross-device testing and is based on touch events known as visually oriented gestures (VOG). 
Along the same lines, in "Replicating Rare Software Failures with Exploratory Visual GUI Testing", Elégroth and colleagues discuss the benefits of visual GUI testing (VGT) based on their own experiences. They speak about how VGT can be used to replicate failures and push forward analysis of infrequent or nondeterministic failures. 

IEEE Software Blog

Behind on your IEEE Software issues? No worries! Summaries from the May/June and July/August issues of IEEE Software are up in the IEEE Software Blog. On the technical side, this issue features blog posts on merge conflicts and using code metrics to predict fault and change proneness. On the not-so-technical (but obviously still software-related) side, this issue features articles that explore behaviors in OS projects, such as understanding the mindset of one-time contributors and female participation in OS software. Among the not-so-technical blog posts, a developer from the Debian project observes and analyzes the social structure within the project based on the cryptographic keyring of the project.

SE Radio

This issue of SE Radio didn't see any new hosts but lots of interesting discussions on interesting topics, ranging from choosing freelancing as a career path to machine learning for predictive applications.  For the software engineers out there, Haroon Meer, founder of Thinkst, spoke with SE Radio on what's going wrong with network security in our applications and what we can do better.
For aspiring software engineers out there, Evgeny Shadchnev discussed code schools, or programs available that prepare students to become software developers in a few months.
For those in management positions, or aspiring to be a manager one day, Ron Lichty joined SE Radio to discuss difficulties and suggestions for managing programmers. He also provided a link to his blog on the topic. Also useful is the information provided by Harsh Sinha, VP of TransferWise, on another type of management known as product management.

Monday, November 6, 2017

NaPiRE: Global Study on Industrial Practices and Problems

By Daniel Méndez and Birgit Penzenstadler
Edited by Mei Nagappan

40 years after Requirements Engineering (RE) was acknowledged for the first time as an independent discipline in an issue of the Transactions of Software Engineering, it has received much attention in research and practice due to its importance to software project success. The importance of RE cannot be refuted as many decisions in software projects are rooted therein; same holds for the problems. As Nancy Leveson (MIT) was cited in an article in The Atlantic:
The serious problems that have happened with software have to do with requirements, not coding errors.
In fact, it has become conventional wisdom that many problems emerge from "insufficient RE" and that the later the problems are discovered, the harder (and, thus, more expensive) they become to fix. Yet it remains difficult to obtain reliable empirical figures that would describe what "insufficient RE" exactly means, how it manifests in the processes and artefacts created, and what root causes and effects this has. Such figures are, however, critical determinants for a problem-driven research, i.e., to support contributions that are in tune with the problems they intend to solve. As a matter of fact, the state of empirical evidence in RE is still weak and much of everyday industrial practices and research as well are both dominated by wisdom and beliefs rather than being governed by empirical evidence. This results in research contributions with a potentially low practical impact and, in the end, a continuously increasing disconnect between research and practice.

The Naming the Pain in Requirements Engineering Initiative

Motivated by this situation where we need a stronger body of knowledge about the state of the industrial practice in RE , we initiated the Naming the Pain in Requirements Engineering (short: NaPiRE) initiative in 2012. The initiative constitutes a globally distributed family of practitioner surveys on Requirements Engineering (RE) with the overall objective to build a holistic theory on industrial practices, trends, and problems. It is run by the RE research community with the purpose of serving researchers and practitioners alike and is, in fact, the first of its kind. In a nutshell, each survey replication aims at distilling
  • the status quo in company practices and industrial experiences,
  • problems and how those problems manifest themselves in the process, and
  • what potential success factors for RE are.
To achieve the long-term objective of increasing the practical impact of research, the community behind NaPiRE has committed itself to open science principles. All publications, but also results obtained from the studies, are open to the public, including the anonymised raw data, codebooks, and analysis scripts. This shall support other researchers in running independent data analyses, interpretations, and replications, and it helps practitioners in assessing their own current situation in context of a broad picture illustrating overall industrial practices.

Current State of NaPiRE

While our first survey round focused on surveying German companies and had a first replication in the Netherlands, the second replication was run in 2014/15 and already took place in 10 countries with the support of 23 researchers. That run yielded fruitful insights into contemporary problems practitioners experience as well as into the criticality of those problems as shown in the following chart.

Top Problems in Requirements Engineering as reported in "Naming the Pain in Requirements Engineering: Contemporary Problems, Causes, and Effects in Practice" (Empirical Software Engineering Journal)[/caption] The colour coding visualises the criticality of the problems in the sense of illustrating the extent to which problems are seen as the main reason for project failure. We can see, for example, that incomplete requirements constitute the most frequently stated problem. At the same time, we can also see that moving targets, although ranked as the fourth most frequent problem, becomes the top priority problem when considering the project failure ratios alone. An overview of the further results including more fine-grained analyses of the root-causes and the effects going beyond a simple notion of "project failure", can be taken from the publications on the project website Motivated by our past success in revealing insights into the status quo, but also in being able to transfer those analytical results into first constructive methods, e.g. in the context of risk management, we have initiated the third replication. By now, the NaPiRE community has grown into an international alliance of nearly 60 researchers who share the vision of contributing our part in increasing the practical impact of research contributions to RE.

Call for Participation

We are reaching out to you, the IEEE Software readers, as a highly relevant community of software practitioners. Please volunteer 20 minutes of your valuable time to contribute your insights and experience by participating in the current run of the survey. To foster problem-driven research with high practical impact, we depend on your input. The link to the survey is: (open until the end of December).

Sunday, October 29, 2017

Software Practitioner Views on Merge Conflicts: What is missing?

By: Nicholas Nelson (@nnelson8675), Oregon State University, USA and Shane McKee (@shanemmckee), Intel Corporation, USA
Associate Editor: Sarah Nadi, University of Alberta (@sarahnadi)
We develop software in teams. We write code in separate environments and attempt to integrate our changes, along with everyone else's changes, to form a coherent shared version of the software. Most of the time this process works, most of the time version control systems can easily integrate different versions of the code into one, but approximately 19% of the time there are problems [1][2]. These merge conflicts require human intervention to resolve conflicting changes to the code, and these interventions take time away from regular development.
Prior research efforts have focused on developing smarter merging algorithms [3][4], systems for proactively conflict detection [2][5][6], and discussing the merits of syntax- and semantic-aware merges [7][8]. However, these efforts have not considered the core component of collaborative software development: the practitioners that are writing the conflicting code. As a research community, we need to remember that software engineering involves software practitioners and their perspectives are critical for tuning any proposed solutions to make real-world impact. This fundamental rationale is why we reached out to practitioners; to obtain their perspectives on merge conflicts.
We found that practitioners rely on their own expertise in the conflicting code to understand and assess merge conflicts. Instead of using precise metrics calculated by tools, practitioners tend to rely on their own simple estimations. We additionally found that the complexity of merge conflicts and the size of merge conflicts factor heavily into the assessment of merge conflicts.

Fig 1. Perceptions of merge toolset support along dimensions of conflict complexity and conflict size

To get a better understanding of these two factors, we asked practitioners in the survey to relate their tools along these two dimensions. The combination of four bubble plots in Fig. 1 represents the results of these four scenario questions, and show the number of responses for a given effect level broken out into experience level groups. The size and depth of color within each bubble are also used to convey the size of responses for any given combination.
To better illustrate how to read and interpret these plots, let’s take an example participant with 1-5 years of experience who indicates that her merge toolset is Extremely Effective for small, simple merge conflicts, she would be represented in the bubble containing 15 in quadrant A1. She would also be represented in the bubble containing 9 in the bottom right plot (quadrant A4) if she indicated that her merge toolset was Moderately effective for large, complex merge conflicts.

Mean scores for each A1-A4 plot from Fig. 1 and the difference across dimensions

The greater picture between each of these plots is that the move from plot A1 to A2 (the vertical axis) represents a change only in the dimension of merge conflict size: from small to large. Similarly, the move from A1 to A3 (the horizontal axis) represents a change only in the dimension of merge conflict complexity: from simple to complex. Fig. 2 is an annotated version of Fig. 1 with mean scores for each plot listed in the corners. These scores represent the mean where responses of Extremely Effective is scored as 5 and Not at all is scored as 1. Using numerical analysis, we examined the impact of both dimensions on the mean score and found the shift from small to large merge conflict size (A1 to A2) results in a difference in mean response of 0.496, whereas the shift from simple to complex merge conflict complexity results in a difference in mean response of 0.930. The change in mean score based on a shift in complexity is more than double the mean score based on a shift in size.
These results suggest that merge tools are currently equipped to handle increases in the size of merge conflicts, but not as well equipped for increases in complexity. With the increasing amount of code being developed in distributed environments, researchers and tool developers need to scale their solutions in both dimensions if they are going to continue to be relevant and useful to practitioners.
Further examination and discussions on merge conflicts can be found in our paper:
S. McKee, N. Nelson, A. Sarma, and D. Dig, "Software Practitioner Perspectives on Merge Conflicts and Resolutions," in International Conference on Software Maintenance and Evolution (ICSME), 2017.
[1] B.K. Kasi and A. Sarma, “Cassandra: Proactive conflict minimization through optimized task scheduling,” in International Conference on Software Engineering (ICSE), 2013, pp. 732–741. 
[2] Y. Brun, R. Holmes, M. D. Ernst, and D. Notkin, “Proactive detection of collaboration conflicts,” in International Symposium and European Conference on Foundations of Software Engineering (ESEC/FSE), 2011, pp. 168-178. 
[3] G. Taentzer, C. Ermel, P. Langer, and M. Wimmer, "A fundamental approach to model versioning based on graph modifications: from theory to implementation," in Software & Systems Modeling, 2014, ed. 13(1), pp. 239-272. 
[4] Y.S. Yoon and B.A. Myers "Supporting selective undo in a code editor," in International Conference on Software Engineering (ICSE), 2015, pp. 223-233. 
[5] A. Sarma, “Palantir: Enhancing configuration management systems with workspace awareness to detect and resolve emerging conflicts,” Ph.D. dissertation, University of California, Irvine, 2008. 
[6] M.L. Guimarães and A.R. Silva, “Improving early detection of software merge conflicts,” in International Conference on Software Engineering (ICSE), 2012, pp. 342–352. 
[7] D. Dig, K. Manzoor, R. Johnson, and T.N. Nguyen, “Refactoring-aware configuration management for object-oriented programs,” in International Conference on Software Engineering (ICSE), 2007, pp. 427–436. [8] J. J. Hunt and W. F. Tichy, “Extensible language-aware merging,” in International Conference on Software Maintenance (ICSM), 2002, pp. 511–520.