Software Practitioner Views on Merge Conflicts: What is missing?

By: Nicholas Nelson (@nnelson8675), Oregon State University, USA and Shane McKee (@shanemmckee), Intel Corporation, USA

Associate Editor: Sarah Nadi, University of Alberta (@sarahnadi)

We develop software in teams. We write code in separate environments and attempt to integrate our changes, along with everyone else's changes, to form a coherent shared version of the software. Most of the time this process works, most of the time version control systems can easily integrate different versions of the code into one, but approximately 19% of the time there are problems [1][2]. These merge conflicts require human intervention to resolve conflicting changes to the code, and these interventions take time away from regular development.

Prior research efforts have focused on developing smarter merging algorithms [3][4], systems for proactively conflict detection [2][5][6], and discussing the merits of syntax- and semantic-aware merges [7][8]. However, these efforts have not considered the core component of collaborative software development: the practitioners that are writing the conflicting code. As a research community, we need to remember that software engineering involves software practitioners and their perspectives are critical for tuning any proposed solutions to make real-world impact. This fundamental rationale is why we reached out to practitioners; to obtain their perspectives on merge conflicts.

We found that practitioners rely on their own expertise in the conflicting code to understand and assess merge conflicts. Instead of using precise metrics calculated by tools, practitioners tend to rely on their own simple estimations. We additionally found that the complexity of merge conflicts and the size of merge conflicts factor heavily into the assessment of merge conflicts.

Fig 1. Perceptions of merge toolset support along dimensions of conflict complexity and conflict size

To get a better understanding of these two factors, we asked practitioners in the survey to relate their tools along these two dimensions. The combination of four bubble plots in Fig. 1 represents the results of these four scenario questions, and show the number of responses for a given effect level broken out into experience level groups. The size and depth of color within each bubble are also used to convey the size of responses for any given combination.

To better illustrate how to read and interpret these plots, let’s take an example participant with 1-5 years of experience who indicates that her merge toolset is Extremely Effective for small, simple merge conflicts, she would be represented in the bubble containing 15 in quadrant A1. She would also be represented in the bubble containing 9 in the bottom right plot (quadrant A4) if she indicated that her merge toolset was Moderately effective for large, complex merge conflicts.

Mean scores for each A1-A4 plot from Fig. 1 and the difference across dimensions

The greater picture between each of these plots is that the move from plot A1 to A2 (the vertical axis) represents a change only in the dimension of merge conflict size: from small to large. Similarly, the move from A1 to A3 (the horizontal axis) represents a change only in the dimension of merge conflict complexity: from simple to complex. Fig. 2 is an annotated version of Fig. 1 with mean scores for each plot listed in the corners. These scores represent the mean where responses of Extremely Effective is scored as 5 and Not at all is scored as 1. Using numerical analysis, we examined the impact of both dimensions on the mean score and found the shift from small to large merge conflict size (A1 to A2) results in a difference in mean response of 0.496, whereas the shift from simple to complex merge conflict complexity results in a difference in mean response of 0.930. The change in mean score based on a shift in complexity is more than double the mean score based on a shift in size.

These results suggest that merge tools are currently equipped to handle increases in the size of merge conflicts, but not as well equipped for increases in complexity. With the increasing amount of code being developed in distributed environments, researchers and tool developers need to scale their solutions in both dimensions if they are going to continue to be relevant and useful to practitioners.

Further examination and discussions on merge conflicts can be found in our paper:

S. McKee, N. Nelson, A. Sarma, and D. Dig, "Software Practitioner Perspectives on Merge Conflicts and Resolutions," in International Conference on Software Maintenance and Evolution (ICSME), 2017.

References:

[1] B.K. Kasi and A. Sarma, “Cassandra: Proactive conflict minimization through optimized task scheduling,” in International Conference on Software Engineering (ICSE), 2013, pp. 732–741. 

[2] Y. Brun, R. Holmes, M. D. Ernst, and D. Notkin, “Proactive detection of collaboration conflicts,” in International Symposium and European Conference on Foundations of Software Engineering (ESEC/FSE), 2011, pp. 168-178. 

[3] G. Taentzer, C. Ermel, P. Langer, and M. Wimmer, "A fundamental approach to model versioning based on graph modifications: from theory to implementation," in Software & Systems Modeling, 2014, ed. 13(1), pp. 239-272. 

[4] Y.S. Yoon and B.A. Myers "Supporting selective undo in a code editor," in International Conference on Software Engineering (ICSE), 2015, pp. 223-233. 

[5] A. Sarma, “Palantir: Enhancing configuration management systems with workspace awareness to detect and resolve emerging conflicts,” Ph.D. dissertation, University of California, Irvine, 2008. 

[6] M.L. Guimarães and A.R. Silva, “Improving early detection of software merge conflicts,” in International Conference on Software Engineering (ICSE), 2012, pp. 342–352. 

[7] D. Dig, K. Manzoor, R. Johnson, and T.N. Nguyen, “Refactoring-aware configuration management for object-oriented programs,” in International Conference on Software Engineering (ICSE), 2007, pp. 427–436. [8] J. J. Hunt and W. F. Tichy, “Extensible language-aware merging,” in International Conference on Software Maintenance (ICSM), 2002, pp. 511–520.