On the Effectiveness of Source Code Metrics for Fault and Change Proneness Prediction in Industrial Automation Domain

by Lov Kumar (NIT Rourkela, India) and Ashish Sureka (ABB, India) 
Associate Editor: Sridhar Chimalakonda (@ChimalakondaSri)

Domain Specific Programmable Logic Controller (PLC) Languages

Several control systems used for factory and industrial automation of electromechanical processes are developed using Programmable Logic Controllers (PLC). PLCs are specifically adapted for the control of a manufacturing process and programmed using domain specific languages. The International Electro-technical Commission (IEC) (a non-profit international standards organization) developed a standard called as IEC 61131-3 which defines the basic programming elements, syntactic and semantic rules for text-based and graphical or visual programming languages for programming PLCs [IEC]. Structured Text (ST) is one the text-based PLC programming languages defined by the IEC 61131-3 standard and is widely used in industrial automation engineering application development. Similarly, Ladder Diagram (LD) programming language is a visual programming language and is one of the five languages defined in the IEC 61131-3 standard. ST is a domain specific language and several of its language features and programming constructs are different than that of general purpose programming languages such as Java, C++, C# and Python. ST has several characteristics which are different than that of general purpose programming languages because the primary purpose of a PLC is to control an industrial process and the ST language is defined for solving problems in a specific domain of factory and industry process automation [Roos2008].

Correlation between Source Code Metrics and Software Quality for PLC Systems

There has been a lot of work done in the area of predicting change proneness and faulty components or modules of software using source code metrics for general purpose programming languages. However, defining software metrics and investigating their impact on important software engineering prediction problems such as change proneness and faulty module identification is relatively unexplored in the domain of applications developed using IEC 61131-3 languages [Kumar2016] [Kumar2017]. We believe that one of the primary reasons of lack of empirical studies in the area of source code analysis for PLC programming languages and their impact on software maintainability and quality is the lack of publicly available data for researchers and scientists in academia. Through this blog, our objective is to share our work in this relatively unexplored but a promising research direction [Kumar2016] [Kumar2017]. In [Kumar2016], we proposed source code level metrics to measure size, vocabulary, cognitive complexity and testing complexity of a visual Programmable Logic Controller (PLC) programming language. We apply Weykur’s property to validate the metrics and evaluate the number of properties satisfied by the proposed metric [Kumar2016]. In [Kumar2017], we study the correlation between the 10 ST source code metrics and their relationship with change proneness. We built predictive models using Artificial Neural Network (ANN) based techniques to predict change proneness of the software [Kumar2017]. Similarly, we are working towards examining whether source code metrics can be used to identify defective and faulty components.

Bringing Both Relevance and Rigor through Industry Academia Collaboration

Our experimental results are encouraging and provides evidences that there is a correlation between source code metrics and software quality and maintainability in the domain of industrial automation and PLC applications. However, more research studies from industry through industry and academia collaboration is needed to further add to the body of knowledge in this topic. It is hard for companies working in the industrial automation engineering domain to make their source code available in public domain for researchers in academia which creates a natural barrier for entry for academics to conduct research in this unexplored area. Industrial research labs does not have all the competencies, resources and time within a single organization to address all the research and technical challenges in this area and we believe that university and academia collaboration is the only possible solution to advance and disseminate knowledge on this topic. Our work is also a collaboration between industry and academia and gave positive results due to the successful partnership. One possible solution to build bridges is to conduct focused workshops on software engineering issues and challenges for industrial and factory automation. Some progress has been made in this direction which is encouraging and more needs to be done.

References

[Kumar2016] Lov Kumar, Raoul Jetley, and Ashish Sureka. 2016. Source code metrics for programmable logic controller (PLC) ladder diagram (LD) visual programming language. In Proceedings of the 7th International Workshop on Emerging Trends in Software Metrics (WETSoM '16). ACM, New York, NY, USA, 15-21. DOI: http://dx.doi.org/10.1145/2897695.2897699

[Kumar2017] Lov Kumar and Ashish Sureka. 2017. Using Structured Text Source Code Metrics and Artificial Neural Networks to Predict Change Proneness at Code Tab and Program Organization Level. In Proceedings of the 10th Innovations in Software Engineering Conference (ISEC '17). ACM, New York, NY, USA, 172-180. DOI: https://doi.org/10.1145/3021460.3021481

[Roos2008] Nieke Roos. 2008. Programming PLCs using Structured Text." International Multiconference on Computer Science and Information Technology (IMCSIT’08)

[IEC] International Electrotechnical Commission, International Standards and Conformity Assessment for all electrical, electronic and related technologies  http://www.iec.ch/

Understanding the Impressions, Motivations, and Barriers of One Time Code Contributors to FLOSS Projects: A Survey

By: Amanda S. Lee, University of Alabama. USA (@amandaslee15)
Jeffrey C. Carver, University of Alabama. USA (@JeffCarver32)
Amiangshu Bosu, Southern Illinois University. USA (@abosu)
Associate Editor: Bogdan Vasilescu, Carnegie Mellon University. USA (@b_vasilescu)

FLOSS, or Free/Libre Open Source Software, is becoming an increasingly important, some may even say dominant, factor in the modern software economy [2]. As opposed to the traditional methods of software development, FLOSS projects function and receive high-quality code submissions often despite the lack of financial compensation and the lack of any formalized management or governance structure [1, 2, 5]. While FLOSS projects are typically guided or managed by a small number of core developers, they survive and, indeed, thrive by attracting contributions from new, talented software developers who join the project. There are many types of contributors, ranging from those who have contributed a number of patches and are on their way to becoming part of the core, to those that lack adequate coding skills and contribute primarily by reporting bugs and editing documentation [3, 4, 6]. To be successful, FLOSS projects must constantly recruit new code contributors to replace those that leave (many do so within a year of joining) [7].

In our work [8], we define a special type of FLOSS contributor called “One-Time code Contributors” (OTCs) as those contributors who have successfully contributed, that is had merged by the project, one, and only one, code patch to a given FLOSS project. This successful contribution indicates that the OTC (1) has an appropriate level of coding skills and knowledge to make a valuable contribution to the project and (2) has the determination necessary to write the code, submit the patch, and participate in the review project to get the patch accepted. Because FLOSS projects could greatly benefit by attracting these technically competent participants to submit more code, two questions arise: (1) why do OTCs not contribute additional patches? and (2) is there any way the FLOSS projects can attract and retain these valuable contributors?

To answer these questions, and understand how OTCs could better contribute to FLOSS projects in the future, we conducted a survey of 184 OTCs from 23 popular FLOSS projects. The remainder of this post summarizes some of our key findings.

Initial Impression of Project Members

When asked about their initial impression of other project members, a large percentage of the respondents indicated they had a positive or very positive impressions of their fellow project members (Figure 1). This result is surprising because previous research indicated that peripheral developers may be neglected, so we expected them to have more negative impressions [6]. Some of the most common positive responses included: project members are skilled, helpful, or responsive. These responses indicate that OTCs appreciate the assistance that other project members can provide. However, some OTCs did report negative impressions of fellow project members. These negative impressions were just as strong as the positive impressions. The most common negative impressions included: project members are busy, unresponsive, or otherwise unhelpful.

Figure 1 - Overall Impressions

Tradeoff Between Skill and Busyness  

There was an interesting interaction between the most common positive impression, skilled, and the most common negative impression, busy. OTCs expected those project members who were more skilled to be less approachable and less obtainable, in other words, busier. Conversely, they expected less skilled project members to be less busy and more approachable. OTCs did not seem displeased with this trade-off—in fact, they seemed to expect it. Even so, the OTCs whose experience was that skilled members were not too busy to pay attention to them reported more positive impressions, while others who experienced the busier skilled members expressed more neutral or negative impressions.

OTCs’ Motivations for Contributing

Previous research found that peripheral contributors (of which OTCs are a subgroup) tend to be motivated more extrinsically (that is by external factors like fixing a bug) than intrinsically (that is, by internal factors such as enjoying coding as a hobby) [4, 7]. This observation would suggest that a large portion of the OTC contributions would be in the form of “drive-by commits,” that is, fixing a flaw without any real desire to join the project. When asked about their motivation to contribute their patch, the respondents listed a variety of motivations, as Figure 2 illuminates. While respondents indicated the desire to fix a bug as the most common motivation, they gave a more intrinsic motivation, share with the community, as the second most common motivation. Third, respondents indicated they were motivated by an employer’s need. These respondents were not hired to contribute to the FLOSS, but rather added or fixed the FLOSS project in support of other employer goals. Fourth, another intrinsic motivation, respondents wanted to add a new feature and have it maintained by the project. Perhaps unsurprisingly, this motivation often overlapped with the intrinsic motivation, share with the community. Other, less common motivations include ‘scratching an itch,’ personal reputation, or curiosity about the project. This list of varied motivations suggest that while, some OTCs may not be interested in long-term project participation, others have deeper motivations and could ultimately be attracted to join the project and contribute more than a single patch.

Figure 2 - OTC’s Motivations

Barriers Faced by OTCs

When asked whether there were any barriers that prevented them from continuing to contribute to the project, only half of the respondents indicated that the faced barriers, as seen in figure 3. The most commonly reported barrier was time. Either it took the respondent too long to make the contribution, or the respondent was too busy with other work. The next most common barriers were patch submission difficulties and entry difficulties. While FLOSS projects and tools oriented to help project newcomers cannot address the time barrier, they can encourage OTCs to continue contributing by reducing the difficulties associated with project entry and with patch submission.

Figure 3 - OTC Barriers

OTCs Who Stopped Contributing Despite Facing Barriers

Of the respondents who did not face barriers, nothing else to contribute was the most common reason to stop contributing.. This response is interesting because it suggests that if these OTCs did find something else to contribute, they might return to the project. In other words, these OTCs could be motivated to continue contributing to the project, because they exhibited no particularly strong desire to leave it. Other reasons for leaving a project included their employer no longer used the project and they never had any intention of becoming a project member. These results suggest that while not all OTCs can be motivated to make additional contributions, some very likely could be, under the right circumstances.

Conclusion

Though OTCs have only successfully contributed one code patch, FLOSS projects may be able to attract some of them to make further contributions. While some OTCs truly are “drive by committers,” whose only interest in the project was to have their patch included, others are more community-minded and more invested in the project. This second group of OTCs may have contributed additional code patches had they not encountered barriers or had they identified another interesting patch to contribute. By lowering the entry barriers and making patch submission easier, FLOSS projects can likely retain more OTCs, thereby increasing the size and skill of the contributor base, which leads to more successful FLOSS projects.

References

1. Lakhani KR, Wolf RG (2005). “Why Hackers Do What They Do: Understanding Motivation and Effort in Free/Open Source Software Projects."  In J Feller, B Fitzgerald, S Hissam, KR Lakhani (eds.),Perspectives on Free and Open Source Software, MIT Press, Cambridge.
2. M. Milinkovich. Keynote, Topic: "Open Collaboration, the Eclipse Way." International Conference on Software Engineering, Buenos Aires, Argentina, May 24 2017.
3. G. Pinto, I. Steinmacher and M. A. Gerosa, "More Common Than You Think: An In-depth Study of Casual Contributors," 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Suita, 2016, pp. 112-123.
4. Setia R. et al., “How Peripheral Developers Contribute to Open-Source Software Development,” Information Systems Research, vol. 23 issue 1 pp. 144-163, 2012.
5. Shah, S. K., “Motivation, Governance, and the Viability of Hybrid Forms in Open Source Software Development,” Management Science, vol. 52 issue 7 pp.1000-1014, 2006.
6. Steinmacher, I. et al., “Social Barriers Faced by Newcomers Placing their First Contribution in Open Source Software Projects.” 18th ACM Conference on Computer-Supported Cooperative Work and Social Computing, Vancouver, BC, Canada, 2015, pp. 1381-1392.
7. Zhou, M. and Mockus, A., “What Make Long Term Contributors: Willingness and Opportunity in OSS Community.” 34th International Conference on Software Engineering (ICSE), Zurich, 2012, pp. 519-528.
8. Lee, A., Carver, J. C. and Bosu, A., "Understanding the Impressions, Motivations, and Barriers of One Time Code Contributors to FLOSS Projects: A Survey." 39th International Conference on Software Engineering (ICSE), 2017, pp. 187-197.

IEEE July/August Issue, Blog, and SE Radio Summary

Associate Editor: Brittany Johnson (@drbrittjaydlf)

The July/August Issue of IEEE Software again provides articles that cover a range of software engineering-related topics. The themes in this issue include reliability and requirements engineering, technical debt, and agile development.

Special in this issue is a look at the history of IEEE Software in the article "Insights from the Past: The IEEE Software History Experiment" by Zeljko Obrenovic. By looking at the history website, this article proposes the practical value in using historical data. There is also discussion of the future of IEEE Software based on historical data.

The focus topic in this issue of IEEE Software was Reliability Engineering. This issue featured the following articles on reliability engineering:

• "Software Reliability Redux" by Diomidis Spinellis
• "Automated System-Level Regression Test Prioritization in a Nutshell" by Per Erik Strandberg, Wasif Afzal, Thomas J. Ostrand, Elaine J. Weyuker, and Daniel Sundmark
• "Safety Analysis of Safety-Critical Systems Using State-Space Models" by Vinay Kumar, Lalit Kumar Singh, Anil Kumar Tripathi, and Pooja Singh
• "Requirements Engineering for Safety-Critical Systems: Overview and Challenges" by Luiz Eduardo Martins and Tony Gorschek

One of the main concerns around reliability engineering is the growing ubiquity of software in safety critical systems. As pointed out by Diomidis Spinellis in "Software Reliability Redux", the more software is integrated into our daily lives the more we need adequate reliability engineering. Going more in depth on the topic, in "Requirements Engineering for Safety-Critical Systems: Overview and Challenges",  Martins and colleagues talk about some of the challenges that comes with engineering safety critical systems. In particular, they focus on dealing with safety requirements, discussing topics from reducing the gap between academia and industry and communicating requirements throughout the development process.

In "Safety Analysis of Safety-Critical Systems Using State-Space Models", Kumar and colleagues made one suggestion for dealing with safety analysis of safety critical systems; using state-space models for safety issue prognosis. They created an approach that takes UML state diagrams and maps them to state-space models, using Petri nets for dynamic behavioral analysis. They were able to validate their approach using a nuclear power plant's emergency core cooling system. Despite their findings, there are still strides that can be made to improve the state-of-the-art in regards to processes and tools in place for building and maintaining reliable safety critical systems.

IEEE Software Blog

The IEEE Software Blogs posts in July and August focused on vulnerabilities. Mehdi Mirakhorli wrote a blog post on tactical vulnerabilities, or vulnerabilities that result from incorrect implementation or deterioration of security tactics during coding and maintenance. Cor-Paul Bezemer and Zhen Ming (Jack) Jiang explored if and how developers use performance testing to identify vulnerabilities. They studied Java-based open source projects on GitHub and found that there may be missing tool support for effective performance testing. Maleknaz Nayebi and Federica Sarro used crowdsourcing in a case study of the Fort McMurray wildfire, along with their method called MAPFEAT, to determine useful mobile app features.

SE Radio

SE Radio welcomed two new hosts in July and August: Kishore Bhatia of BlockApps and Bryan Reinero of MongoDB.  For Kishore's first broadcast, he spoke with Kieren James-Lubin about security topics such as Blockchains, Crytocurrency, Bitcoins, and Distributed Ledger. For Bryan's first broadcast, he and Jason Hand discuss handling outages and responding to program failures. 

Other topics discussed on SE Radio in July and August include type driven development, rules engines, and P vs. NP.

IEEE May/June Issue, Blog, and SE Radio Summary

Associate Editor: Brittany Johnson (@drbrittjaydlf)

The May/June issue of IEEE Software provided a wide range articles on software-related issues; from energy-aware systems to deep learning, there's something for just about everyone. Although the focus of this issue was automotive software, there are features on various other software topics.

The focus topic in this issue of IEEE Software was Automotive Software. This issue featured the following articles on Automotive Software.

• "Future Automotive Architecture and the Impact of IT Trends" by Matthias Traub, Alexander Maier, and Kai L. Barbehön
• "Supporting the Management of Reusable Automotive Software" by Xabier Larrucea, Alastair Walker, and Ricardo Colomo-Palacios
• "Secure Automotive Software: The Next Steps" by Lee Pike, Jamey Sharp, Mark Tullsen, Patrick C. Hickey, and James Bielman
• "Deep Learning in Automotive Software" by Fabio Falcini, Guiseppe Lami, Alessandra Mitidieri Costanza

As suggested by the Editor, Diomidis Spinellis, in his article "How Abundance Changes Software Engineering", increase in software processing power has lead to changes in how we use computing technologies. This includes the relatively recent innovation of automotive software. In "Future Automotive Architecture and the Impact of IT Trends", Traub and colleagues discuss the opportunities that advances in IT and consumer-electronics afford the automotive industry. Much of the discussion focuses on the importance of seamless and scalable architecture for automotive software.

With the creation of things like Google's self driving car, and the potential for this to be a commercially available product, the importance of security in the software behind our vehicles is increasing. The authors of "Secure Automotive Software: The Next Steps" discuss this issue in depth, starting with the challenges faced by developers that work on automotive software.  Based on these challenges, the authors discuss some recommendations they have for improving the security of software used in the auto industry. Some of their recommendations include using static analysis for compile-time assurance and cryptography for runtime protection.


This issue also included articles on the feature topics of mobile app development, agile development, continuous deployment, and the business of software engineering  encompassed in the following articles:

• "Is It Worth Responding to Reviews? Studying the Top Free Apps in Google Play" by Stuart Mcllroy, Weiyi Shang, Nasir Ali, and Ahmed E. Hassan [Mobile-App Development]
• "Benefit Points: The Best Part of the StoryBenefit Points: The Best Part of the Story" by Jo Erskine Hannay, Hans Christian Benestad, and Kjetil Strand [Agile Development]
• "The Top 10 Adages in Continuous Deployment" by Chris Parnin, Eric Helms, Chris Atlee, Harley Boughton, Mark Ghattas, Andy Glover, James Holman, John Micco, Brendan Murphy, Tony Savor, Michael Stumm, Shari Whitaker, and Laurie Williams [Continuous Deployment]
• "What Differentiates Chilean Niche Software Companies: Business Knowledge and Reputation" by Sergio F. Ochoa, Romain Robbes, Maira Marques, Luis Silvestre, and Alcides Quispe [The Business of Software Engineering] 

IEEE Software Blog

The topics in the blog posts for May and June show a little more diversity than last month's...figuratively and literally. Anna Filippova of Carnegie Mellon contributed a blog post on how brainstorming can be used to support inclusiveness in diverse teams. Other topics discussed in these blog posts include cross-stack configuration errors, library adoption, and the effect of casual contributions on software quality.

SE Radio 

In this issue, SE Radio welcomed another new host, Matthew Farwell of Nexthink. In his debut, he spoke with Yakov Fain about Angular, including who should use it and why. Many of the episodes in this issue center around technologies and how developers are using them, including Elasticsearch, Docker, and LLVM.  SE Radio also deployed a listener survey to get feedback from their listeners -- don't forget to provide your feedback!