Sunday, April 2, 2017

Scaling up the software development process

by Mark Basham, Diamond Light Source (@basham_mark)
Associate Editor: Christoph Treude (@ctreude)

What should I talk about at the first RSE conference in the UK?

New conferences don't start every day, and so when a community which you have been a part of for a while decides to run one and asks for abstracts, it's important to take part.  The community I’m talking about is the UK’s RSE or Research Software Engineer community, and it has been gathering strength over the last few years.  Originally started by the SSI (Software Sustainability Institute, the initial aim of the group was to create a job title and career path for the oft underappreciated group of researchers who support their research group by writing and maintaining, often critical, software.  Given this audience the question is really what should I talk about?  I contribute to several open source software packages, which could always do with more publicity (,, but that felt like a shameless plug which would only interest a very minor group, and I wanted something which would appeal to a larger audience.

Project Management?

One more general area in which I have been working is project managing a large cross group software project at Diamond Light Source (  When I started working the project about 2 years ago, I had little experience of project managing such a complex project, although Diamond runs projects routinely.  I cannot understate that the amount I have learnt from my peers at Diamond whilst making the transition from developer to manager during this project is huge, and wasn't long before I was directed towards a selection of texts in the field.  The wealth of information available on the topic is absolutely astounding, and almost immediately influenced my management decisions, but coming from my background as a scientist, this wealth of knowledge had passed me by almost entirely.  It seemed like exactly the kind of thing on which to base a presentation.

The Books?

There are many good books about project management for software, but the ones which I discovered and found particularly relevant were the Mythical Man Month, Peopleware, and User Story Mapping.

The Mythical Man Month

This book is from the 70s but still has many relevant points for the modern developer.  The key take home for me was that if you want to write software well, as in tested, integrated and ready for others to use, it takes about 10 times longer than to just hack it up.


This is a great book, and I have enjoyed other books by the Atlantic Systems Guild, such as “Adrenaline Junkies to Template Zombies” and “Slack”.  The thing that resonated with me here was how damaging context switching was, and it is something I see regularly in our predictably overstretched team.

User Story Mapping

The last book I wanted to talk about was User Story Mapping, simply because getting requirements right is always difficult, and the book suggested a clear and concise method for doing this, as well as a nice discussion on the merits and pitfalls of user stories.

So what are you going to talk about?

OK, so three books do not make a presentation, but they do lay the foundations for a talk based around work we conducted at Diamond to try to get the aforementioned project back on track.  There was one deliverable which for many valid reasons was falling behind the rest of the project, and put the whole initiative in jeopardy.  Diamond took action. First we identified a team, and freed up their time to focus on the deliverable to stop them from having to multitask.  We moved them out of their usual open plan environment and into a new office for the duration of the intervention, increasing their workspace by around 70% and reducing off topic distractions.  Finally we trialled a new method for identifying requirements and planning the work based on the Story Mapping approach.  The cost of these interventions was high, but the result was significant, bringing the deliverable back on track with a generally high level of design, code quality and tests in place. This journey seemed like a perfect topic for a talk to a growing community which may meet similar challenges.

Final Title?

“Scaling up the software development process, a case study highlighting the complexities of large team software development” (  Personally I enjoyed preparing it, and the associated paper is available if you are interested in the results of the experiment.