Transitioning towards continuous experimentation in a large software product and service development organisation – A case study

Sezin Gizem Yaman*, Fabian Fagerholm, Myriam Munezero, Jürgen Münch, Mika Aaltola, Christina Palmu, Tomi Männistö

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

4 Citations (Scopus)


Context: Companies need capabilities to evaluate the customer value of software-intensive products and services. One way of systematically acquiring data on customer value is running continuous experiments as part of the overall development process. Objective: This paper investigates the first steps of transitioning towards continuous experimentation in a large company, including the challenges faced. Method: We conduct a single-case study using participant observation, interviews, and qualitative analysis of the collected data. Results: Results show that continuous experimentation was well received by the practitioners and practising experimentation helped them to enhance understanding of their product value and user needs. Although the complexities of a large multi-stakeholder business-to-business (B2B) environment presented several challenges such as inaccessible users, it was possible to address impediments and integrate an experiment in an ongoing development project. Conclusion: Developing the capability for continuous experimentation in large organisations is a learning process which can be supported by a systematic introduction approach with the guidance of experts. We gained experience by introducing the approach on a small scale in a large organisation, and one of the major steps for future work is to understand how this can be scaled up to the whole development organisation.

Original languageEnglish
Title of host publicationProduct-Focused Software Process Improvement - 17th International Conference, PROFES 2016, Proceedings
EditorsSousuke Amasaki, Tommi Mikkonen, Michael Felderer, Pekka Abrahamsson, Anh Nguyen Duc, Andreas Jedlitschka
Number of pages16
Publication statusPublished - 6 Nov 2016
MoE publication typeA4 Article in a conference publication
EventInternational Conference on Product-Focused Software Process Improvement - Trondheim, Norway
Duration: 22 Nov 201624 Nov 2016
Conference number: 17

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10027 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceInternational Conference on Product-Focused Software Process Improvement
Abbreviated titlePROFES


  • Agile software development
  • Case study
  • Continuous experimentation
  • Customer development
  • Customer involvement
  • Experiment-driven software development
  • Lean startup
  • Organisational transition
  • Product management

Fingerprint Dive into the research topics of 'Transitioning towards continuous experimentation in a large software product and service development organisation – A case study'. Together they form a unique fingerprint.

Cite this