CPL - Chalmers Publication Library
| Utbildning | Forskning | Styrkeområden | Om Chalmers | In English In English Ej inloggad.

Experimentation that matters: A multi-case study on the challenges with A/B testing

Helena Holmström Olsson ; Jan Bosch (Institutionen för data- och informationsteknik, Software Engineering (Chalmers)) ; Aleksander Fabijan
Lecture Notes in Business Information Processing:8th International Conference on Software Business, ICSOB 2017; Essen; Germany; 12 June 2017 through 13 June 2017 (18651348). Vol. 304 (2017), p. 179-185.
[Konferensbidrag, refereegranskat]

From having been exclusive for companies in the online domain, feature experiments are becoming increasingly important for software-intensive companies also in other domains. Today, companies run experiments, such as e.g. A/B tests, to optimize product performance and to learn about user behaviors, as well as to guide product development and innovation. However, although experimentation with customers has become an effective mechanism to improve products and increase revenue, companies struggle with how to leverage the results of the experiments they run. In this paper, we study the reasons for this and we identify three key challenges that make feature experimentation a difficult task. Our research reveals the following challenges: (1) the impact of experiments doesn’t scale, (2) business KPIs and team level metrics are not aligned and (3) it is unclear if the available solutions are applicable across domains.

Nyckelord: A/B testing; Data-driven development; Feature experimentation



Denna post skapades 2017-12-15.
CPL Pubid: 253814

 

Läs direkt!


Länk till annan sajt (kan kräva inloggning)