Organizing Graphical User Interface tests from behavior‐driven development as videos to obtain stakeholders' feedback

Research output: Contribution to journalArticleResearchpeer review

View graph of relations

Details

Original languageEnglish
JournalJournal of Software: Evolution and Process
Publication statusE-pub ahead of print - Aug 2024

Abstract

Demonstrating software early and responding to feedback is crucial in agile development. However, it is difficult for stakeholders who are not on-site customers but end users, marketing people, or designers, and so forth to give feedback in an agile development environment. Successful graphical user interface (GUI) test executions can be documented and then demonstrated for feedback. In our new concept, GUI tests from behavior-driven development (BDD) are recorded, augmented, and demonstrated as videos. A GUI test is divided into several GUI unit tests, which are specified in Gherkin, a semi-structured natural language. For each GUI unit test, a video is generated during test execution. Test steps specified in Gherkin are traced and highlighted in the video. Stakeholders review these generated videos and provide feedback, for example, on misunderstandings of requirements or on inconsistencies. To evaluate the impact of videos in identifying inconsistencies, we asked 22 participants to identify inconsistencies between (1) given requirements in regular sentences and (2) demonstrated behaviors from videos with Gherkin specifications or from Gherkin specifications alone. Our results show that participants tend to identify more inconsistencies from demonstrated behaviors, which are not in accordance with given requirements. They tend to recognize inconsistencies more easily through videos than through Gherkin specifications alone. The types of inconsistency are threefold: The mentioned feature can be incorrectly implemented, not implemented, or an unspecified new feature. We use a fictitious example showing how this feedback helps a product owner and her team manage requirements. We conclude that GUI test videos can help stakeholders give feedback more effectively. By obtaining early feedback, inconsistencies can be resolved, thus contributing to higher stakeholder satisfaction.

Keywords

    BDD, GUI test, feedback, video

ASJC Scopus subject areas

Cite this

Organizing Graphical User Interface tests from behavior‐driven development as videos to obtain stakeholders' feedback. / Shi, Jianwei; Mönnich, Jonas; Klünder, Jil et al.
In: Journal of Software: Evolution and Process, 08.2024.

Research output: Contribution to journalArticleResearchpeer review

Shi J, Mönnich J, Klünder J, Schneider K. Organizing Graphical User Interface tests from behavior‐driven development as videos to obtain stakeholders' feedback. Journal of Software: Evolution and Process. 2024 Aug. Epub 2024 Aug. doi: 10.1002/smr.2721
Download
@article{4f6d4058a1d04aeb98297a2ca1c9248f,
title = "Organizing Graphical User Interface tests from behavior‐driven development as videos to obtain stakeholders' feedback",
abstract = "Demonstrating software early and responding to feedback is crucial in agile development. However, it is difficult for stakeholders who are not on-site customers but end users, marketing people, or designers, and so forth to give feedback in an agile development environment. Successful graphical user interface (GUI) test executions can be documented and then demonstrated for feedback. In our new concept, GUI tests from behavior-driven development (BDD) are recorded, augmented, and demonstrated as videos. A GUI test is divided into several GUI unit tests, which are specified in Gherkin, a semi-structured natural language. For each GUI unit test, a video is generated during test execution. Test steps specified in Gherkin are traced and highlighted in the video. Stakeholders review these generated videos and provide feedback, for example, on misunderstandings of requirements or on inconsistencies. To evaluate the impact of videos in identifying inconsistencies, we asked 22 participants to identify inconsistencies between (1) given requirements in regular sentences and (2) demonstrated behaviors from videos with Gherkin specifications or from Gherkin specifications alone. Our results show that participants tend to identify more inconsistencies from demonstrated behaviors, which are not in accordance with given requirements. They tend to recognize inconsistencies more easily through videos than through Gherkin specifications alone. The types of inconsistency are threefold: The mentioned feature can be incorrectly implemented, not implemented, or an unspecified new feature. We use a fictitious example showing how this feedback helps a product owner and her team manage requirements. We conclude that GUI test videos can help stakeholders give feedback more effectively. By obtaining early feedback, inconsistencies can be resolved, thus contributing to higher stakeholder satisfaction.",
keywords = "BDD, GUI test, feedback, video",
author = "Jianwei Shi and Jonas M{\"o}nnich and Jil Kl{\"u}nder and Kurt Schneider",
note = "Publisher Copyright: {\textcopyright} 2024 The Author(s). Journal of Software: Evolution and Process published by John Wiley & Sons Ltd.",
year = "2024",
month = aug,
doi = "10.1002/smr.2721",
language = "English",

}

Download

TY - JOUR

T1 - Organizing Graphical User Interface tests from behavior‐driven development as videos to obtain stakeholders' feedback

AU - Shi, Jianwei

AU - Mönnich, Jonas

AU - Klünder, Jil

AU - Schneider, Kurt

N1 - Publisher Copyright: © 2024 The Author(s). Journal of Software: Evolution and Process published by John Wiley & Sons Ltd.

PY - 2024/8

Y1 - 2024/8

N2 - Demonstrating software early and responding to feedback is crucial in agile development. However, it is difficult for stakeholders who are not on-site customers but end users, marketing people, or designers, and so forth to give feedback in an agile development environment. Successful graphical user interface (GUI) test executions can be documented and then demonstrated for feedback. In our new concept, GUI tests from behavior-driven development (BDD) are recorded, augmented, and demonstrated as videos. A GUI test is divided into several GUI unit tests, which are specified in Gherkin, a semi-structured natural language. For each GUI unit test, a video is generated during test execution. Test steps specified in Gherkin are traced and highlighted in the video. Stakeholders review these generated videos and provide feedback, for example, on misunderstandings of requirements or on inconsistencies. To evaluate the impact of videos in identifying inconsistencies, we asked 22 participants to identify inconsistencies between (1) given requirements in regular sentences and (2) demonstrated behaviors from videos with Gherkin specifications or from Gherkin specifications alone. Our results show that participants tend to identify more inconsistencies from demonstrated behaviors, which are not in accordance with given requirements. They tend to recognize inconsistencies more easily through videos than through Gherkin specifications alone. The types of inconsistency are threefold: The mentioned feature can be incorrectly implemented, not implemented, or an unspecified new feature. We use a fictitious example showing how this feedback helps a product owner and her team manage requirements. We conclude that GUI test videos can help stakeholders give feedback more effectively. By obtaining early feedback, inconsistencies can be resolved, thus contributing to higher stakeholder satisfaction.

AB - Demonstrating software early and responding to feedback is crucial in agile development. However, it is difficult for stakeholders who are not on-site customers but end users, marketing people, or designers, and so forth to give feedback in an agile development environment. Successful graphical user interface (GUI) test executions can be documented and then demonstrated for feedback. In our new concept, GUI tests from behavior-driven development (BDD) are recorded, augmented, and demonstrated as videos. A GUI test is divided into several GUI unit tests, which are specified in Gherkin, a semi-structured natural language. For each GUI unit test, a video is generated during test execution. Test steps specified in Gherkin are traced and highlighted in the video. Stakeholders review these generated videos and provide feedback, for example, on misunderstandings of requirements or on inconsistencies. To evaluate the impact of videos in identifying inconsistencies, we asked 22 participants to identify inconsistencies between (1) given requirements in regular sentences and (2) demonstrated behaviors from videos with Gherkin specifications or from Gherkin specifications alone. Our results show that participants tend to identify more inconsistencies from demonstrated behaviors, which are not in accordance with given requirements. They tend to recognize inconsistencies more easily through videos than through Gherkin specifications alone. The types of inconsistency are threefold: The mentioned feature can be incorrectly implemented, not implemented, or an unspecified new feature. We use a fictitious example showing how this feedback helps a product owner and her team manage requirements. We conclude that GUI test videos can help stakeholders give feedback more effectively. By obtaining early feedback, inconsistencies can be resolved, thus contributing to higher stakeholder satisfaction.

KW - BDD

KW - GUI test

KW - feedback

KW - video

UR - http://www.scopus.com/inward/record.url?scp=85200344261&partnerID=8YFLogxK

U2 - 10.1002/smr.2721

DO - 10.1002/smr.2721

M3 - Article

JO - Journal of Software: Evolution and Process

JF - Journal of Software: Evolution and Process

SN - 2047-7473

ER -

By the same author(s)