Overall, agreement on continuous respiratory rate between five reviewers was good, with an average respiratory rate of 9.19 beats per minute between the rate counted by the highest and lowest reviewer. There was substantial agreement on the breathing status between the reviewers, corresponding to all five reviewers agreeing on 70% of the videos. Agreement measures differed somewhat between age groups and were generally lower in younger age groups.
The findings indicate that especially distortions observed in the videos, for example, distortions caused by restless and crying children, impact the inter-rater reliability of the tool. This was confirmed by qualitative assessments of reviewers' experiences. It was also noted that visual reference standards have inherent limitations due to human subjectivity, as humans do not always mark breaths in the same way, particularly when there is distortion.
However, annotators perceived the respiratory rate to be more accurate with the annotation software than using manual counting, as it allowed them to distinguish between normal breaths, uncertain breaths, distortion and movement. They also mentioned that tool functionalities, like slowing down, changing color and adjusting brightness helped them in distinguishing between these different kinds of breaths and movements.