Table 2: Rating system that individuals in journal teams used to assign their own self-assessment scores (SA-score)
In order to synthesise and describe processes that define best practice in peer review and to define potential obstacles encountered by journal teams, five raters (TG, CG, ECM, EM, MW) also rated the answers to each question using the qualitative feedback provided by the journal teams. The journal responses were anonymised with a journal number assigned to each journal. The questions were then divided between the raters according to the five Essential Areas, with one rater assessing and rating every journal response within a specific Essential Area. The raters followed a consistent rating system (Table 3) devised after the questions and after the self-assessments had been completed. The raters scored the responses from one to three, and in addition rated a non-applicable answer as ‘N/A’ and an unaccountable answer as ‘U’. The raters’ scores are referred to as Rated scores (R-scores).