Contents
1. What is consumer peer review?
2. How can I volunteer for consumer peer review?
3. Consumer peer review in Editorial Manager
4. Consumer review feedback forms
5. Help and support
1. What is consumer peer review?
Thank you for your interest in being a consumer peer reviewer for Cochrane. Cochrane uses the word consumers to mean patients, carers and members of the public with an interest in healthcare.
Consumer peer review means giving feedback on Cochrane protocols and reviews before they are published.
- a protocol is a shorter document explaining how the authors plan to carry out the review
- a Cochrane Review looks at the results of studies for a treatment or intervention. The authors analyse the data and present the evidence for the effectiveness of the treatment. Cochrane Reviews all contain a short introduction explaining the results for members of the public. This is called the Plain Language Summary. Read Plain Language Summaries of our reviews.
Why is consumer peer review important?
Consumer feedback is important in making sure that Cochrane reviews are:
- easy to understand
- written in plain language
- helpful to patients and carers needing to make decisions about their own or their family's healthcare
Cochrane is committed to involving consumers in producing health evidence. We want to make sure our reviews address questions that are important to patients. This makes Cochrane Reviews both more relevant and trustworthy. Many of Cochrane's funders and partners also expect and require us to involve consumers in our work. They recognise that consumer involvement can improve health outcomes for all: consumers help to produce Cochrane evidence; our evidence informs healthcare policy and guidelines; policy and guidelines improve practice; better practice means better outcomes for patients.
To understand more about the importance of peer review for Cochrane, read our peer review policy and Statement of principles for consumer involvement in Cochrane (2017).
What will I be asked to do as a consumer peer reviewer?
You will complete a feedback form with questions that guide you through the different sections of the protocol or review. The forms are available online or as Word documents. You will have the chance to think carefully about the review and raise issues that you think are important.
Please explain any changes you would like the authors to make. For instance, you could suggest specific changes to wording to make the review easier to understand. Your comments should be respectful and objective.
You may not want to comment on all the sections of the review, but your views, as a consumer, on the plain language summary (for example whether it's language can be understood by other consumers), background, outcomes, implications for practice, implications for research, abstract and synopsis, will be particularly helpful.
What are the benefits of being a consumer peer reviewer?
When you give feedback on a Cochrane Review, you are making a difference. Consumer feedback improves the quality of Cochrane evidence. Your comments help us make sure our reviews are useful to patients, carers and the public.
Cochrane believes it is important to reward everyone who contributes to our work. Your contributions as a consumer reviewer will earn you Cochrane Membership. Track your contributions as a peer reviewer in your Cochrane Account.
Cochrane has no policy on payment for involvement and in general peer review is not rewarded financially.
Training and resources
- Evidence Essentials - follow this course if you are new to peer review and Cochrane evidence. The four modules introduce evidence-based medicine, clinical trials, and how to understand Cochrane evidence as a member of the public.
- Conducting an Intervention Review - follow this course to find out more about the methods and research used in Cochrane Reviews. As a consumer reviewer, you can access this course for free.
The content of Cochrane Reviews and protocols is explained in detail in the Cochrane Handbook for Systematic Reviews of Interventions.
Your experience
You may have become involved in Cochrane's work because of your lived experience of healthcare conditions. This is often underpinned by complex emotions. Whilst this is often positive, you may need support if a review topic is directly relevant to your health status or condition. There is support available for you.
2. How can I volunteer for consumer peer review?
Join Cochrane Engage
Cochrane Engage is our online volunteer hub. It connects Cochrane Review authors and editors with people who have the time and expertise to get involved in our work. Join and set up your profile on Cochrane Engage so you can search for tasks that match your interests and skills.
Cochrane’s Editorial Service
Cochrane Reviews are edited and published by Cochrane's Centralised Editorial Service. The network of Cochrane Review Groups is currently being wound up (at August 2022).
3. Consumer peer review in Editorial Manager
Cochrane uses a system called Editorial Manager to collect online feedback on draft protocols and reviews. These instructions will explain how you can:
- agree to an invitation to provide written feedback
- open a copy of the draft protocol or review
- provide feedback on an online form
- find Help
If you can, open the draft review on a computer or laptop – not a phone or tablet. This will make the text easier to read. We recommend using Chrome or Firefox as your browser.
If you need any help, or have questions, please contact the Cochrane Support Team.
Agree to an invitation to provide written feedback
You will receive an email from the Cochrane Review Group editing the draft review, inviting you to provide feedback. The email will include details of the draft review, and a suggested deadline for your comments. The email will also include a link to accept the invitation to review. Please accept your invitation within 7 days or you will be automatically uninvited. When you click on the Agree to Review link in your invitation email, you will be taken to the Cochrane Account login screen.
Enter your Cochrane Account username (your email address) and password. If you have any difficulty logging in, please contact the Cochrane Support Team.
If you do not wish to provide written feedback, or the deadline is not convenient, you can click the Decline to Review link in the email.
When you have logged in to Editorial Manager you will see a message thanking you for agreeing to review. Click on Pending Assignments.
What if I have already agreed to provide feedback?
If you have already been in contact with a Cochrane Review Group – for instance, agreed to review on Cochrane Engage – the email you receive will take you straight to the online form. Click on the Pending Assignments link in the email and log in on the Cochrane Account login screen. When you log in to Editorial Manager, you can go straight the step below to open a copy of the review.
Open a copy of the draft protocol or review
Open your Pending Assignments folder. Click View Submission in the left-hand menu to open a PDF of the draft review.
If you are offered a choice of PDFs, always click the link at the top of the list. You can open the PDF to read online, or download and save it.
Provide feedback on an online form
When you have read through the Plain Language Summary and any other sections of the review you are interested in, click Submit Recommendation in the left-hand menu to open the online comments form.
At the top of the form the dropdown Recommendation menu asks you for your overall opinion on the draft review. Please choose your recommendation based on the extent of changes you think the authors need to make before the review can be published.
- Accept – no further changes needed
- Minor Revision – some small changes needed
- Major Revision – substantial changes needed
- Reject – draft review is not suitable for publication
You may wish to complete the rest of the questions on the form before returning to give your overall recommendation.
Complete the form answering each question in turn. When giving your feedback, please focus on how useful the review will be to patients, carers and the public. You can write as much or as little as you like. Preview the questions you will answer. At the bottom of the form, there is space for you to add any general comments to the editors of the Cochrane Review Group who asked you for feedback.
If you need to save your answers before you have finished, click Save & Submit Later at the bottom of the form. When you have completed all the questions, click Proceed. If you see a warning that not all questions are complete, please go to the top of the page and make sure you have chosen your overall recommendation.
You will see a summary of all your answers to check over. You can click Back to make changes. If you are happy with your answers, click Submit Review to Editorial Office and then OK. You will see a message thank you for providing feedback.
If you would like to receive a copy of all the feedback on this review from other reviewers, or the authors’ responses to your comments, please contact the Cochrane Review Group publishing the review. You can do this by replying to your invitation email, or contact the Cochrane Support Team.
4. Consumer review feedback forms
Cochrane has different feedback forms for protocols and reviews, and for different types of review. The questions below are on all forms:
Conflicts of Interest
- Are you an employee (full-time or part-time) of an organization with a financial interest in the topic of the article?
- Do you own a commercial organization with a financial interest in the topic of the article?
- Do you have any other direct financial interests relevant to the topic of the article?
- Do you personally own a patent (or pending application for a patent) for an intervention, diagnostic test or prognostic marker that is relevant to the topic of the article?
- Do you have a relationship with any of the authors of the article?
- Are you an author of a study that might be included in the review?
Acknowledgement
- Do you agree to being acknowledged as a peer reviewer in the published article?
- Do you agree to being acknowledged as a peer reviewer in a list of peer reviewers on a Cochrane website?
- If yes, please include your name and any affiliation as you wish it to appear.
Example feedback form: Protocol: Intervention review (Consumer)
- Background: Does the background explain the topic clearly (i.e. are the healthcare need and intervention clear)? If not, which words or phrases are not clear, or how would you describe them? Does the background address the hopes and concerns of people considering the treatment? Is it clear “why it is important to do the review”?
- Objectives: Are the objectives of the Cochrane protocol clearly described? If not, which words or phrases are not clear, or can you suggest any improvements? Are the objectives relevant, and do you think they would help patients, carers and the public in making a healthcare decision? If not, please provide your reasons.
- Types of studies: Is it clear what types of studies (e.g. randomised controlled trials, observational studies) are to be used? Are the study designs appropriate? If not, please suggest the addition or deletion of any study designs.
- Types of participants: Do the proposed participants cover all relevant groups of people who might want to use this treatment? If not, who else would it be helpful to include or exclude?
- Types of interventions: Are the study interventions and comparisons/controls clearly described? Are the included interventions appropriate? If not, please explain.
- Types of outcome measures: Are the outcome measures (benefits and harms/side effects) the ones that are important to consumers, patients and the public? Can you highlight any other outcomes that are important to users of this review?
- Search methods for identification of studies: Do you have any comments on this section?
- Data collection and analysis: Do you have any comments on this section?
- Declarations of interest: Does the Cochrane protocol acknowledge possible interests of the authors (e.g. personal or financial) that could have influenced them?
- Language and style of writing: Is the Cochrane protocol reasonably easy to understand? Is the technical language used appropriately, and where possible, explained? If not, which sections need to be clearer and can you suggest any improvements? Is any language insensitive to consumers? Please suggest alternative phrases if possible.
- Words requiring further definition: Please list below any words that you think need further definition.
- Additional comments: Please add any other comments that you may have:
Example feedback form: Review: Intervention (Consumer)
- Title: Please note that the title has already been peer reviewed and agreed to in the published Cochrane protocol (available on the Cochrane Library). We therefore we do not expect you to comment on the title. If you can suggest an improvement, however, please do so here and the editors will consider it.
- Abstract: Is there anything mentioned in the Cochrane review that may be important, but is missing from the Abstract? Do you think that the Abstract overstates or understates what was found in the Cochrane review?
- Plain language summary title: Does the Plain language summary title reflect the title of the Cochrane review, and is it easy to understand? If not, can you identify which words or phrases are difficult to understand, or could you suggest any improvements to the wording?
- Plain language summary population: Is the health problem or issue being addressed stated clearly?
- Plain language summary interventions: Are the interventions and comparisons/controls examined in the Cochrane review stated clearly and succinctly in this section?
- Plain language summary findings: Does the Plain language summary report the main findings from the Cochrane review clearly and accurately? Does it report on adverse effects or harms?
- Plain language summary quality of evidence: Does the Plain language summary describe the overall quality of the evidence, and comment on any issues that could affect the findings of the review?
- Plain language summary consistency: Do you think the findings in the PLS are consistent with the Abstract and the rest of the Cochrane review? Is there anything mentioned in the review that may be important, but is missing from the PLS? Do you think that the PLS overstates or understates what was found in the review?
- Plain language summary usefulness: Do you think the Plain language summary would help patients, carers and the public in making a healthcare decision? If not, is there anything missing from the Plain language summary that you think should be included? Do you have any other suggestions for improvement?
- Writing style of the Plain language summary: Is the Plain language summary written in plain language and easy to understand? Are sentences too long or wordy? Are there any parts that you think should be rewritten?
- Terminology in the Plain language summary: Are abbreviations, research terms and technical terms avoided or explained? If any terms require clarification, please include them here.
- Background/Objectives/Criteria for considering studies for this review/Search methods for identification of studies/Data collection and analysis: Please note that these sections were published in the Cochrane protocol (available in the Cochrane Library) and have therefore been peer reviewed. If you would like to provide any comments, please do so here and the editors will consider them. Major suggestions for change are more likely to be considered for future updates of the Cochrane review than this version.
- Results: Can you understand the format of the results? Is it clear whether the intervention was effective or not?
- Quality of the evidence and risk of bias: Do the results include information about the overall quality of the evidence, and risk of bias?
- Study funding: Does the section ‘Included studies’ and the ‘Characteristics of included studies’ table include details about the funding sources for the studies?
- Discussion: Do the authors discuss harms as well as benefits? Are the effects of the treatments over- or understated?
- Author’s conclusions, implications for practice: Is this section clear and reasonably easy to understand?
- Author’s conclusions, implications for research: Do you think the authors have identified the important areas for future research? Are there any missing? Are there any benefits or harms important for healthcare users that are not addressed in the studies that you would like to see highlighted here?
- Declarations of interest: Does the Cochrane review acknowledge possible interests of the authors (e.g. personal or financial) that could have influenced them? If you don’t have any comments on declarations of interest, please leave blank.
- Summary of findings table: Do you think the outcomes included in the Summary of findings table are the most important outcomes to people with experience of the health condition? If not, please list them here.
- Language and style of writing: Is the Cochrane review reasonably easy to understand? Is the technical language used appropriately, and where possible, explained? If not, which sections need to be clearer and can you suggest any improvements? Is any language insensitive to consumers? Please suggest alternative phrases if possible.
- Words requiring further definition: Please list below any words that you think need further definition.
- Additional comments: Please add any other comments that you may have:
5. Help and support
We want to help and support people who provide consumer feedback for us.
If you have questions about the online forms, logging in, or Cochrane Membership - contact the Cochrane Support Team.
Cochrane's Consumer Engagement Officer, Richard Morley, offers consumers advice and support. The Cochrane Consumer Network Executive is drawn from the wider Cochrane Consumer Network membership and can offer support and advocate for consumers - contact the Consumer Network.
Acknowledgements
This guidance was developed by Richard Morley, Cochrane Consumer Engagement Officer, Ursula Gonthier, Membership and Support Manager, Ann Shackleton, Member Engagement and Data Specialist, and reviewed by Rachel Plachcinski, Consumer Network Executive member. Cartoons by Karen Morley, Cochrane consumer.