Clinicians face mountains of medical literature, and no one can keep up. Much of it is trash. Doctors and other health care professionals must know how to evaluate medical literature to provide the best possible care to their patients; but few know how. Here is some help to evaluate articles:
Title and General
- Is the title appropriate?
- Are the authors appropriately identified?
- Does information on conflict of interest or other disclaimers appear in the article?
Abstract
- Does the abstract entice the reader to read the article?
- Does the abstract correctly summarize the study and its findings?
Objective
- Objective(s)
- Study type – observational (cross sectional, case control, cohort) or experimental (randomized controlled trial, vaccine trial)
- Are the objectives reasonable and worthwhile?
- Strengths and limitations of the objective and study type
- How could these affect the results or conclusions (biased, generalizable)?
Methods
- Study design –
- How conducted?
- Who conducted?
- Methods appropriate for objective?
- Strengths and limitations
- How could these affect the results or conclusions (internal and external validity)?
Study population
- Who are the study subjects?
- How selected?
- Who selected?
- Strengths and limitations –
- How could these affect the results or conclusions (biased, generalizable)?
Study maneuver
- What treatments/interventions planned?
- How administered?
- Who will provide the maneuvers (where and when)?
- Are these appropriate for the study objectives?
- Strengths and limitations of planned maneuver –
- How could these affect the results or conclusions (biased, generalizable)?
Study observations/measures and data to be collected
- What observations/data collection is planned (On whom, where and when; are they consistent)?
- How will they be made or collected (by what method, are they standardized, are observers blinded)?
- Who will make the observations and collect the data, where and when?
- Are these appropriate for the study objectives?
- What are the strengths and limitations of the planned observations (exclusion, masked or unmasked, sources of bias, likely to be reliable, valid)?
- How could these affect the results or conclusions (biased, generalizable)?
Analysis
- What is the planned data analysis (what methods will be used, how will the data be grouped, what tests will be used for each analysis planned, what will be considered statistically significant)? Are assumptions based on prior studies?
- How will the data be analyzed?
- Who will conduct the analysis?
- Is the planned analysis appropriate for the study objective and the type and level of data collected (parametric or non-parametric tests, one-sided vs. two-sided tests of significance, stratification when appropriate)?
- What are the strengths and limitations of the planned analysis (what is the power of the planned data collection/analysis)?
- How could these affect the results or conclusions (failure to detect a true difference)?
Results
- What results/observations are presented/not presented (are all the findings presented for all subjects)?
- How were the results obtained (what analysis was used, was the planned analysis completed)?
- How were the results which are presented/excluded determined and why (data insufficient, poor response, not significant)?
- Are the results appropriate for study objectives, planned observations and analysis? Are they correctly performed and interpreted, internally consistent and valid (arithmetic errors, what is considered statistically significant, appropriate comparisons)?
- What are the strengths and limitations of the analysis results (statistical vs. biologic/clinical significance, statistical power)?
- How could these affect the results or conclusions?
Discussion and Conclusions
- What are the conclusions?
- How or on what basis were they made (are they justified by the results and analysis, is that analysis sufficient to determine whether significant differences may be due to incomparability of groups or methodological considerations)?
- Are the above limitations under all other sections adequately addressed in this section?
- How do they affect the conclusions? Is this adequately assessed?
- What are the strengths and limitations of the conclusions?
- What recommendations do authors have for future study?
The volume of scientific literature has exploded in the past 50 years. Sadly, the quality of many studies is poor. Only by knowing how to evaluate articles in the medical literature will doctors protect themselves and their patients from being deceived.
