Critical Appraisal of a Research Study

What are the key concepts of critical appraisal? This blogpost features a very good series of critical appraisal training videos published by Cochrane Mental Health on Youtube (full credits can be found at the bottom of this blogpost).

Hierarchy of Evidence

Retrieved from https://www.sketchbubble.com/en/presentation-hierarchy-of-evidence.html on 18th February 2023

Critical Appraisal Introduction

In this first video, the key concepts of critical appraisal are introduced, and the learning objectives for the series are clearly defined.

Systematic Reviews and Meta Analysis

In this second video, we are introduced to the fundamentals of systematic reviews and the ways in which we can apply the critical appraisal concepts of validity, trustworthiness of results, and value and relevance using the CASP checklist. The full text version of the study by Hay et al (2019) mentioned in this video can be found here.

Randomised Controlled Trials

In this third video, we are introduced to the fundamentals of randomised controlled trials and the ways in which we can apply the critical appraisal concepts of validity, trustworthiness of results, and value and relevance using the CASP checklist. The full text version of the study by Sugg et al (2018) mentioned in this video can be found here.

Cohort Studies Critical Appraisal

In this fourth video, we are introduced to the fundamentals of cohort studies and the ways in which we can apply the critical appraisal concepts of validity, trustworthiness of results, and value and relevance using the CASP checklist. The full text version of the study by Gerhard et al (2015) can be found here.

Case Control Studies

In this fifth video, we are introduced to the fundamentals of case control studies and the ways in which we can apply the critical appraisal concepts of validity, trustworthiness of results, and value and relevance using the CASP checklist. The full text version of the study by Drucker et al (2018) can be found here.

Cross Sectional Studies

In this sixth video, we are introduced to the fundamentals of cross-sectional control studies and the ways in which we can apply the critical appraisal concepts of validity, trustworthiness of results, and value and relevance. The full text version of the study by Boden et al (2010) can be found here.

Diagnostic Studies Critical Appraisal

In this seventh and last video, we are introduced to the fundamentals of diagnostic studies and the ways in which we can apply the critical appraisal concepts of validity, trustworthiness of results, and value and relevance using the CASP checklist. The full text version of the study by Hollis et al (2018) can be found here.

Reference

The above embedded videos are part of a project which was developed to enhance research use and development across two NHS Trusts.

The project founding partners were:

  • Cochrane Common Mental Disorders
  • Northumberland, Tyne and Wear NHS Foundation Trust (NTW), UK
  • Tees, Esk and Wear Valleys NHS Foundation Trust (TEWV), UK

Delivery of the project was supported by the Centre for Reviews and Dissemination at the University of York.

Funding Acknowledgement: The production of the critical appraisal modules was jointly funded by:

  • Economic and Social Research Council (ESRC), UK – as part of the University of York ESRC Impact Acceleration Account (ES/M500574/1)
  • Northumberland, Tyne and Wear NHS Foundation Trust (NTW), UK
  • Tees, Esk and Wear Valleys NHS Foundation Trust (TEWV), UK
  • University of York, UK

Cochrane Review Group Funding Acknowledgement: The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Common Mental Disorders Group.

Disclaimer: the views and opinions expressed herein are those of the module authors and do not necessarily reflect those of the ESRC, NIHR, the National Health Service (NHS), the Department of Health and Social Care or the University of York.


Did you find the above nursing information useful? Follow us on Facebook and fill in your email address below to receive new blogposts in your inbox as soon as they’re published 🙂

Cross Sectional Study Critical Appraisal

A cross sectional study is observational in nature. It involves collection of information about a population at a particular point in time.

  • A descriptive cross sectional study would assess distribution and frequency eg. measuring the prevalence of cancer amongst a defined population
  • An analytical cross sectional study would examine the association between variables to identify determining factors related to health eg. examining the association between living a sedentary lifestyle and having hypertension

Hierarchy of evidence

Retrieved from https://www.sketchbubble.com/en/presentation-hierarchy-of-evidence.html on 18th February 2023

Advantages

  • affordable – a cross sectional study requires no follow-ups since only one set of data is analysed, making this a low-cost research method
  • efficient – a cross sectional study is ideal for studying exposures or conditions that are reasonably common, and which require only one-time assessment
  • no risks – this type of study requires no long-term considerations.
  • potential completeness – due to easily accessed key data points

Disadvantages

  • collecting data at one point in time leads to limited causation testing especially where exposure and/or outcome are expected to change over time
  • needs to be an adequate representation of the population being studied
  • requires a larger sample size for accuracy basis
  • bias may affect results if for example incomplete responders are related to a specific group
  • may result in an association, however such association may not be the reason for the association
  • unable to measure incidence

Critically Appraising a Cross Sectional Study

When critically appraising a cross sectional study you need to focus on the following:

  • Sampling
  • Non-response
  • Methods used for measuring variables of interest
  • Controlling for confounders in the analysis

Sampling

  • note sampling bias – population needs to be clearly identified since final results will be inferred onto the target population
  • consider choice of sampling frame – how was the sample selected from the actual population? Remember that when it comes to measuring prevalence, the actual population is of utmost importance. Thus, consider sampling procedure used eg. random vs convenience sampling, using inclusion or exclusion criteria etc
  • consider the procedure used for the selection of participants – was inclusion/exclusion criteria used? And was the sample taken at random or was it convenience sampling?
  • consider sampling size – ideally, previous studies performed within the same area should be sought so that the occurrence frequency within the sample reflects the occurrence within the target population
  • consider expected precision of results – rare occurrence and precise results require a bigger sample

Non-Response

  • respondents may differ from non-respondents – respondents are more likely to be interested in the subject being studied, which may lead to more adherence to suggestions/requirements. Thus, replacing non-respondents to increase the sample size may still not bypass the sampling bias resulting from no response
  • researchers are required to report the response rate as well as to compare the characteristics of both the respondents and non-respondents

Controlling Confounders

A confounding factor is a third variable in a study which examines a possible cause-and-effect connection. It is related to both the supposed cause and supposed effect of the study. At times it is difficult to separate the true effect of the independent variable from the effect of the confounding variable.

Whilst performing a research study, it is important that potential confounding variables are identified and a plan is drawn so that their impact is reduced.

Example of a Cross Sectional Study

Breast feeding and obesity: cross sectional study (full reference in the references section): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC28161/

Appraisal Tool for Cross Sectional Studies (AXIS)

Cross Sectional Study Appraisal Checklist

NOTE: To view blogpost featuring Cochrane videos on all types of studies please click here.

References

BMJ Open (2016). Appraisal Tool for Cross-Sectional Studies (AXIS). Retrieved from https://bmjopen.bmj.com/content/bmjopen/6/12/e011458/DC2/embed/inline-supplementary-material-2.pdf?download=true on 17th December 2022

von Kries, R., Koletzko, B., Sauerwald, T., von Mutius, E., Barnert, D., Grunert, V., & von Voss, H. (1999). Breast feeding and obesity: cross sectional study. BMJ (Clinical research ed.), 319(7203), 147–150. https://doi.org/10.1136/bmj.319.7203.147


Did you find the above nursing information useful? Follow us on Facebook and fill in your email address below to receive new blogposts in your inbox as soon as they’re published 🙂