Critical appraisal and gender or ethnicity bias

I do a session with students on a Fundamentals of applied health research module which includes a brief consideration of bias in studies.   I inherited the session from a colleague and was familiar already with only some of the topics.   I have gained confidence in teaching the others and this year felt the section on bias was beginning to make sense.   That bit is certainly informed by work I am involved in about decolonisation and diversification.   So this year I found myself wondering if the "standard" critical appraisal checklists ask if the study being appraised shows any bias in relation, for example, to gender or ethnicity.   The "standard" checklists from CASP and SIGN don't, and neither does the Cochrane Risk of Bias tool.

The CASP and SIGN lists ask if the control and intervention groups are matched, but of course, both groups could be biased.

They ask, to be fair, if the study can be applied to your setting, so that leaves room to think about this sort of bias.

I remember in a teaching session ages ago thinking about this, and asking, for example, if the study you are reading has been conducted in Sweden, is the study applicable to Leicester.

I ought to have asked if the study population is like the population I work in, and for example, if the study population is mostly white, can the results be applied to a patient from a BAME group.   Or, if the study is mostly about men, can it be applied to my female patient.   

I used to lead a critical appraisal workshop with PhD students.  They were from a wide range of disciplines, clinical, laboratory, social sciences.    I talked about critical appraisal, then they appraised a paper they had brought with them.   Because I had found it impossible to find a paper that might be relevant to everyone, I used a paper from the Annals of Improbable Research (while worrying a bit about cultural aspects of humour, and whether everyone would find it amusing).    

Because of their wide range of backgrounds and choice of paper, not everyone could use the CASP lists and neither could I.   So I made my own, designed to work with any sort of academic paper.

I have added a question to that list to ask about gender or ethnicity bias.   The wording was amended after advice from my Leicester Learning Institute colleague Tracy Dix and some of her contacts.   My thanks to her, and them.

So to the section on applying the paper to your work, there is a new question:

13. Is the sample population representative of the wider population to whom the results may be generalised? For example, in human studies, does the sample represent a range of demographic characteristics, such as age, ethnicity, socio-economic status, gender and sexuality? (This list is not exhaustive, so consider other characteristics which may be relevant to your study).

The list is on Google Drive.  It has a Creative Commons BY-NC-SA 4.0 licence, so if you can use it in your practice or teaching (non commercial, of course), please do.