In the last few days, I have been in an email exchange with some other PhD students about literature review.  The conversation made me realise how much my understandings of what this is have changed over the course of my PhD journey as I have drawn on both my tradition of systems thinking and literature about literature review.

As I often do when I start to write a blog, I looked back to see what I have written before on the topic.  To my suprise, what I am thinking now was clearly signalled in a few blogs written quite a while ago:

• Discovering a landscape of research practice (1 February 2014) – in which I use ideas from Morgan (2007) to move away from typologies of research paradigms to think of paradigms as shared understandings and practices amongst groups of researchers.
• Can systematic reviews be approached systemically (28 November 2014) – in which I grapple with the hegemony of Cochrane-style systematic reviews and draw on Checkland’s distinction between systematic and systemic to consider two different orientations to literature review
• Doh – it’s another landscape (7 December 2014) – in which I connect up the above two posts and start to understand different forms of literature review in terms of shared understandings and practices amongst groups of researchers
• Design turn and the systematic review protocol (12 December 2014) – where I realise that the task of planning or creating a protocol for a literature review can be thought of as designing a learning system (rather than creating a rigid plan).

The email exchange with my peers was to do with the extent to which different disciplines (a) require a structured (auditable/replicable) approach to a literature review, and (b) require the methods of the review to be written up (made transparent).  Rather oddly considering where I started I found myself towards the more formalised/more transparent end of the spectrum…..perhaps I have taken more on board from the understandings and practices of health researchers than I thought I had!

But the more I think about it, this stance is as much to do with the idea of taking purposeful action and that clarity can be gained from considering what you are going and why – and therefore the best how.  If I am to chart my course through a set of literature do I really have an idea of what my purpose is and the best means to achieve that purpose.

Part of the problem I have grapped with is the way in which literature review is talked about as both a means and an end.  So we end up with some sort of circular and unhelpful purposeful action

To produce a literature review

By means of literature review

In order to have done a literature review

This can then get adapted to all sorts of different forms of literature review

To produce a scoping review by means of a scoping review in order to have done a scoping review

To produce a rapid review by means of rapid review in order to have done a rapid review

and so on

But if I think of literature review as a purposeful activity, I start to realise that it can have different purposes at different points.  So if ‘by means of literature review’ is the how – what is the what?

Here’s a general brainstorm:

• to engage with debates in an academic field
• to develop an understanding of the concepts used in an academic field
• to draw a conclusion on the effectiveness of intervention X for condition Y
• to synthesise what is known about a phenomenom
• to get an answer/gain insights into a research question
• to develop a conceptual framework for my research
• to identify gaps in a field of research
• to identify common research approaches (and associated challenges) when studying X

As with any definition of a purposeful activity system, these general ‘snappy’ definitions can be unfolded further using systems tools such as PQR, CATWOE, root definitions or Critical Systems Heuristics.  This helped me understand where I was inadvertently pursuing more than one purpose through one activity I called literature review.  In some cases, I was comfortable with juggling and continuing to pursue multiple purposes.  In other cases, I chose to be more focused on the purpose that had more priority.  And of course, I kept my purposes under review – it can change over time, rather than tie me down.

A benefit I have drawn from studying literature on literature review (my system to learn about literature review by means of reviewing literature on literature review in order to enrich my literature review practice!) is being able to consider the ‘how’ of literature review itself.  This zooms in to a lower level.  If ‘literature review’ is the what, what is the how?

There are a range of lower-level activities logically implied by the general activity of literature review, such as:

• searching for publications
• making decisions/judgements about what is worth pursuing further
• reading publications
• writing the literature review

But beneath each of these activities (activities NOT steps) are many, many questions and considerations.

Literature search

How will I search for literature?  My casual practice is head for google (or google scholar if I am really pushing it), type in a phrase and browse results until I get bored or time runs out.

But once I started taking on the rigour of academic research practice, I realised how random this is.  It is also potentially wasteful because a few weeks later I may not remember I did that and end up doing it again.  So, I do need to think through what I want to search for and how best to find it including the strengths, limitations and implications of the choices I make for my learning.

Choices include

• whether to use generic internet search engines or managed databases or a combination
• whether to use keyword searches (dominant method), snowballing (ancestry and citation searching), handsearching, author searching and/or a combination, rather than to berry pick. (There are lots of articles on pros and cons of different search methods to help weigh up these options)
• whether to include cut-off dates or to add in other limiters – and the reasons behind these (which links to inclusion/exclusion criteria below)

And some associated management issues

• whether to record every step and judgements so that later you can write up the process
• where and how to manage the records

Making decisions/judgements about what is worth pursuing further

Faced with a long list of titles or abstracts, how will I filter them to decide which ones to actually read.  Is it right/ethical to just select the ones that grab my attention and fit with my argument? (cherry pick).

Whilst some advocate the strict creation of inclusion/exclusion criteria prior to the start of the work, others expect them to evolve a little more organically (depends of course on the purpose of the particular literature review).  I have realised that actually I have always had criteria – the important thing is that I surface what these area and in making them explicit, I can then think about whether they are reasonable or not.  Making them explicit isn’t always possible at the outset, I may have a general sense of ‘oh this looks useful’, but overtime I start to understand why some things do look useful and others don’t.

Reading publications

Reading really isn’t a sufficient enough term for the amount of engagement that is needed here.

According to the purpose of the particular literature review, I will focus on different things here – am I interested in the framing of the paper? am I interested in the concepts used and what they mean? am I interested in the research methods? am I interested in the findings/conclusions?

One of the things that is going on here is ‘quality control’.  This can be conscious and explicit or much more implicit.  Knowing whether a publication has been through peer review or not is an important aspect of quality control.  There are also various standards around on how to quality assess studies which is important if I want to rely on the results.  Equally, there are some helpful ways of critically analysing the argument of a publication to see whether it all stacks up.

It isn’t enough to simply read the publications, I have to make sure I record or mark up for later the parts I want to look at again.  This could be called ‘analysis’ or ‘extraction’. It could be done with a highligher pen (or electronic equivalent) – but there are other methods with various degrees of formality that can be used (according to my purpose).  At the most formal end of the spectrum is an ‘extraction’ form – basically a list of all the things I want to get from every publication I draw on in your review.  I’ve used spray diagrams, coding and indexing in CAQDAS, note taking and so on.  The important thing for me was to start looking for themes – lessons that can be learned across the set of publications rather than treating each one individually.  This comes over time, through iteration, rather than in a straight forward way (for me, anyway!)

Writing the literature review

As with reading, writing is probably too simple a word.  Writing exposes what I do and don’t understand, what I need to re-look at, how I understand my themes.

Most importantly I am writing for a purpose and that purpose could be different for different contexts.  For example, in a conventional research article/thesis the purpose of the literature review chapter is to give an overview of the field and its gaps to derive research questions.  In studies that are literature reviews, the review gives the answer to a question (“what is known about…”).

The purpose should also be considered in decisions about whether or not to write about the method of the review itself.  If I am leading up to a claim that “noone has researched X using Y before”, then I really need to provide evidence of how much searching I did to meet that claim.  If the answer is “I googled once, five years ago until I was bored” – then the credibility isn’t there.  Hence the importance of the purpose in designing the whole inquiry in the first place.

 

I feel this is a bit of a ramble, touching on lots of things and leaving nothing tangible.  Adapting the words of Ison (2017), I’d say that there is no prescription for a literature review, they have to be designed.  The design process requires consideration of purpose.  And, it is also helped by drawing on ideas and methods discussed and debated by those engaged in literature review.  These ideas and methods don’t have to tie you down, they expand the potential for active choices, rather than defaulting to what you know.

And now, more than five years on I have finally realised the value of studying literature review and how I can use those ideas to enhance my research practice.  They help me to know what I do when I do what I do.

Republish