Now that my ‘talk samples’ are coming in and I have transcribed a couple and listened to a couple more, I feel it is time to decide how I am going to analyse them. 

In terms of actual ‘tools’ I have decided to use an on-line ‘software as a service’ analysis tool called Dedoose.  From the looks of the information this has been developed ‘by researchers, for researchers’ to help with qualitative and mixed methods research.  It seems to do – and make easier – exactly what I thought I would be doing with a highlighter pen and notes in margins.  It is also a lot, lot cheaper that the software I have seen mentioned in research reports I have read called NViVo – but it does kind of the same thing.

The video tutorials for Dedoose (which are available even if you don’t subscribe) talk about the need to set up both ‘descriptors’ and a ‘coding system’.

‘Descriptors’ are relatively easy – ‘characteristics’ of the people from whom the ‘talk samples’ came – and therefore ways in which I may want to ‘cut’ the data.  I’ve decided to note Board member/non-Board member and then constituent type (Council, NHS, VCS, University) – as these were the factors that underpinned my preferred sampling.  However, given that each talk sample is an exploratory case study, I am not planning on comparing and contrasting between different descriptors – it’ll just be a useful way of being able to show who the samples have come from.

‘Coding’ is more difficult.  The Dedoose materials describe it as the conceptual framework to organise, understand and communicate research findings….a looser way of thinking about it is a series of ‘tags’ that I can apply to highlighted bits of text (known as excerpts).  But before I leap into a frenzy of ‘tag’ creation, I just wanted to stand back and think about what I am about to do when I do it.

For me the key question is – should my coding system be ‘theory-informed’ or not?  This reminds me of a decision between:

• an etic approach – “using an imposed frame of reference”  or
• an emic approach – “working with the conceptual framework of those studied” (Taylor, 2001, 16 quoting from Silverman, 1993, 24)

There is also some more helpful pointers on this distinction in the Wikipedia article [accessed 4 Feb 2012].  In particular the helpful wikipedia phrase “an etic account is a description of a behavior or belief by an observer”.  I’ve already noted that what I am doing in my research is part of a social dynamic – my experience of what people ‘say’ will lead to me making claims or otherwise that that person has systems thinking capabilities.  I am using my frame of reference to do that – and I am then drawing up the description on my terms.  That makes my approach ‘etic’. (I would imagine an emic approach would require me to become much more familiar with the data before getting a ‘feel’ for the themes/issues arising and then developing a coding system, so I wouldn’t even be thinking of this yet)

But in order for research to be ‘credible’ my ‘imposed frame of reference’ can’t just be a random set of things that resonate with me.  The T847 materials note that theory can provide a framework for comparison purposes – act as points for consideration and possible ways to think about the data.  Again that seems to fit with my research approach and question.

I covered off a lot of the theory in my first assignment as a way of locating the project in relation to previously published work (partly reproduced in this blog), but I don’t think I was clear then in terms of how I was going to use it to actually DO my analysis.

So perhaps I need to look at it in terms of the two part research question, that I articulated in my second assignment.

To what extent do those involved in leading partnership working for wellbeing and health in the participating English city demonstrate:

• systems thinking capabilities

and

• an appreciation of practices that are consistent with those that enable rather than constrain systems thinking capabilities

in their talk?

So let’s take each of the parts of the question in turn.

I explored the issue of systems thinking capabilities in this post – a kind of mega-brain dump of different issues I would have to experience to make a claim that someone is thinking or acting systemically.  However, I don’t think that will translate very well into a coding system.

In order to do a small ‘triangulation of theories’ I am going to draw on two different authors for my coding system (acknowledging that those two authors are former colleagues in OU systems group so not that ‘independent’ of each other).

Firstly, a coding system relating to Ros Armson’s explanation that systems thinking is both holism and pluralism (I can reference this book properly at the moment, because a colleague has borrowed it – just hope I am remembering it right.).  This leads to two codes:

• for excerpts that show an appreciation of multiple interacting variables and interdependence (holism)
• for excerpts that show an appreciation of the value of multiple perspectives (pluralism)

These two factors have resonance with Reynolds and Holwell’s (2010, 6) discussion of the trap of reductionism and trap of dogmatism (again authors who are linked with OU systems group).

A couple of issues arising looking at these codes.  Armson does say that systems thinking is BOTH holism and pluralism i.e. not just one or the other.  So what do I do if a talk sample demonstrates lots of one of these and none of the other – this could purely be to do with the nature of the narrative the person is telling, rather than an ‘absent competence’ – one to remember when writing up.  At least Dedoose will help me to highlight which samples include both or just one.

The other issue is how diverse do the ‘multiple perspectives’ have to be – just perspectives of immediate colleagues, or different professionals or working with the community.  Or indeed it could be working on own but deliberately ‘taking different perspectives’.  I think I’ll wait to see what sort of things get coded with the headline code first – if there are many, many instances I may then have to put in a nuance of sub-codes.  So initially I will include all these ‘mentions’ – of valuing different viewpoints. (hey a little bit of ’emic’ coming in there – I’ll wait to see what the data says!).

Okay, the second ‘theory’ I want to draw on is Ison’s (2010, 24-25) explanation of his ‘criterion’ of what it is to be a systems thinker.  This is with the caveat of systems thinking arising from a social dynamic (as explained in this blog).  Drawing from Ison’s writing, I am going to have codes for:

• use of concepts and ideas linked to the discipline of Systems
• refer to use of systems approaches and tools
• explicitly connect with the history of Systems scholarship (e.g. mention a particular author)

This is a slightly ‘tighter’ set of codes – I imagine I will use them less, but we’ll see.

Moving on now to the second part of the research question – the one about practices that constrain/enable.  I think my original assumption around this has changed slightly since listening to the few recordings I have.  I am going to create codes for:

• positive/appreciative talk about practices that enable systems thinking
• negative/critical talk about practices that constrain systems thinking
• positive/appreciative talk about practices that constrain systems thinking – this is to ensure I seek and highlight negative instances (analytic induction)
• negative/critical talk about practices that enable systems thinking – again to ensure I seek and highlight negative instances (analytic induction)

For each of the above, I also think I need to distinguish the ‘level’ of practice being referred to:

• personal level – examples of enabling may be reflective practice or emotion of inquiry
• interpersonal (micro) level – emphasis on relationships and working with others
• partnership (meso) level – ways of organising and ‘doing’ partnership working
• national (macro) level – national policy content and ways of implementing policy
• global – more general discourse

I draw these out because they seem to be the different levels that came out in my literature review – even though I did not frame that in that way at the time.  I need to feed back these newer ideas into structuring my Background section when I do my final project.

One final thing to think through.  So far I have used ‘systems thinking’ in a way that for me acknowledges both the systemic and the systematic, drawing on Ison’s duality (2010, 191).  Ison says “my ideal, aware, systems practitioner is one who is able to distinguish between systemic and systematic thinking and is able to embody these distinctions in practice” (page 193).  His book includes two tables (Table 8.1 on page 192 and Table 7.1 on page 158) that show comparisons of the characteristics of systemic and systematic.  My question is – do I acknowledge the systematic?  Because I am using samples generated through the appreciative inquiry Discovery interviews it may just be that the person chooses a narrative about systematic working (another reason why a nil result should not be taken as proof of absent competence).  The other thing I am wondering is that the ‘language’ of systematic thinking is very familiar to people but not so the ‘language’ of systemic thinking.  If a talk sample uses the word ‘goal’ is that because they are thinking systematically, or could they be trying to articulate ‘purpose’ or something similiar?  It’s a hard one this, but I think to get started I’ll use Ison’s table 8.1, page 192 and have four codes:

• talk that reveals (to me) systemic thinking
• report of systemic action
• talk that reveals (to me) systematic thinking
• report of systematic action

how I draw this up in my conclusions will depend on how much I use the codes.  I may find something interesting.

The Dedoose system allows me to ‘weight’ each excerpt when I am attaching a code – for example, on a scale where 1 is ‘weak example’ and 10 is ‘really strong example’.  I feel uncomfortable about doing this because it feels much more ‘evaluative’ (see the exchange of comments I had with Joan).  So I don’t think I will use the weighting system – at least until I am much more familiar – it may be if I get lots and lots of excerpts against a particular code that using the ‘weighting’ facility could help me filter them in some way.  We’ll see,

Okay so after all that here is my ‘code tree’

‘Evidence’ of systems thinking

• Armson’s aspects:
• talk that reveals (to me) that appreciation of multiple interacting variables and interdependence (holism)
• talk that reveals (to me) that drawing on multiple perspectives (pluralism)
• potential for further sub-codes if early analysis shows nuances would be important
• Ison’s criterion:
• use of concepts and ideas linked to the discipline of Systems
• reference to use of systems approaches and tools
• explicit connection with the history of Systems scholarship
• The dualism (NB when use these code will need to add note drawing from Ison Table 7.1 or 8.1 to pinpoint which specific characteristic I am alluding to):
• talk that reveals (to me) systemic thinking
• report of systemic action
• talk that reveals (to me) systematic thinking
• report of systematic action

Opinion of practices that constrain/enable

• positive/appreciative talk about practices that enable systems thinking
• personal level
• interpersonal (micro) level
• partnership (meso) level
• national (macro) level
• global
• negative/critical talk about practices that constrain systems thinking
• personal level
• interpersonal (micro) level
• partnership (meso) level
• national (macro) level
• global
• positive/appreciative talk about practices that constrain systems thinking
• personal level
• interpersonal (micro) level
• partnership (meso) level
• national (macro) level
• global
• negative/critical talk about practices that enable systems thinking
• personal level
• interpersonal (micro) level
• partnership (meso) level
• national (macro) level
• global

References