Peers and pairing methods

The choice of peers and the methods by which they were paired was crucial. Care was taken by the ILCOP core team to match teams with different and complementary strengths and weaknesses, with the intention of avoiding unhelpful dynamics, such as a ‘good’ team feeling it had little to gain from being paired with a ‘bad’ team. This proved largely successful. We didn't want trusts whose results on the National Lung Cancer Audit were perhaps not so good to feel it was a case of someone coming in with fantastic results, and them feeling almost victimised, demoralised, ‘It's them and us’. It's about, ‘Well we can do something well and you can do this well’, so they're both on a fairly even footing. (ILCOP Core Team Member, I-35)

Including a minimum of three core MDT members in the visits was also important because it ensured that participants felt the review was conducted by peers working in similar roles and settings, and it brought together different professional perspectives on the same issue, generating rich discussion. So the huge benefit of ILCOP has been the fact that your peers who have the same problems that you have, have gone to your place and had a look at it so you're going to listen to what they point out and say, rather than some external guy who's never done thoracic surgery who's now coming down to talk down and tell you. (Physician, I-59)

Minimising logistic burden

Arranging reciprocal visits between busy practitioners with multiple commitments within a limited timeframe was a significant challenge. Substantial investment was needed by the core team in administrative tasks, such as room bookings and travel arrangements, and ensuring that data were easily available for presentation. They were bringing us together and saying what days can you do it …, sort of, trying to plan your days when we got there and they were very supportive with trying to, sort of, bring data along, so we didn't have to do all the trawl ourselves and things like that. (Clinical Lead, I-63)

The challenges of organising visits led to considerable delays for some participants, sometimes resulting in a loss of focus, clarity of understanding and time for implementation.

Structure of visits

Also important to the optimal functioning of the model was the creation of safe environments for learning and sharing. It's allowed us to focus on how we're doing things in what feels like quite a safe way—sort of unthreatening, maybe. It's about the time and space to focus on what we're doing and to try and find ways of improving. (Clinical Nurse Specialist, I-02)

Creating a ‘safe space’ required careful management. The prospect of RP2PR could seem daunting or potentially adversarial for participants. The reciprocity of visits—with visiting teams subsequently becoming host teams—had a helpful disciplining effect: that ‘reviewers’ knew they would later be ‘reviewed’ by the same team encouraged visitors to be respectful and constructive in their feedback. Reciprocal visits were important in allowing relationships of trust to develop over time between the teams, which in turn supported more open sharing and learning both formally and informally. Actually we learnt more probably just from the gossip afterwards, on the second occasion. (Clinical Lead, I-01)

Observations suggested that having most of the day together—rather than a short meeting—was important in encouraging openness. Visitors had time to fully understand the host trust's situation, the challenges they faced and how they had got to where they were now. This helped avoid misunderstandings and feelings of judgemental accusations, and generated more locally appropriate solutions while avoiding defensive responses of ‘that won't work here’. It took all day to develop a sufficient understanding of the things that are going on, for people to figure out that this is actually one of the problems. It was right in the last session they began to think about the strategies for how to develop it … I felt that was a big contrast from the beginning to the end of the day. (Fieldnotes)

Allowing host teams to discuss and present their views on their national audit data, patient experience data, or the MDT meeting before their visitors commented also contributed to the creation of a ‘safe’ environment. Another key strategy was structuring discussion to include direct peer-to-peer (eg, nurse-to-nurse) discussion first, then discussion within teams, and then feedback to/from the paired team. Participants appreciated the opportunity to share challenges and working practices with their direct counterpart in the opposite team. It helped to strengthen the ‘voice’ of groups at risk of being marginalised by predominantly clinical discussions, and raised some individuals’ confidence in contributing to subsequent group discussions. I've actually seen my own MDTC sort of, start to take on a bit more responsibility because I think it empowered her a bit to feel that she is a more important part of the team. (Clinical Lead, I-12)

However, some found it more difficult to identify relevant and useful practices to apply in their own role, and some non-clinical participants also felt less able to participate in the clinically focused discussion. Once they start talking about technical stuff, I've glazed a bit and I was interested but it was irrelevant. (MDTC, I-08)

Facilitator role

Independent facilitation was important to ensuring inclusion of all voices, focus on the issues at hand, and good timekeeping, while avoiding protracted or bad-tempered discussions. The facilitator was also valuable in steering conversations towards doing the best with what teams had rather than complaining about deficits, and ensuring that discussions were concerned not only with identifying problems, but also focused on recognising teams’ strengths and drawing on the range of experiences and expertise to generate solutions. I hear the Clinical Lead saying ‘there are a whole host of things I thought were pretty atrocious about the other team’ she was getting into her stride with the criticisms and recommendations … And this is the point at which the facilitator goes over and tries to encourage the team to be gentle and to frame things constructively. (Fieldnotes)

Most teams responded positively to working with an external facilitator and valued her work, but this was a challenging role. Being non-clinical meant that the facilitator was accepted by teams as objective and impartial, but also meant that it was more difficult to challenge participants’ views on clinical grounds, and could on occasion result in some resentment. There was just too much trying to manage us … I found that a bit difficult and I had feedback from other members of the team that found that quite difficult as well. (Physician, I-19)

Some teams were more open and accepting of feedback than others; the more closed teams limited what could have been gained from the process. It is also possible that in their desire to be respectful, some teams held back honest criticism. This highlights the challenges in facilitating a productive process through which teams felt able to expose their service, and themselves, to scrutiny and challenge. I didn't feel that they were being quite as objective about their processes and their MDT as we had on our day. I felt they were being a little bit protective, particularly one member was very much, ‘Oh we're fantastic and we're doing everything fine.’ … I gave up fairly quickly trying to discuss any of it. (Clinical Nurse Specialist, I-13)

Credibility of the review process

The fact that the project was led by a respected institution (the Royal College of Physicians), and built on a previous, well received RCP-led project with chest physicians helped establish initial credibility for ILCOP. Because it's got backing and stamps with lots of, you know, the Royal College of Physicians’ cancer action team, lots of people had officially said this was OK and sponsored it and it looked like a reputable study. (Clinical Lead, I-47)

At NHS trust level, the requirement for approval to participate from chief executives helped legitimise time spent on participation. It was equally important, however, that what was presented for review was perceived as credible. National Lung Cancer Audit data and patient experience data were used in a targeted way to identify weaker areas of service provision and factors in MDT dynamics that seemed to contribute to variations in outcomes. However, due to time lags and incompleteness, some teams felt the data were not an accurate reflection of current service standards and were therefore of limited value. The data was a bit out of date … Our data wasn't so good, but we knew why it wasn't so good and we knew it was a data collection issue, so, it wasn't as useful as it could have been. (Clinical Lead, I-16)

In some cases the number of patient experience questionnaires returned was low, prompting participants to challenge the validity of the results. Some participants remained unpersuaded of the usefulness of such data, or indeed of the existence of a ‘problem’ to be addressed. He was very rude and said, ‘This is all bollocks, blah, blah, blah’… I think I said that ‘I'm not really certain that this is useful, going through everything laboriously, particularly when you've got N=4, I don't know what value that is’. (Physician, I-19)

It was not only data that were presented for review, however. The observation of a live MDT meeting was perceived by participating teams as especially valuable, credible, and resistant to ‘gaming’ in contrast to paper-based peer review. It was also an important source of ideas for the visiting team. The national peer review … was just very much to do with collecting data and nothing useful had come out of it at all … the whole emphasis of ILCOP reciprocal peer review was completely different, on looking at what are we doing and where we want to improve. (Clinical Lead, I-12) When you actually get across and see another team doing their, their normal MDT, I think most of the useful ideas come through that. (Physician, I-60)

Exchange visits and ‘live’ peer review offered a constructively disruptive perspective on team dynamics and service standards which had become normalised and accepted as immutable by teams, even though they were sometimes clearly suboptimal. Equally, the opportunity to discuss such issues with an external party helped teams to feel more able to tackle such sensitive issues.

Ensuring implementation and impact

RP2PR was generally a positive experience for participants, who often reported that it mobilised collective action in relation to quality that would not otherwise have happened. Taking part in an externally driven process was seen as an important ‘push’; it legitimised taking time out from busy schedules, and motivated teams to reflect on where improvements were needed, their successes, and where other teams were experiencing similar challenges. You tend to feel pretty isolated and carry on doing MDTs like you've always done … you just plod in your own, sort of, furrow, so it was really to get an idea of, see how other people's MDTs worked and see if there was anything we could do to improve our service. (Clinical Lead, I-63) You get to see not only what you do badly but also what you do well, and I think it's nice to have that positive feedback sometimes … and just acknowledging that we maybe share the same difficulties, even that can be quite a relief. (Clinical Lead, I-19)

Overall, teams saw RP2PR as an engaging, productive way to identify areas for improvement and generate solutions. For such energy to convert into benefits for patients, it was necessary for action to follow. Improvement plans detailing goals and methods for improvement were developed and submitted by 29 of the 30 teams in ILCOP. However, turning intentions into action was not straightforward. Some teams, by their own admission, had not fully understood what they had signed up to. It was also easy for the core ILCOP team—immersed in project planning for many months—to underestimate how much explanation participants new to the process would need. In some instances, disagreement or lack of communication about who would subsequently be taking the improvement plans forward stalled progress. It seemed very doable and that sort of thing, obviously once we got involved we realised it was more than that, which we didn't really sign up to and that's when things got a bit difficult. (Physician, I-22)

Another challenge was securing cooperation from more peripheral MDT members (eg, where improvement plans required changes to systems in the pathology department), suggesting that RP2PR may work best with the involvement of as broad a range of professionals as possible. Directly involving managers in the RP2PR process (including observation of MDT meetings, which managers would not normally attend) also seemed to be especially useful, for example by persuading managers to provide support to improvement work, offer resources, and help teams to align requests with existing policies, targets and other managerial edicts. Well I think the most important thing was probably getting our manager on board, and I think that really made a huge difference because once she saw a couple of things that really would make a big difference, were very simple to do, and were within her power—you see that's the thing, it's all about who's got the know-how and the power to do these things. (Clinical Lead, I-01)

The ILCOP core team reviewed improvement plans submitted by the teams and provided comments on why and how changes could be made. This was important, as many teams lacked quality improvement experience. It also helped minimise the risk that teams would choose to focus on ‘easy wins’ in order to lessen the burden of improvement work or avoid setting themselves up for failure. Encouraging teams to set themselves appropriately challenging goals and to make progress required careful negotiation, and underlined the importance of building supportive relationships between the core and participating teams. You know, gently coercing us to come out with action plans and things that we should do and timetable and things, so but they weren't in your face but they, sort of, put just enough pressure on you to, sort of, make sure it got done. (Clinical Lead, I-63)

Ensuring follow-through and impacts that would benefit patients was undoubtedly a challenge, however. While there were clear examples of improvements (box 2), there was also some evidence of a lack of clarity around what constituted ‘success’, and that some teams’ definitions changed over time to reflect their actual achievements rather than the original goals of their improvement plans. This tension may have been exacerbated by difficulties the core team experienced getting teams to submit local measurements to drive and refine improvement efforts during the project.