In 2016, Thames Valley (TV) Perinatal Mental Health (PMH) Network mapped regional service availability for women with PMH problems, revealing a dearth of performance evaluation processes.
In response the Network designed an online self-assessment tool for NHS services working with women with or at risk of developing PMH problems. This enables maternity, health visiting, secondary care mental health and Improving Access to Psychological Therapies (IAPT) services to benchmark performance against NICE QS115 and Royal College of Psychiatrists PMH workforce standards. It has been endorsed by NICE and can be found here https://pmh.visualdatamatrix.com/
Results are available on a live dashboard and can be viewed at regional, Local Maternity System, CCG and service level.
Benchmarking against NICE QS, the PMH Matrix identifies good practice and gaps in service provision and data collection; guiding service development tailored to the local service landscape and demand, mapping performance progress over time.
The Matrix is being implemented across NHSE South regions.
Aims and objectives
The aim was consistent with the NICE QS. We used the QS wording throughout the Matrix.
Aim: to improve quality of care for women with or at risk of developing PMH problems.
- Apply NICE evidence-based quality standards:
- across the care pathway, in universal and specialist primary and secondary care services
- to reflect the multidisciplinary approach required to care for women with PMH problems
- Improve accountability of performance through enhanced transparency:
- creating a platform with live service performance data, that can drive quality improvement
- NHSE Five Year Forward View prioritises PMH for mental health and maternity services, which obliges services to demonstrate quality of PMH care.
- Improve equity of services:
- the 2016 TV survey showed marked service heterogeneity. Using an agreed set of evidence-based standards, NICE QS is helping to reduce regional inequalities in service availability and quality.
- Develop a service evaluation tool that generates contemporaneous performance data to ensure that services:
- use NICE QS115 SMART measures of service improvements
- develop in response to local need using live data
- map quality trends over time
- use competitive drivers for improvements through benchmarking against counterpart services.
Oversight of the Matrix by the regional network mitigates risk of unhealthy competition and provides a platform for shared learning of good practice.
- Facilitate improved coordination of a multidisciplinary and multiagency approach for women with complex health needs:
- confidential enquiries into maternal deaths identify that fragmented care between services contributes to poor maternal and infant outcomes, including maternal death
- applicability of NICE QS115 across maternity, health visiting, IAPT, secondary care mental health services with service performance at CCG and regional level provides a mechanism for commissioners and service leads to coordinate local service developments across primary and secondary care
Reasons for implementing your project
2016 baseline mapping survey showed:
- patchy service provision
- paucity of service evaluation
- services not routinely using NICE QS to evaluate performance; preponderance of using referral data to demonstrate demand; negligible use of direct quality measures
- absence of coordination between services expected to work closely for women with complex health needs
- many IT systems did not collect data relevant to NICE QS
- services in TV were heterogeneous, developed opportunistically without overarching strategy, creating inequalities in service availability and quality.
Opportunities identified to:
- enable earlier identification of and intervention for women with PMH problems
- promote informed decision-making by women, including family planning,
- promote use of preventative measures by maternity and health visiting services, reducing burden on specialist services
- provide improved care, reducing impact of PMH problems on families.
Opportunities to increase quality and efficiency of care:
- In an IAPT service the Matrix highlighted women often waiting over 6 weeks to start treatment. The service used findings to enhance the speed of access to treatment and to improve women’s experience by proactively contacting and monitoring women on the waiting list.
- Data relating to NICE QS115 is not always available through service IT systems. Identified as a QI opportunity.
- Matrix development led by the TV Network.
- members involved in testing and piloting at every stage of development
- women with lived experience were members of the network during development
Identifying location and reach of services.
- the Matrix covers the 14 Local Maternity Systems in the South East and South West
- 238 organisations could potentially register
- the Matrix makes it possible to contemporaneously monitor referral rates for mental health and IAPT services.
- bringing together data from IAPT, Adult Community Mental Health and Specialist Perinatal Mental Health services, total referrals can be seen for a locality, enabling identification of good provision or gaps in service
- engagement with local stakeholder services in Thames Valley through the regional Network has enabled:
o 20 TV services to register and start to use the Matrix
o a further 28 services to register across the south of England in the last 3 months during roll-out
o other key stakeholders including Local Maternity Systems and CCGs, to access Matrix dashboard
How did you implement the project
Overview of implementation stages:
- Network membership buy-in following 2016 survey
- trust buy-in and agreement to participate
- software development (commissioned through developer, tested and piloted with network)
- roll-out across TV
- monitoring and supporting participation
- roll-out across south of England
- obtaining Trusts’ agreement to participate in regional audit where substandard performance would be made public needed lengthy negotiation and encouragement to be transparent and focus on the value of benchmarking to improve care
- mapping services eligible to use the Matrix and establishing contact with each service
- engaging and keeping track of participating services and clinicians. This was achieved by robust membership support, IT monitoring and project management
- asking frontline clinicians to undertake this quarterly audit without extra time or funding, made more challenging by additional NHSE monitoring requirements for specialist PMH services.
These were supported by:
- Regional NHSE funding for costs of network leadership roles, software development and project manager
- ensuring the Matrix design was minimally onerous for participants; software was developed to make it visually appealing and simple to use, with network membership consulted at each development stage.
Implementation has required on-going leadership and support, provided by network leads and the project manager. Network relationships have been key to maintaining participation and colleagues committing their time to undertake quarterly audits, which has strengthened the Network, evolving into a highly productive network. This has been key to members promoting the virtues of the Matrix and commitment to using it and NICE QS to drive long-term service improvement.
Implementation has also involved:
- the appointment of a project manager with a remit to assist services to register and use the Matrix, implementing it across the south of England. The role was extended for an additional five months to continue implementation and ongoing support for Matrix users.
- meeting with clinical leads, network managers and clinicians in PMH networks which meet quarterly, geographically spread across the south of England
- QI workshops across the south with support from NICE to introduce the Matrix and facilitate networking between services
- meeting individually and in groups with clinicians to introduce the Matrix, register services and help with initial data entry
The project has led to:
- widespread engagement and participation across TV
- increased awareness of NICE, QS115 and CG192
- development of IT systems to improve data collection
- Improved transparency of service performance
The project met initial objectives:
1) uses NICE QS115 as its basis
2) the Matrix dashboard is accessible to all participants so performance can be tracked over time and compared with other services
3) the Matrix brings together data across services so it is now possible to identify the actual (rather than estimated) demand on local services pathway
4) Data collected is already being used for QI initiatives to improve quality and efficiency. These include:
- IAPT waiting times initiative to ensure that women are assessed within 2 weeks of referral and start treatment within 4 weeks of assessment. They are also contacted while on the waiting list to monitor how they are.
- liaison with trust IT services to improve collection of data relevant to NICE QS115
- detailed trust-wide audit of the use of sodium valproate in women of childbearing age was undertaken in a specialist PMH service following findings from quality standard 1
5) early opportunities to identify women at risk of PMH problems were being missed when a maternity service discovered they could not identify whether women are asked about their emotional well-being at antenatal and postnatal appointments. They are changing maternity records to prompt and record this
• improved engagement with service users and improved shared decision-making. In some adult mental health services teams found that information about the potential impact of pregnancy on a woman and her baby was either not being discussed or such a discussion was not being recorded. This has prompted discussions within the services to ensure that discussions both take place and are recorded.
Key learning points
- sustained leadership and local ownership are central to success, minimising delays or gaps in data entry ensuring provide a useful overview across a service or region
- PMH Network relationships were key, engaging people to commit time and effort
- early buy-in from frontline clinicians and service/trust leads is crucial as part of a commitment to transparency. Trust in sharing data is crucial where standards are not being met, to drive improvements forward
- importance of detailed user piloting and testing, ensuring the product is useable and minimally burdensome.
- need for alignment with other national drivers to reduce over-burdening services with data monitoring (especially specialist PMH services).
- relationship with the software developer crucial in ensuring the project meets specific needs of involved organisations
- implementation delays occurred due to internal organisational discussions before services could commit to sharing data
How would we do it differently? Need to:
- anticipate extra time for implementation across a wide geography
- target specialist PMH teams for additional support in light of existing NHSE reporting requirements
- encourage network leads to own the Matrix and roll-out, providing local leadership and capitalising on local relationships
Pointers for other organisations facing similar challenges:
- involve people using the tool throughout; and engage trusts and commissioners to commit to transparency early to ensure the tool is useful and useable as a lever for quality improvement
What might be successful and what to avoid:
- QI workshops offered the opportunity for services to meet
- keep the interface clear and straightforward to use
- test everything carefully before launching a tool
- ensure teething problems are ironed out before implementation
- ensure resources are available from experts in visual presentation of complex data to create an easy-to-use and visually accessible dashboard from the outset