Draft IAPT assessment criteria for digitally enabled therapies

NHS England and NHS Improvement’s national Improving Access to Psychological Therapies (IAPT) team have developed a draft assessment criteria for a group of products known as digitally enabled therapies (DETs). DETs are products that deliver a substantial portion of therapy via its content but are designed to be used with therapist assistance. The criteria aims to enable a streamlined assessment of DET product suitability for use within IAPT services.

The assessment criteria are currently in draft while the model is being tested. We are looking for feedback from IAPT commissioners, digital developers, service providers, clinicians and service users on the draft assessment model. Provide your feedback in our survey.

Information:

Download the draft IAPT assessment criteria for DETs.

Why is the IAPT criteria being introduced?

The speed of progress in developing therapist-supported digital therapies is such that there is a need for a process that supports innovation and provides a robust assessment of products. The aim is to give national direction on what criteria need to be met by DET products to make them suitable for use within IAPT services.

NHS England and NHS Improvement have worked with NHSX to develop complementary criteria for assessment of digitally enabled therapies, and guidance for assessors and developers, to support IAPT service providers and commissioners to have confidence in using digitally enabled services.

The intention is to:

  • support IAPT commissioners and providers to procure DET products with confidence
  • support digital developers to design and deliver high-quality DET products for IAPT services
  • promote the use of high-quality DET products and encourage digital innovation within IAPT services
What are the criteria based on?

There are 3 main elements to the IAPT assessment:

A ‘core’ element

This is comprised of 8 assessable criteria, including passing the national baseline assessment Digital Technology Assessment Criteria for Health and Social Care. If a product meets the core 8 criteria it can move to the second element of assessment.

An in-depth clinical content and evidence of effectiveness assessment

This reviews the product’s conformance to NICE-recommended therapy and IAPT protocols as well as the product’s level of evidence of effectiveness. This will be conducted by approved IAPT assessors. The result of this element of the assessment is a product that is: ‘not endorsed’, ‘minimally endorsed’, ‘partial endorsed’ or ‘fully endorsed’.

Self-reported disclosure statements

These statements are reported against a set of additional criteria. These disclosure statements from the developer to provide additional information about their product will be published to support providers and commissioners to select appropriate DET products.

Which products can be assessed?

The IAPT assessment criteria are for digitally enabled therapies and should:

  • be designed to treat one of the 13 clinical conditions covered by IAPT: depression, generalised anxiety disorder, social anxiety disorder, panic disorder, agoraphobia, post-traumatic stress disorder, health anxiety (hypochondriasis), specific phobia, obsessive-compulsive disorder, body dysmorphic disorder, irritable bowel syndrome, chronic fatigue syndrome, or medically unexplained symptoms not otherwise specified
  • be designed to treat adults (18+ years)
  • deliver a substantial portion of the therapy via a digital product, alongside therapist assistance
  • have successfully completed the national baseline assessment criteria, currently the Digital Technology Assessment Criteria for Health and Social Care (DTAC)

Once the assessment criteria is introduced, it is expected that all products defined as DETs that meet the criteria as set out above, would take reasonable steps to complete the assessment process.

How does this fit with the DTAC?

The IAPT assessment is conditional on first passing DTAC, which assesses a product’s conformance to clinical safety, data protection, technical security, interoperability and usability and accessibility standards. Therefore, these are not assessed within the IAPT specific criteria.

Information for IAPT commissioners and providers

This assessment criteria model has been designed to provide a consistent and proportionate set of evidence to support the commissioning and procurement of DET products. The criteria are designed to support local decision-making in choosing what DET products are most suitable for local IAPT services. It is important that individuals with the relevant subject matter expertise undertake the assessment and are involved in procurement decision-making of DETs. For example, the service’s clinical safety officer or clinician with equivalent responsibilities should assess the product’s content, evidence of effectiveness and clinical safety documentation to ensure the appropriate implementation of a product.

Guidance notes for completing the assessment criteria

There are 3 main elements to the IAPT assessment: a core criteria, clinical content and evidence of effectiveness, and self-reported statements.

Core assessment criteria

The core criteria were developed to deliver a yes/no response and must be met before the product can be considered for the clinical content assessment.

High-level view:

  • The product has passed the national baseline Digital Technology Assessment Criteria for Health and Social Care (DTAC).
  • The product is designed to treat 1 of the 13 clinical conditions covered by IAPT: Depression, generalised anxiety disorder, social anxiety disorder, panic disorder, agoraphobia, post-traumatic stress disorder, health anxiety (hypochondriasis), specific phobia, obsessive-compulsive disorder, body dysmorphic disorder, irritable bowel syndrome, chronic fatigue syndrome, or medically unexplained symptoms not otherwise specified.
  • The product is designed to treat adults.
  • The product is designed to deliver a NICE recommended therapeutic intervention.
  • The product is designed to be used in IAPT services with therapist assistance.
  • The product is designed to allow two-way communication between the therapist and patient.
  • The product allows the therapist to view a patient’s written inputs on the programme.
  • The product can be commissioned by NHS organisations.
Clinical content and evidence of effectiveness assessment

Clinical assessment

The clinical content assessment has been developed to ensure that products used within IAPT services meet the IAPT principles of providing evidence-based treatment. Therefore, the content and structure of the treatment within the DET must mirror a NICE recommended psychological therapy intervention for the relevant condition.

NICE guidance provides recommendations regarding the type of therapy that is effective for conditions based on evidence derived from RCTs. The therapy protocols used by IAPT practitioners and services are based on these research study therapy protocols. Any product used in IAPT needs to also be based on the approved therapy protocols in Annex C of the assessment criteria.

Two clinical content criteria will be assessed by IAPT expert assessors. To perform this assessment the assessors will require access to both the therapist and patient interfaces of the product.

High-level view:

Evidence of effectiveness assessment

The evidence of effectiveness assessment has been developed to ensure that products have an adequate level of evidence as described by the Evidence Standards Framework for digital health technologies. DETs are categorised as Tier C intervention products and therefore are required to evidence that they meet the best practice evidence standard that is currently described as:

High-quality intervention study (quasi-experimental or experimental design), which incorporates a comparison group, showing improvements in relevant outcomes, such as:

  • patient-reported outcomes (preferably using validated tools) including symptom severity or quality of life
  • other clinical measures of disease severity or disability
  • healthy behaviours
  • physiological measures
  • user satisfaction and engagement
  • health and social care resource use, such as admissions or appointments

The comparator should be a care option that is reflective of standard care in the current care pathway, such as a commonly used active intervention.

The proposed assessment model aims to enable innovation by allowing products of differing maturity to enter the IAPT market - with different levels of national endorsement - while maintaining an emphasis on the need for products to provide high quality evidence of effectiveness. The assessment requires products to provide evidence of effectiveness for review by the clinical assessors.

New digital therapies without best practice standard efficacy evidence (generated through an RCT) would be required to pass the baseline DTAC and IAPT DET assessment. This would result in the product achieving “minimum endorsement”. This means the product will be supported to pilot in a small number of interested IAPT services (up to 5 IAPT services) for a fixed period of time (up to 2 years). During the piloting phase, the product developer is expected to work with local services to generate some evidence of effectiveness. Once there is some evidence of effectiveness, this would result in the product achieving “partial endorsement”. This means the product will be supported to scale to a larger number of interested IAPT services (up to a further 10 IAPT services) for another fixed period of time (up to 2 years). During the scaling phase, it is expected that the product developer works with local services to generate RCT-level evidence. Once the product is able to demonstrate gold-standard RCT-level evidence, this would result in the product achieving “full endorsement”. This means the product will be supported for wide scale adoption.

Once DET products have RCT evidence of effectiveness they can be considered for wide scale adoption in IAPT.

High-level view:

  • The product allows the collection of sessional outcome data from the patient.
  • The product presents a graphical view of the sessional outcome data to the patient.
  • The product presents a graphical view of the sessional outcome data to the therapist.
  • The product presents a graphical view of each patient’s level of risk to the therapist.
  • The product indicates to the therapist if a patient has increased risk over the course of treatment.
  • The product uses IAPT approved outcome measures for each condition as detailed in the IAPT manual.
  • The product has the capability to flow outcomes data to the IAPT system suppliers in a format that can be incorporated into the IAPT minimum dataset.
  • The supplier has training materials for the implementation of the product within IAPT settings.
  • The supplier has the capacity to deliver telephone and online helpdesk support.
  • The supplier has committed to the maintenance and update of the technology and content of the product .
  • The supplier has disclosed their decommissioning cost model.
  • The supplier has published their APIs to enable interoperability between other technology products used within IAPT.
Self-reported assessment criteria

The self-reported criteria have been developed to indicate the capabilities of products that are desirable within IAPT DETs. These criteria are not pass/fail, instead developers are obliged to disclose their answers to these criteria to support commissioners and providers of IAPT services in deciding which products are most appropriate for them.

Provide your feedback

We would like to hear from DET developers and IAPT commissioners, service providers, therapists and service users to ensure the criteria works for everyone.

You can view the draft assessment criteria here.

If you are interested in contributing to the development of the criteria, please fill in this survey.

The implementation of the assessment model will be iterative and based on feedback received from the IAPT community.