In March 2018, the National Institute for Health and Care Excellence (NICE) released an update to its guideline on the recognition, diagnosis and management of ADHD in children, young people and adults. The updated guideline recommends methylphenidate as first-line pharmacological treatment for children aged over 5 years, adolescents and adults, with lisdexamfetamine recommended as first-line in adults only (NICE 2018). However, these recommendations contrast markedly with previous Cochrane reviews, which demonstrated that the quality of the evidence supporting the use of methylphenidate was low and that the results should be interpreted with caution (Castells et al. 2011; Castells et al. 2018; Storebø et al. 2015; Storebø et al. 2018). This article was written by the authors of the previous Cochrane reviews and provides a critical review of the updated 2018 NICE guideline, focusing on the evaluation of its reporting standards and methods for evidence synthesis with regard to pharmacological treatment recommendations (Faltinsen et al. 2018).
Issues with selective reporting
- Lack of a priori registration of protocols: The NICE reviews on efficacy and adverse events are similar to regular systematic reviews, and included protocols to outline study objectives; however, these protocols were not published a priori or registered on Preferred Reporting Items for Systematic Reviews and Meta-Analyses, which represents the international prospective register of systematic reviews. The authors stated that this limits the consistency, accountability and transparency of the NICE guidelines, increasing the risk of selective reporting bias.
- Exclusion of unpublished data and open-label trials: The authors stated that when systematic reviews exclude unpublished data, valuable clinical information can be lost. Furthermore, estimates derived from meta-analyses become more vulnerable to systematic errors and misleading effect estimates. In addition, the clinical trials included in the NICE reviews were double-blind only; however, the differences in adverse events and observable behavioural effects observed between drug treatments and placebo comparators could lead to unblinding. The authors were of the belief that this may have influenced the efficacy outcomes reported by NICE, particularly quality of life and ADHD symptom severity.
- Selection of efficacy outcomes: NICE only reported outcomes assessing ADHD symptom severity and quality of life, excluding outcomes related to substance use, accidents, academic functioning and suicidal behaviour. Whilst these outcomes can be under-reported in clinical trials, the authors stated that they are clinically relevant to clinicians and people with ADHD; therefore, observational evidence should be assessed in combination with randomised data to ensure their inclusion.
- Limited reference to Cochrane reviews: The authors stated that NICE did not cite the two largest Cochrane reviews evaluating the efficacy and adverse events of methylphenidate and amfetamines from 2015 to 2016, which included considerably more studies for the respective interventions compared with the systematic reviews conducted by NICE.
Adjusting for multiple comparisons
- Lack of adjustment strategies for multiple comparisons: Despite the fact that the NICE review on efficacy and adverse events included a large number of outcomes and comparisons, the authors held the belief that no adjustment strategies for multiple comparisons were performed. When this is the case, the chance of false-positive results is dramatically increased. Furthermore, the authors stated that a large number of meta-analyses retrieved data from only one clinical trial with a limited number of participants, which may have increased the risk of inaccurate estimates with wide confidence intervals.
Generalisability of results
- Lack of long-term clinical trials: The clinical trials included in the NICE reviews generally lasted less than 12 weeks, which the authors regarded as a limitation in the generalisability of the data for long-term treatment. The short-term duration of the trials was a major reason for not recommending pharmacological treatment as a first-line strategy in children under the age of 5 years; however, the same recommendations did not extend to adolescents and adults, despite the evidence base also originating from short-term trials. The authors held the opinion that these recommendations are inconsistent and not evidence-based.
Risk of bias and evidence quality
- Low-quality evidence: Both the NICE review and previous Cochrane reviews found widespread low-quality evidence, which limits the interpretability of study findings. The authors stated that the majority of clinical trials in the NICE reviews were subject to a high risk of bias and industry-funded, which may have led to more favourable results and conclusions. The authors also opined that it is contradictory of NICE to include strong recommendations for the use of methylphenidate and lisdexamfetamine in the guideline, despite acknowledging the low quality of evidence.
- Lack of non-randomised studies: NICE largely evaluated randomised studies, and evaluated the overall reporting of harms in the literature as inconsistent, emphasising that studies used conflicting methods to measure adverse events. The authors suggested the inclusion of non-randomised studies, which are often better than randomised studies at assessing adverse events, may have improved the overall evidence base.
- Evidence base supporting the ratio between benefits and harms of ADHD drug treatments: Both the NICE and previous Cochrane reviews demonstrated prevalent adverse events for methylphenidate and amfetamines. However, the authors suggested that the evaluations of the ratio between the benefits and harms of ADHD treatments may be limited by the fact that NICE did not measure all-cause treatment discontinuation in people with ADHD; the authors stated that this is an important measure of acceptability that weighs symptom improvement against safety.
The authors concluded that several of the NICE evidence-based recommendations are useful, including the emphasis on baseline assessments, qualified practitioners for prescribing medications and social support for patients. Furthermore, the 2018 NICE guideline update utilises a more comprehensive appraisal of the clinical evidence in comparison with other recent practice guidelines. Despite this, the authors stated that the 2018 NICE guidance on pharmacological treatments for ADHD is based on systematic reviews limited by methodological flaws and low-quality studies – factors that are not presented in a transparent manner to their readership. The authors suggested that visual cues offered by the GRADE Working Group could serve as clear indications of evidence strength and certainty of evidence. They also suggested that close collaboration between NICE and The Cochrane Collaboration could facilitate improved clinical decision-making.
Castells X, Ramos-Quiroga JA, Bosch R, et al. Amphetamines for attention deficit hyperactivity disorder (ADHD) in adults. Cochrane Database Syst Rev 2011; 6: CD007813.
Castells X, Blanco-Silvente L, Cunill R. Amphetamines for attention deficit hyperactivity disorder (ADHD) in adults. Cochrane Database Syst Rev 2018; 8: CD007813.
Faltinsen E, Zwi M, Castells X, et al, Updated 2018 NICE guidance on pharmacological treatments for people with ADHD: a critical look. BMJ Evid Based Med 2018; Epub ahead of print.
NICE guideline 2018. Attention deficit hyperactivity disorder: diagnosis and management. Available at: https://www.nice.org.uk/guidance/ng87. Accessed February 2019.
Storebø OJ, Ramstad E, Krogh HB, et al. Methylphenidate for children and adolescents with attention deficit hyperactivity disorder (ADHD) (review). Cochrane Database Syst Rev 2015; 11: CD009885.
Storebø OJ, Pedersen N, Ramstad E, et al. Methylphenidate for attention deficit hyperactivity disorder (ADHD) in children and adolescents – assessment of adverse events in non-randomised studies. Cochrane Database Syst Rev 2018; 5: CD012069.