Systematic reviews of Randomized Controlled Trials (RCTs) are studies designed to identify, summarise and critically appraise the best available evidence on therapeutic interventions and provide allowance to the decision regarding patient care and health policy.1,2 This study design aims to avoid the unnecessary and redundant production of primary studies, provides quick answers to health questions, and can combine results from similar studies in meta-analysis, increasing the statistical power and confidence in estimated effects.1 By applying adequate methodological recommendations for developing a systematic review of intervention, a careful evaluation of the included RCTs is conducted, directly implying the reliability of the evidence through the results obtained since RCTs are the most suitable primary study design to answer a clinical treatment question. Thus, the applicability of the results on the efficacy and safety of treatments in clinical practice can help health professionals, policymakers, patients, and guideline producers.1

Rigorously developed evidence in dermatology has increased over time, primarily due to concerns about the impact of methodological flaws on the estimated effects of interventions evaluated in clinical trials. As a result of the numerous publications of RCTs on dermatological treatments, a growing number of systematic reviews have been published in this area, and the appropriate methodological recommendations for preparing and conducting a systematic review, as recommended by Cochrane, are not always followed by the authors, leading to systematic errors and biased results, most of the time overestimating the evaluated treatment.1,3,4

Regarding that, in the last years, meta-epidemiological studies have been conducted to evaluate the impact of some methodological limitations of clinical studies on the observed results. Additionally, these meta-research studies verify the methodological quality to support future research precluding the occurrence of the same biases and drawing reliable evidence to establish decision-making processes in healthcare. The unit of analysis in a meta-epidemiological study is individual studies and not patients, as in traditional clinical research.5,6

The treatment of dermatoses, such as vesiculobullous diseases, has been investigated by several studies. These clinical skin conditions present the appearance of vesicles or blisters as primary clinical manifestations, directly affecting the patient's quality of life. These are affections of variable etiology and may result from infection, genetic or metabolic alterations, drug hypersensitivity, and autoimmune conditions.7–9 Since the emergence of corticosteroid therapy, these patients have had access to treatment resources, which could be associated with other drug classes, such as immunosuppressants.10 The severity of the disease and the need for effective and safe treatments reinforce the requirement for clinical studies with high methodological quality. Thereby, the purpose of this meta-epidemiological study is to map and critically appraise systematic reviews that assessed pharmacological treatments for vesiculobullous diseases.

This meta-epidemiological study follows the recommendations proposed by Murad et al. 2017 for reporting meta-epidemiological studies,4 in addition to the pertinent items of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020).11

Given the growing number of publications involving the treatment of vesiculobullous diseases, this meta-epidemiological study aimed to analyze the methodological quality of the systematic reviews published on this condition. Considering only the high level of evidence, nine systematic reviews of RCTs were identified, assessing different interventions for vesiculobullous skin diseases. Unfortunately, most were rated critically low and low quality according to the AMSTAR-2 criteria. Additionally, some reviews did not present adequate search strategy components and did not assess the certainty of the evidence through the GRADE approach and failed to identify and report awaiting classification and ongoing studies, as recommended by the Cochrane Handbook of Systematic Reviews of Interventions.13 Overlapping RCTs were also evaluated, simultaneously showing repeats in at least five reviews.

Using a tool such as AMSTAR-2 allows readers to identify the methodological quality of the evidence derived from a systematic review of randomized and non-randomized clinical trials, directly impacting the intended use of these results for clinical decision-making. It is essential to verify the meaning of the deviations committed by the review authors that were categorized as low or critically low quality.11

Systematic reviews are subject to bias; however, using a rigorous methodology allows researchers to access minimal systematic errors in their results.13 In this meta-epidemiological study, systematic reviews rated as low or critically low quality presented methodological weaknesses such as no report published register review protocol. The elaboration and publication of a previous review protocol reduce the possibility of reporting bias, that is when a study deviates from the methodological planning components and presents only selected outcomes that generally benefit the intervention of interest.31 Protocol drafting protects the results, even if they are harmful; any protocol differences justified in the published article also contribute to the reliability of the results.

None of the included systematic reviews explained the selection of study designs for the review, an important criterion when there are randomized clinical trials available on the research question, but there is a tendency to include non-randomized ones that may show incomplete summaries of the real treatment effects.11 Additionally, none of the included systematic reviews provided a list of excluded studies on the second selection stage with justifications, creating the risk that they remain invisible and the impact of their exclusion on the review unknown. The individual financing of the included RCTs in the systematic reviews was also not described by any review, and it could infer bias regarding financial conflict of interest.

The discussion and interpretation of the impact of the risk of bias of RCTs in the review findings are essential to support clinical decision-making, as is the analysis of these risks when pooling studies in meta-analysis. Heterogeneity between included RCTs should also be assessed through clinical question variation (PICO) and methodological considerations for having as homogeneous studies as possible.11

It was possible to observe a publication date interval between included systematic reviews. The newest review was published in 2021, classified as critically low quality, and did not include all available RCTs with the PICO question analyzed, included by older reviews. In addition, more than half of the included RCTs regarding pemphigus are duplicates between systematic reviews. This fact reflects the need to conduct practically simultaneous but methodologically heterogeneous systematic reviews, which may overestimate the intervention effects and confuse health professionals, policymakers, and patients. It is worth mentioning that one of the roles of a systematic review is to avoid duplication of effort by bringing together all published primary studies on an issue. This study observed that the critically low quality of several reviews duplicates unnecessary efforts and makes it unfeasible to use the findings to support clinical practice. In the end, despite the various reviews identified and several RCTs published, methodological rigor is still needed to clarify the best treatments for vesiculobullous diseases.

Furthermore, additional analysis regarding search strategies and the certainty of the evidence assessment using the GRADE approach is methodological criteria that could be incorporated into an updated version of the AMSTAR-2 tool, given the substantial impact these stages had on the results derived from a systematic review of interventions.

The present study has the advantage of conducting a comprehensive database search without any date or language restriction, making it difficult not to find published systematic reviews comprising the clinical question, and a rigorous methodology to select and critically appraise the included reviews, besides using a validated tool worldwide used to assess the methodological assessment of systematic reviews. To our knowledge, this is the first meta-epidemiological study to evaluate the quality of systematic reviews on treatment for vesiculobullous skin diseases.

There is a scarcity of systematic reviews with randomized clinical trials in some bullous dermatoses such as dermatitis herpetiformis, epidermolysis bullosa, erythema multiforme, Stevens-Johnson syndrome, bullous impetigo, hydroa vacciniforme, linear IgA bullous dermatosis. This is due to the low incidence of these diseases, which makes it challenging to design randomized clinical trials, the most appropriate study design for treatment questions. Therefore, the authors did not include systematic reviews that evaluated other study designs.

Regarding the included reviews, there are limitations related to methodological flaws and low quality of evidence. In addition, the AMSTAR-2 tool has some limitations related to its use; the tool was created in 2007 and updated in 2017, requiring adaptation and more time for application,32 and not as detailed a description of the search, requiring additional analysis to assess that.

Considering the methodological quality of the systematic reviews on treatment for vesiculobullous diseases, 5 (55.6%) assessed bullous pemphigoid, epidermolysis bullosa, pemphigus vulgaris, and foliaceus were classified as critically low-quality. One review (11.1%) assessing pemphigus vulgaris and foliaceus was graded as high quality, 2 (22.2%) regarding toxic epidermal necrolysis and bullous pemphigoid as moderate quality, and 1 (11.1%) assessing pemphigus vulgaris and foliaceus was classified as low quality. Additionally, 26 randomized clinical trials overlapped in the included reviews, search strategies were not fully reported in most of them, and the GRADE approach was not used to assess the certainty of the evidence in any of the reviews. Therefore, there is an urgency to improve the methodological quality of future vesiculobullous disease systematic reviews to avoid unnecessary efforts and increase the reliability of this evidence to support the clinical decision.

None declared.

Kamilla Mayr Martins Sá: Conception and design of the study, data collection, article writing, acquisition, analysis, and interpretation of data, critical review of the literature, and final approval of the final version of the manuscript.

Juliana Cavaleiro Rodrigues: Acquisition, analysis, and interpretation of data, critical review of the literature, final approval of the final version of the manuscript.

Lígia Borges da Silva: Acquisition, analysis, and interpretation of data, critical review of the literature, final approval of the final version of the manuscript.

Giovanna Marcılio Santos: A critical review of the literature, critical review of important intellectual content, and final approval of the final version of the manuscript.

Mileny Esbravatti Stephano Colovati: A critical review of the literature, critical review of important intellectual content, final approval of the final version of the manuscript.

Ana Luiza Cabrera Martimbianco: Conception and design of the study, effective participation in the orientation of the research, article writing and critical review of important intellectual content, acquisition, analysis, and interpretation of data, critical review of the literature, final approval of the final version of the manuscript.

None declared.