HOW TO MAINTAIN RIGOR WHEN CONDUCTING AN INTEGRATIVE REVIEW?

¿CÓMO MANTENER EL RIGOR EN LA CONDUCCIÓN DE UNA REVISIÓN INTEGRATIVA?

COMO MANTER O RIGOR NA CONDUÇÃO DE UMA REVISÃO INTEGRATIVA?

https://doi.org/10.31011/reaid-2024-v.98-n.4-art.2371

¹João Vitor Andrade

²Juliana Cristina Martins de Souza

¹PhD student in Nursing. Universidade Federal de Alfenas. Alfenas, Brasil. Orcid: https://orcid.org/0000-0003-3729-501X

² PhD student in Nursing. Universidade Federal de Alfenas. Alfenas, Brasil. Orcid: https://orcid.org/0000-0002-1941-2262

Corresponding author

João Vitor Andrade

Rua Gabriel Monteiro da Silva, 700, Centro, Alfenas, MG, Brazil. CEP 37.130-001. Telefone: +55(35) 3701-9471, E-mail: jvma100@gmail.com.

Submission: 29-08-2024

Approval: 30-09-2024

Dear Editor,

We are writing you this letter to discuss a growing trend in the healthcare field: the trivialization of the integrative review, an originally systematized method that has become more subjective over time. The integrative review, designed to synthesize the results of studies on a given theme and promote a broad understanding of existing knowledge⁽¹⁾, has been carried out frequently, often without the necessary methodological rigor. This scenario compromises both the quality and reliability of findings, resulting in conflicting or inaccurate conclusions.

To illustrate the increase in integrative review publications, a simple search on PubMed using the term “integrative review[Title/Abstract]” found 5,437 studies, of which 59.55% (3,238) were published between 2020 and 2024. This growth shows the popularity of this type of manuscript, but also highlights the urgent need to establish a clear protocol and consensus for conducting these reviews. In light of these developments, it is essential to answer a recurring question among authors and readers: how can rigor be maintained when conducting an integrative review?

The integrative review has a key role in identifying gaps in the literature and formulating new hypotheses and practical guidelines. However, even with the six stages of this research method being clearly defined [(1) formulation of a research question or objective, (2) systematic literature search according to established criteria, (3) categorization and critical evaluation of the selected studies, (4) evidence analysis and synthesis, (5) interpretation and discussion of the results, and (6) dissemination of the findings]^(1-3), the immaturity or lack of reliability and commitment of researchers has been a recurring challenge in the application of this method, negatively impacting the conduct and validity of integrative reviews.

In Brazil, the most cited articles as references for integrative reviews are: “Integrative review: a research method for incorporating evidence in healthcare and nursing⁽¹⁾” (9,630 citations until 28/08/2024), and “Integrative review: what it is and how to do it⁽²⁾” (5.567 citations until 28/08/2024). However, several authors who use these texts as a basis seem to have failed to analyze them carefully enough, resulting in reviews replete with errors.

Among the most common mistakes are inadequate choice of theoretical framework; incorrect use of acronyms when formulating research questions; inadequate definition of information sources; inappropriate use of specific thesaurus for each information source; searching in all fields without adequate filtering; time delimitation without justification; incorrect application of data extraction tools; extensive and poorly structured synthesis tables; lack of evaluation of the strength of evidence and methodological quality of the studies; incorrect inclusion of elements specific to systematic reviews; limitation of languages without clear criteria; and lack of adequate synthesis of the evidence.

Regarding inadequate use of the theoretical framework selected, this error is perhaps the simplest to correct, as all that the authors have to do is to become familiar with the method, which can be done by reading studies such as those previously mentioned, or other well-conducted review studies. It should be noted that authors can create a checklist based on the framework adopted and fill it in as each stage of the integrative review is completed, thus ensuring that all the necessary steps are followed in an organized and methodologically correct way^(1,3).

Failure to develop a search strategy can be directly related to poorly formulated research questions. For this reason, it is recommended to use an acronym or strategy to help structure a research question. One of the most commonly used acronyms is PICO (Population; Intervention; Comparison; Outcome), however, due to the difficulty in defining some components of this acronym when conducting integrative reviews, it is not the most recommended one^(3-5).

Thus, the PCC structure is suggested (Population [refers to the group of interest], Concept [theme or intervention that will be researched], Context [describes the setting where the study is conducted]), in which it is necessary to clearly define each of these components(4). For instance, a research question based on this structure would be: “What is the evidence on pain management (Concept) in cancer patients (Population) in home-based palliative care (Context)?” In this case, the focus of the integrative review would be to search for studies that address pain management in cancer patients, specifically in the home care context.

Whereas using the PICo structure (Population [refers to the group of interest], Interest [represents the phenomenon or experience to be explored], Context [describes the environment or situation in which it occurs]), more focused research questions can be created⁽⁴⁾. An example would be: “What is the evidence on the experience (Interest) of elderly patients (Population) in long-term care institutions (Context)?” In this case, the integrative review would research the experience of elderly patients in the context of long-term care institutions, focusing on the experience of the care they receive.

These acronyms are useful tools for structuring the questions so that they facilitate the search for relevant evidence. It is worth noting that their main function is to ensure that no relevant terms are excluded from the search, increasing the effectiveness of the search strategy and the comprehensiveness of the results^(4-5).

In addition, the inadequate definition of information sources (databases, libraries), often selected by the authors on the basis of their personal familiarity, rather than considering their relevance to the topic, ultimately limits the scope of the review⁽⁶⁾. There is no consensus in the literature on the ideal number of information sources to be used in integrative reviews, and this choice is often based on the personal opinions of the researchers, ranging from the use of a single source to several. Based on the authors’ experience, it is recommended to use at least three sources to ensure that the review is robust and comprehensive. In addition, it is important for researchers to become familiar with the scope and specificities of each of the available information sources⁽⁶⁾.

Figure 1 presents six information sources widely used in integrative reviews: Medical Literature Analysis and Retrieval System Online (Medline/PubMed), Embase, Scopus, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Latin American and Caribbean Health Sciences Literature (LILACS), and Web of Science. It should be noted that there are several other sources⁽⁶⁾, which were not included in this reflection. Another factor worth mentioning is that the analysis of gray literature in integrative reviews is optional and is not mandatory for conducting this type of study⁽³⁾.

Figure 1 - Global coverage of the main information sources for integrative reviews in the field of healthcare.

Source: Prepared by the authors, based on the scope listed on the websites of the sources of information.

Also noteworthy is the inadequate use of thesauruses specific to each information source. It is not uncommon to find reviews that have applied the Health Sciences Descriptors (DeCS) or Medical Subject Headings (MeSH) to all the information sources consulted, when in fact these controlled terms are only applicable to some specific sources^(3,5).

To resolve this problem, it is recommended to use the most suitable thesaurus for each information source when collecting studies. For example, MeSH should be used in Medline/PubMed, Scopus and Web of Science; DeCS in LILACS; Emtree in Embase; and Subject Headings in CINAHL^(3,5). In the case where an information source lacks a specific thesaurus that represents or conceptualizes the objects of the research question, the use of keywords is suggested. However, when an adequate thesaurus is available, the use of keywords is not recommended, as it may result in a less precise and broader search than necessary.

The “all fields” search without adequate filtration of the information source records can lead to significant problems when conducting integrative reviews. This search method, as it covers all available data fields, often results in a large number of studies that may not be directly relevant to the research question. This practice not only unnecessarily increases the workload of researchers, but can also compromise the quality of the review, as the excess of irrelevant data makes the analysis more complex and prone to errors⁽⁵⁾.

To avoid this problem, it is recommended that authors should restrict their search to the title, abstract, and keyword fields, where the main terms that represent the studies are located. This enables a more specific and targeted search, improving the screening and selection of studies that effectively respond to the objective of the review, ensuring greater precision and relevance in the analysis⁽⁵⁾.

It should also be noted that authors often establish time delimitations when developing integrative reviews without adequate justification or with justifications based on national events. This can render such delimitations inadequate, since a time delimitation is only valid when there is a review covering the previous period. Furthermore, a national event is only relevant if the integrative review aims to analyze a specific problem in the country where the event occurred. Therefore, it is recommended that time delimitations should be based on facts of international relevance, such as those published by global institutions such as the World Health Organization and other associations. It is therefore best to avoid arbitrary time delimitations⁽³⁾.

In the data extraction stage, the inadequate use of instruments stands out. Authors often use instruments that have been refined in dissertations or theses, but which are not aligned with the study theme in question. To avoid these inadequacies, it is suggested that authors should develop their own instruments (data extraction forms), adapting them to the specific needs of the review⁽³⁾. For instance, if the aim is to carry out an integrative review on nursing care for patients with a tracheostomy, the essential items to consider when extracting data include: authors, title, publication year, publication country, study objective, study delimitation, population/sample, nursing care mentioned, main results, limitations, and conclusion.

Furthermore, it is common to find inadequate use of tables summarizing the studies, which often cover several pages of the manuscript and fail to effectively fulfill the function of data comparison⁽³⁾. To address this problem, it is recommended that authors should organize the information extracted from the studies into more than one table, each covering less than one page, to ensure clarity and conciseness⁽⁷⁾. The authors can also provide more extensive tables as supplementary material, allowing keen readers to access the full content and make more detailed comparisons.

In integrative reviews, the lack of evaluation of the studies’ levels of strength of evidence and critical evaluation of methodological quality are also recurring inconsistencies. It should be noted that these concepts are distinct. The level of strength of evidence refers to the hierarchical position of a study within the pyramid of scientific evidence, varying according to methodological design⁽⁸⁾.

It is common to find researchers using pyramids in which systematic reviews are at the top, which can introduce significant bias since, by definition, integrative reviews only analyze primary studies^(1,3). Likewise, there is the use of pyramids that place randomized clinical trials at the top and qualitative studies at the bottom, even when the aim of the review is to analyze the meaning or experience of a given situation. For this reason, it is recommended that authors should use the hierarchy in three types (Intervention and/or Treatment or Diagnosis/Diagnostic Test; Prognosis and/or Prediction or Etiology; Meaning), with a maximum of seven levels of evidence each, as proposed by Melnyk and Fineout-Overholt⁽⁸⁾.

On the other hand, methodological quality refers to the critical evaluation of the studies' internal and external validity, considering factors such as the control of biases, the suitability of statistical methods, the generalizability of the results, and the work’s limitations⁽³⁾. Examples of instruments for assessing methodological quality include the Form for Critical Review of Qualitative Studies⁽⁹⁾ and Quantitative Studies⁽¹⁰⁾, both developed by the McMaster University Occupational Therapy Evidence-Based Practice Research Group. For mixed methods studies, the Mixed Methods Appraisal Tool (MMAT)⁽¹¹⁾ is used. Meanwhile, the Scale for Evaluating Articles with Heterogeneous Methodologies for Integrative Reviews⁽¹²⁾ can be applied to qualitative, quantitative, or mixed methods studies.

Errors related to the insertion of systematic review elements in integrative reviews are also pointed out, such as the use of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and the Peer Review of Electronic Search Strategies (PRESS) guidelines. Although these constructs can be useful in certain parts of an integrative review, they need to be adapted to the specific context of this type of review. In other words, the authors should clearly demonstrate the adaptations they have made, adding this information as supplementary material, including the fact that PRISMA goes beyond a simple flowchart.

In addition to the classic studies on how to conduct integrative reviews^(1-3), new guidelines have been developed to improve the quality of these reviews. One such example is the recent publication of INTEGRA [an acronym that represents the stages of the integrative review process: (I) idea of the study problem, (N) question and objective, (T) search tactic, (E) execution or use of the search, (G) level and quality control of the results, (R) filtered results, and (A) analysis and discussion], which aims to reduce subjectivity, increase standardization, and strengthen scientific evidence in integrative reviews, offering clear and updated guidelines for carrying out this type of study⁽¹²⁾. It should be emphasized that beyond the publication of the guideline, the adequate use of it by the authors who adopt it as a reference is more important, avoiding the recurrence of several of the errors previously discussed.

It is worth considering the range of languages used in integrative reviews, as some studies are limited to a single language, while others cover two. Although the selection of languages can be influenced by the authors’ knowledge and the resources available, it is important to note that approximately 95% of academic output is published in English, while Portuguese and Spanish combined comprise less than 1% of this output⁽⁶⁾. Therefore, in this technological age, authors need to use software, programs, or artificial intelligence for translation, in order to analyze abstracts and full texts, enabling a more comprehensive and complete review of the available literature⁽³⁾.

Finally, there is a lack of evidence synthesis in most published integrative reviews. Numerous researchers have not effectively contemplated what is expected of this type of study. To remedy this issue, it is recommended that, at the end of the discussion and in the conclusions of their texts, the authors should clearly highlight that: “This review found evidence on [...]” or “This review did not find evidence on [...]”^(1,3).

To retain rigor in the conduction of an integrative review, it is necessary to apply clear methodological criteria, standardize the stages, and ensure transparency throughout the entire evidence search and synthesis process. We believe that this debate can foster the improvement of more rigorous and consistent practices in the conduction of integrative reviews, strengthening both the reliability and relevance of the results obtained. We are grateful for the opportunity to discuss this topic and hope that these reflections can spark the continuous improvement of integrative reviews in the healthcare field.

REFERENCES

1. Mendes KDS, Silveira RCCP, Galvão CM. Revisão integrativa: método de pesquisa para a incorporação de evidências na saúde e na enfermagem. Texto contexto enferm. (Online). 2008 Oct; 17(4):758–64. Doi: 10.1590/S0104-07072008000400018

2. Souza MT, Silva MD, Carvalho R. Integrative review: what is it? How to do it? Einstein (São Paulo). 2010 Jan; 8(1):102-6. DOI: 10.1590/S1679-45082010RW1134

3. Toronto CE, Remington R. A step-by-step guide to conducting an integrative review. Berlim: Springer, 2020. Doi:10.1007/978-3-030-37504-1

4. Araújo WC. Recuperação da informação em saúde: construção, modelos e estratégias. ConCI. 2020 Jul 10;3(2):100-34. Doi: 10.33467/conci.v3i2.13447

5. Souza JC, Andrade JV, Paraizo-Horvath CM, Dázio EM. Perspectivas teóricas sobre estratégias de busca em revisões integrativas. Contrib. cienc. soc. 2023 Oct 6;16(10):19830-40. Doi: 10.55905/revconv.16n.10-070

6. Toledo IP, Savi MGM, Réus JC, Porporatti A, Canto GL. Busca nas bases de dados. In: Canto GL (org.). Revisões sistemáticas da literatura: guia prático. 1ed. Curitiba: Brazil Publishing; 2020. p. 40-49. Doi:10.31012/978-65-5016-353-2

7. American Psychological Association. Publication manual of the American Psychological Association. 7th ed. Washington, DC: American Psychological Association; 2020. Available from: https://apastyle.apa.org/

8. Melnyk BM, Fineout-Overholt E. Evidence-based practice in nursing & healthcare: A guide to best practice. 5ed. Philadelphia: Wolters Kluwer, 2023. Available from: https://shop.lww.com/Evidence-Based-Practice-in-Nursing---Healthcare/p/9781975185725

9. Letts L, Wilkins S, Law M, Stewart D, Bosch J, Westmorland M. Guidelines for critical review form: Qualitative studies (Version 2.0). McMaster university occupational therapy evidence-based practice research group. 2007 Sep 22:1-2. Available from: https://www.canchild.ca/system/tenon/assets/attachments/000/000/360/original/qualguide.pdf

10. Law M, Stewart D, Pollock N, Letts L, Bosch J, Westmorland M. Guidelines for critical review form: quantitative studies [Internet]. Hamilton: McMaster University Occupational Evidence-based Practice Research Group; 1998. Available from: https://canchild.ca/system/tenon/assets/attachments/000/000/366/original/quantguide.pdf

11. Pluye P, Robert E, Cargo M, Bartlett G, O'Cathain A, Griffiths F, Boardman F, Gagnon MP, Rousseau MC. Proposal: A mixed methods appraisal tool for systematic mixed studies reviews. ScienceOpen. 2011.

12. Valencia-Contrera M, Vilchez-Barboza V, Robazzi ML, Quintana-Zavala M, Castro-Bastidas J, Ardiles-Irarrazabal RA, et al. INTEGRA methodology for the development of integrative reviews: origins, guidelines, and recommendations. Data and Metadata [Internet]. 2024 Sep 2;3:401. Doi: 10.56294/dm2024.401

Support and Thanks:

This work was carried out with the support of the Coordination for the Improvement of Higher Education Personnel - Brazil (CAPES) - Financing Code 001.

Authorship criteria (author contributions)

Both authors contributed equally at all stages.

Declaration of conflict of interest

Nothing to declare

Scientífic Editor: Ítalo Arão Pereira Ribeiro. Orcid: https://orcid.org/0000-0003-0778-1447

Rev Enferm Atual In Derme 2024;98(4): e024389                       

Atribuição CCBY