An overview of the integrative research review

Prévia do material em texto

<p>As specialization increases and the volume of com-</p><p>pleted research expands, the scholarly individual is</p><p>constantly challenged to possess an accurate and current</p><p>understanding of information pertinent to his or her area</p><p>of practice and/or research. Integrative reviews assist in</p><p>maintaining a current knowledge base in a particular</p><p>research area.1 An integrative review of the literature is</p><p>defined as one in which “past research is summarized by</p><p>drawing overall conclusions from many studies.”2(p47)</p><p>Through the process of systematically analyzing and</p><p>summarizing the research literature, a well-prepared</p><p>integrative review can precisely represent the state of</p><p>the current research literature. The integrative literature</p><p>review can also be used to evaluate the strength of the</p><p>scientific evidence, identify gaps in current research,</p><p>identify the need for future research, build a bridge</p><p>between related areas of work, identify central issues in</p><p>an area, generate a research question, identify a theoret-</p><p>ical or conceptual framework, and explore which</p><p>research methods have been used successfully.1</p><p>Several terms are closely associated with the inte-</p><p>grative review including literature review, systematic</p><p>review, and meta-analysis. A literature review is “a</p><p>critical summary of research on a topic of interest,</p><p>often prepared to put a research problem in con-</p><p>text.”3(p722) The literature review is frequently found at</p><p>the beginning of scholarly research articles providing a</p><p>foundation for the proposed research questions and</p><p>methods.4 A systematic review is “carefully synthe-</p><p>sized research evidence designed to answer focused</p><p>clinical questions.”5(p1) Stevens notes, “systematic reviews</p><p>are also known as evidence summaries and integrative</p><p>reviews.”5(p1) Finally, a meta-analysis is “a technique</p><p>for quantitatively combining and thus integrating the</p><p>results of multiple studies on a given topic.”3(p723) In</p><p>addition, primary research should be differentiated</p><p>from the integrative review process. Table 1 delineates</p><p>the similarities and differences.</p><p>Generally, there are 4 questions that the reviewer</p><p>answers when a body of knowledge is evaluated using</p><p>the integrative review technique. (1) What is known?</p><p>(2) What is the quality of what is known? (3) What should</p><p>be known? and (4) What is the next step for research or</p><p>practice? The answers to these questions are discovered</p><p>through the integrative review process. The reviewer</p><p>evaluates the current state of knowledge, judging its</p><p>quality so that the future directions for research studies</p><p>in the area are clearer.</p><p>Definitions</p><p>The process of conducting an integrative literature</p><p>review should be approached with the same intensity</p><p>and scientific rigor used when conducting primary</p><p>research. Cooper1 conceptualizes the integrative review</p><p>as occurring in 5 stages: (1) problem formulation, (2)</p><p>data collection or literature search, (3) evaluation of</p><p>data, (4) data analysis, and (5) interpretation and pres-</p><p>entation of results.</p><p>To focus the integrative review, the reviewer must</p><p>begin by defining the purpose,7 the problem,1 and the</p><p>An overview of the integrative research review</p><p>The integrative literature review has many benefits to the scholarly reviewer,</p><p>including evaluating the strength of the scientific evidence, identifying gaps in</p><p>current research, identifying the need for future research, bridging between related</p><p>areas of work, identifying central issues in an area, generating a research question,</p><p>identifying a theoretical or conceptual framework, and exploring which research</p><p>methods have been used successfully. The 5-stage integrative review process</p><p>includes (1) problem formulation, (2) data collection or literature search, (3) eval-</p><p>uation of data, (4) data analysis, and (5) interpretation and presentation of results.</p><p>Maintaining scientific integrity while conducting an integrative research review</p><p>involves careful consideration to threats to validity. Strategies to overcome these</p><p>threats are reviewed. The integrative review methodology must involve detailed</p><p>and thoughtful work, the outcome of which can be a significant contribution to a</p><p>particular body of knowledge and, consequently, to practice and research.</p><p>(Progress in Transplantation. 2005;15:8-13)</p><p>Cynthia L. Russell, RN, PhD</p><p>University of Missouri-Columbia,</p><p>Columbia, Mo</p><p>To purchase electronic or print reprints,</p><p>contact:</p><p>The InnoVision Group</p><p>101 Columbia, Aliso Viejo, CA 92656</p><p>Phone (800) 809-2273 (ext 532) or</p><p>(949) 448-7370 (ext 532)</p><p>Fax (949) 362-2049</p><p>E-mail reprints@aacn.org</p><p>8 Progress in Transplantation, Vol 15, No. 1, March 2005</p><p>Russell3_05pgs.qxd 2/15/05 10:10 AM Page 8</p><p>research questions under study.2,7 The reviewer should</p><p>ask, “What are the concepts I want to study?”1(p5) The</p><p>answer to this question facilitates delineating the scope</p><p>of the review, which also may depend on the history of</p><p>the research in the area.2 If the research topic has a long</p><p>history and much research has been conducted in the</p><p>area, the reviewer must narrow the research question.</p><p>On the other hand, if the topic is new and little research</p><p>has been conducted, the research question may need to</p><p>be broadened so that an adequate amount of informa-</p><p>tion is located. For example, if the reviewer’s initial</p><p>research question is “What interventions are most effec-</p><p>tive in increasing treatment compliance in liver trans-</p><p>plant recipients?” but no intervention studies have been</p><p>conducted with this population, then the reviewer may</p><p>need to broaden the research question to, “What inter-</p><p>ventions are the most effective in increasing treatment</p><p>compliance in all transplant recipients?”</p><p>Problem Formulation</p><p>Cooper1 notes that the problem identification process</p><p>should include development of conceptual and opera-</p><p>tional definitions of variables to be examined. The con-</p><p>ceptual definition defines how the reviewer abstractly</p><p>conceives the issue under study. For example, the</p><p>reviewer interested in evaluating the literature regard-</p><p>ing noncompliance must decide whether the concept</p><p>of noncompliance will include all aspects of treat-</p><p>ments (ie, diet, exercise, smoking cessation, appoint-</p><p>ment keeping, medications) or just one aspect of</p><p>noncompliance (ie, medication taking). An example</p><p>of a classic conceptual definition of medication non-</p><p>compliance is the extent to which the patient’s behavior</p><p>coincides with clinical prescriptions.8 The operational</p><p>definition defines how the reviewer will measure the</p><p>concept. To facilitate the integrative review, definitions</p><p>of conceptual variables must be clearly and concisely</p><p>Problem formulation</p><p>Data collection or</p><p>literature search</p><p>Data evaluation</p><p>Data analysis</p><p>Interpretation and</p><p>presentation</p><p>Table 1 Primary research and the integrative review: similarities and differences1</p><p>Limited to published topics</p><p>Conceptual and operational</p><p>definitions may develop as</p><p>the review progresses</p><p>Studies may contain many different</p><p>operational definitions</p><p>Data collection involves systematic</p><p>search for all reports relevant to</p><p>the topic</p><p>Target population is all relevant</p><p>studies that pertain to the topic</p><p>Sample limited by samples in</p><p>primary research</p><p>Individual studies are examined for</p><p>the strength of relationships/</p><p>treatment effects; study may be</p><p>discarded or weighed differently</p><p>if methodological quality is poor</p><p>Statistical procedures not usually</p><p>applied to data, until recently</p><p>Interpretation and presentation</p><p>guidelines are available (the</p><p>American Psychological</p><p>Association’s Publication</p><p>Manual 6)</p><p>Selection of topic based on</p><p>researcher interests and</p><p>social issues</p><p>Conceptual and operational</p><p>definitions must be defined</p><p>before beginning the study</p><p>Use of 1 or 2 operational</p><p>definitions of the concept</p><p>Data collection involves asking</p><p>people questions and/or</p><p>observing behavior</p><p>Target population is usually</p><p>specific to a category of</p><p>individuals</p><p>Sample may be limited only</p><p>by researcher time and</p><p>resources</p><p>Individual data points are</p><p>examined; responses</p><p>may be discarded because of</p><p>participant issues</p><p>Statistical procedures</p><p>are</p><p>almost always applied</p><p>to data</p><p>Few guidelines available</p><p>for interpretation and</p><p>presentation</p><p>Both require definition</p><p>of variables and expla-</p><p>nation of</p><p>relationship(s)</p><p>between variables</p><p>Both involve identifying</p><p>target populations and</p><p>sampling frame</p><p>and evaluating the</p><p>differences</p><p>Both involve identifying</p><p>and interpreting</p><p>outliers</p><p>Both involve finding pat-</p><p>terns in data samples</p><p>and drawing conclu-</p><p>sions about target</p><p>populations</p><p>Both should involve inter-</p><p>pretation and dissemi-</p><p>nation of indings</p><p>Similarities</p><p>Primary research Integrative reviewStages</p><p>Differences</p><p>9Progress in Transplantation, Vol 15, No. 1, March 2005</p><p>The integrative research review</p><p>Russell3_05pgs.qxd 2/15/05 10:10 AM Page 9</p><p>delineated by the reviewer. For example, if the reviewer</p><p>decides to investigate only immunosuppressive medica-</p><p>tion noncompliance, an operational definition of this</p><p>concept must be formulated (ie, medication noncompli-</p><p>ance is defined as taking less than 80% of the prescribed</p><p>doses of immunosuppressant measured by an electronic</p><p>monitoring device). Unlike the primary research process,</p><p>the reviewer may need to modify the conceptual and</p><p>operational definitions if the targeted reports utilized</p><p>broader or narrower definitions.9 For example, if the</p><p>selected definition of medication noncompliance yields</p><p>only a few articles, then the reviewer must broaden his</p><p>or her operational definition. The revised definition</p><p>would be taking less than 80% of the prescribed doses</p><p>of immunosuppressive medications as measured by any</p><p>method not just an electronic monitoring device.</p><p>The reviewer should also delineate the relation-</p><p>ships between the variables under study. Using a the-</p><p>oretical framework to guide this early phase of the</p><p>review process will enhance the ability of the reviewer</p><p>to “fit” the results of the process into the body of devel-</p><p>oping nursing knowledge. For example, Bandura’s</p><p>Social Cognitive Theory10 may be used as the theoret-</p><p>ical framework for a medication compliance study. In</p><p>this case, self-efficacy and outcome expectancy beliefs</p><p>are 2 key concepts from this theory that the reviewer</p><p>may choose to examine in relationship to the health</p><p>behavior of medication compliance.</p><p>Threats to Validity</p><p>Maintaining scientific integrity while conducting</p><p>an integrative research review involves paying great</p><p>attention to threats to validity. Several of these threats</p><p>must be considered during this phase of the integrative</p><p>review. Cooper1 notes that the reviewer must not define</p><p>the operational definitions too narrowly because the</p><p>quality of the findings can be impaired when other def-</p><p>initions are not considered. On the other hand, Cooper</p><p>warns against defining the operational definitions too</p><p>broadly because this can lead to overlooking important</p><p>study details and incorrectly interpreting results. To</p><p>overcome these 2 threats to validity, Cooper suggests</p><p>that the reviewer use the broadest conceptual definition</p><p>possible and pay impeccable attention to the differences</p><p>in study methodologies. The reviewer must balance the</p><p>conceptual definition and methods review constantly</p><p>during this dynamic process.</p><p>Data Collection or Literature Search</p><p>The second stage in the integrative review process</p><p>is the data collection or literature search phase.9 Cooper1</p><p>notes that reports of past pertinent research are the data</p><p>for this review process. Identifying the target and acces-</p><p>sible population are 2 key steps in this stage. The target</p><p>population includes individual or groups the reviewer</p><p>hopes to represent in the integrative review. Results</p><p>could be generalized to this population. Examples of tar-</p><p>get population inclusion criteria are individuals within a</p><p>particular age range, with a particular diagnosis, who</p><p>have completed a particular treatment therapy. When</p><p>conducting an integrative review, there are 2 important</p><p>components of the target population: (1) all previous</p><p>published reports on a topic and (2) the population of</p><p>people within those reports that the reviewer is targeting.</p><p>For example, the target population for an integrative</p><p>review may be all studies of adult kidney transplant</p><p>recipients examining immunosuppressive medication</p><p>noncompliance.</p><p>In an integrative review, the accessible population</p><p>is all published reports related to a topic. The accessi-</p><p>ble population includes individuals or groups included</p><p>in the primary research that the reviewer is able to</p><p>Chisholm</p><p>et al,11</p><p>2000</p><p>Table 2 Sample data collection tool for an integrative review</p><p>Conceptual definition: Not clearly defined</p><p>Operational definitions: Compliance</p><p>measured by comparing refill records to</p><p>prescription from medical record.</p><p>Measured as (number of</p><p>immunosuppressive doses filled/</p><p>number of doses prescribed) x 100</p><p>for each month.</p><p>Included 2 medications:</p><p>cyclosporine and tacrolimus.</p><p>Compliance defined as >/= 80%.</p><p>Noncompliance defined as < 80%.</p><p>Serum drug levels used to validate</p><p>compliance assessments. Targets defined</p><p>as target (cyclosporine at least 250 ng/mL,</p><p>tacrolimus at least 8 ng/mL)</p><p>versus subtarget.</p><p>N = 18</p><p>Gender, 15 men,</p><p>3 women</p><p>Transplant, 12</p><p>cadaveric, 6 living</p><p>donor</p><p>Inclusion criteria:</p><p>• 18 years of age or</p><p>older</p><p>• First transplant</p><p>• Participated in free</p><p>medication program</p><p>Included all kidney</p><p>transplant recipients</p><p>receiving free</p><p>medications for</p><p>12 months after</p><p>transplantation</p><p>Design: Descriptive</p><p>No conceptual</p><p>framework</p><p>presented</p><p>Examined</p><p>medication</p><p>compliance</p><p>rates when</p><p>medications</p><p>were free to</p><p>patients</p><p>Conceptual</p><p>framework Sample</p><p>Author,</p><p>year</p><p>Group assignment,</p><p>methods/design Measurement of variables</p><p>10 Progress in Transplantation, Vol 15, No. 1, March 2005</p><p>Russell</p><p>Russell3_05pgs.qxd 2/15/05 10:10 AM Page 10</p><p>obtain information about. Selected databases, years of</p><p>publication, and subject headings are examples of</p><p>accessible population inclusion criteria. During this</p><p>stage, Ganong7 suggests establishing inclusion criteria</p><p>for the studies. Given that writing an integrative</p><p>review involves a circular approach, Ganong believes</p><p>that inclusion criteria should be tentative. Substantive</p><p>or methodologic changes should be made if suggested</p><p>by the findings from the reviewed literature.</p><p>Data collection should involve several strategies.</p><p>Cooper9 delineates informal, primary, and secondary</p><p>channels. Informal channels include (1) personal research</p><p>findings; (2) the invisible college, which includes</p><p>reviewers conducting similar research sharing infor-</p><p>mation with each other; (3) sharing between students</p><p>and professors; and (4) attendance at professional meet-</p><p>ings and conferences. Primary channels include review</p><p>of journals, and the ancestry approach that involves</p><p>finding research articles by examining the reference</p><p>lists of other articles. Secondary channels include</p><p>research bibliographies and government documents</p><p>(eg, Government Printing Office works), research reg-</p><p>isters (eg, Computer Retrieval of Information on Sci-</p><p>entific Projects), and reference databases (eg, CINAHL,</p><p>MEDLINE). All available methods should be used to</p><p>obtain information for the review.</p><p>Finally, a data collection tool should be developed.</p><p>Characterizing the data to be collected is an important</p><p>part of the integrative review.7 Common study charac-</p><p>teristics include sample size, group assignment meth-</p><p>ods, measurement of variables, attrition, data analysis</p><p>methods, conceptual framework used, and signifi-</p><p>cance of findings. Table 2 contains an example of a</p><p>completed data collection tool. Additional examples</p><p>of data collection tools are available.1 Ganong7 suggests</p><p>using tables to organize data in a clear and concise for-</p><p>mat. Using a concise format for data collection greatly</p><p>enhances the reviewer’s ability to ascertain consistent</p><p>information from all information sources.</p><p>Threats to Validity</p><p>The first threat to validity in the data collection stage</p><p>is inadequate sampling.9 The reviewer considers how the</p><p>collected studies could be different from all studies.</p><p>Realistically, it is difficult for the reviewer to obtain all</p><p>possible studies for review. The reviewer must identify</p><p>which studies were not able to be included, describe why</p><p>they were not included, and then discuss how they might</p><p>have shown different results from those selected.</p><p>The second threat to validity is discrepancy</p><p>between collected studies and the target population.</p><p>To address the second threat to validity, the reviewer</p><p>must determine how the elements contained in the col-</p><p>lected studies might be different from the target popu-</p><p>lation. Cooper1 notes that comprehensive integrative</p><p>reviews contain articles conducted at different times;</p><p>in different places; with different samples; of different</p><p>ages, genders, and race, while using different methodol-</p><p>ogies. This approach increases generalizability because</p><p>the target population is often better represented in the</p><p>reviewed articles.</p><p>Cooper1 suggests 4 strategies to enhance validity</p><p>in this stage including: (1) conducting an exhaustive</p><p>data collection strategy; (2) clearly delineating in the</p><p>review information about data collection such as</p><p>sources, years, and keywords; (3) presenting all selec-</p><p>tion biases; and (4) summarizing demographics of the</p><p>subjects included in the samples.</p><p>Data Evaluation</p><p>The third stage in the integrative review process</p><p>is the data evaluation phase. During this phase, the</p><p>Descriptive</p><p>statistics,</p><p>χ2 analysis,</p><p>Kaplan-Meier</p><p>Attrition rates not found.</p><p>Mean age: 48.1 years ± 9.3.</p><p>Mean compliance rates:</p><p>cyclosporine 88.9% (range</p><p>75-100%); tacrolimus 99.9%</p><p>(range 92-100%); 48% of</p><p>noncompliant and 14% of</p><p>compliant had subtarget serum</p><p>concentrations (χ2= 12.9;</p><p>P< .001); Compliance at 5</p><p>months was 95%, at 7 months</p><p>75%, at 12 months 48%;</p><p>Mean time until first noncompliant</p><p>month for all patients was 9.8</p><p>(CI 8.6-11.0)</p><p>Those receiving free medications were</p><p>generally compliant during the first year.</p><p>Over time compliance decreased.</p><p>Costs do not completely explain noncompliance.</p><p>Mean compliance was >80% throughout the</p><p>study.</p><p>Comparison of compliance rates is difficult</p><p>because of different measures.</p><p>If compliance rate is >80%, likely to have</p><p>target serum drug levels.</p><p>Impossible to know if</p><p>patients actually took</p><p>medications; could</p><p>have received them</p><p>from somewhere</p><p>else.</p><p>Data analysis</p><p>methods Attrition/results Significance of findings Limitations</p><p>11Progress in Transplantation, Vol 15, No. 1, March 2005</p><p>The integrative research review</p><p>Russell3_05pgs.qxd 2/15/05 10:10 AM Page 11</p><p>reviewer critically judges whether the data element or</p><p>result is worthy of remaining in the study data set.1 This</p><p>can be decided a priori, in which case the decision to</p><p>include or exclude certain articles is made before data</p><p>collection, or a posteriori, in which case the decision is</p><p>made to include all articles but less weight is then given</p><p>to the poorer (ie, less scientifically rigorous) articles.1</p><p>The reviewer evaluates the data for unreliable values,</p><p>which might include whether the findings from one</p><p>study are too different from the other studies to be con-</p><p>sidered, and whether data recording errors exist. The</p><p>reviewer also must evaluate the reliability of each</p><p>study’s findings, both in and of itself and in comparison</p><p>with all of the other studies included in the review.1</p><p>Threats to Validity</p><p>One threat to validity in the data evaluation</p><p>phase is the tendency to positively evaluate research</p><p>that is congruent with the reviewer’s own beliefs and</p><p>negatively evaluate those studies that are not. The</p><p>reviewer must also evaluate each study’s methodology</p><p>to determine whether the findings are valid. Cooper1</p><p>offers several suggestions for enhancing objectivity</p><p>in methodology evaluation. One approach is to score</p><p>each study’s methods using a list of threats to validity.</p><p>Methodologic evaluation lists can also be used. Addi-</p><p>tional information about rating the quality of studies is</p><p>available.12-15 Unfortunately, there is no universally</p><p>accepted model for use.</p><p>Data Analysis</p><p>Analysis and interpretation are included in the</p><p>fourth stage of the integrative review process. Cooper1(p104)</p><p>defines this stage as “reducing the separate data points</p><p>collected by the inquirer into a unified statement about</p><p>the research problem.” Depending on the goal of the</p><p>integrative review, analysis may involve statistical</p><p>tests. For example, if a review of the development of</p><p>the concept medication noncompliance is the goal of</p><p>an integrative review, then statistical tests would not</p><p>be required. Historically, there has been sparse use of</p><p>statistical techniques for analysis of integrative review</p><p>data.1 Meta-analysis, defined as application of statisti-</p><p>cal procedures to a group of research report findings,</p><p>is an appropriate statistical technique which could be</p><p>used.1 Cooper1 notes that with recent increases in</p><p>research results, quantitative reviewing techniques</p><p>have been utilized more frequently. For example, if a</p><p>review of the predictors of medication noncompliance</p><p>is the goal of an integrative review, then a meta-analysis</p><p>would be appropriate.</p><p>Threats to Validity</p><p>Threats to the analysis and interpretation stage are</p><p>several. The first threat involves reviewers not follow-</p><p>ing appropriate rules of inference.1 The assumptions</p><p>for statistical tests may not always be clearly delin-</p><p>eated. If additional secondary analyses are under-</p><p>taken, the reviewer may have to assume that the</p><p>primary researcher interpreted rules of inference cor-</p><p>rectly. The second threat is inferring causality that is</p><p>inappropriate when examining research review data.</p><p>Cooper1(p155) notes, “study-based evidence is capable of</p><p>establishing causal precedence among variables while</p><p>review-based evidence is always purely associa-</p><p>tional.” Cooper suggests the following methods for</p><p>reviewers to preserve validity: (1) assumptions should</p><p>be made clear by the reviewer when discussing results</p><p>and inferences, (2) important interpretation rules</p><p>should be identified, and (3) evidence that is single</p><p>study-based should be clearly delineated from that</p><p>which is review-based.</p><p>Interpretation and Presentation</p><p>The interpretation and presentation phase is the</p><p>fifth and final stage of the integrative review as iden-</p><p>tified by Cooper.1 Dissemination of the findings from</p><p>an integrative literature review is paramount to the</p><p>development of our knowledge base. Cooper notes</p><p>that there is no one accepted template for reporting</p><p>integrative research reviews. Several journals do offer</p><p>guidelines, but they are inconsistent. Cooper suggests</p><p>a format for integrative review reporting that is simi-</p><p>lar to primary research, which includes introduction,</p><p>methods, results, and discussion sections.</p><p>Threats to Validity</p><p>Several threats to validity exist when writing the</p><p>integrative review. First, the reviewer might omit</p><p>important details and information about how the inte-</p><p>grative review was conducted.1 This omission could</p><p>affect the availability of information about the rela-</p><p>tionships between variables under study. Second, the</p><p>ability to reproduce review findings is impaired if</p><p>details of the study methods and moderators of rela-</p><p>tionships are not stated. Suggestions to reduce these</p><p>threats include giving much attention to all possible</p><p>details of the report. In reporting integrative reviews</p><p>the reviewer must be so explicit that another reviewer</p><p>could follow the same strategy without difficulty.</p><p>Summary</p><p>In summary, the integrative literature review has</p><p>many benefits to the scholarly reviewer including eval-</p><p>uating the strength of the scientific evidence, identify-</p><p>ing gaps in current research, identifying the need for</p><p>future research, bridging between related areas of</p><p>work, identifying central issues in an area, generating</p><p>a research question, identifying a theoretical or con-</p><p>ceptual framework, and exploring which research</p><p>methods have been used successfully.1 This review</p><p>methodology must involve detailed and thoughtful</p><p>12 Progress in Transplantation, Vol 15, No. 1, March 2005</p><p>Russell</p><p>Russell3_05pgs.qxd 2/15/05 10:10 AM Page 12</p><p>13Progress in Transplantation, Vol 15, No. 1, March</p><p>2005</p><p>The integrative research review</p><p>work, the outcome of which can be a significant con-</p><p>tribution to a particular body of knowledge and, con-</p><p>sequently, to practice and research.</p><p>References</p><p>1. Cooper HM. Synthesizing Research: A Guide for Literature</p><p>Reviews. 3rd ed. Thousand Oaks, Calif: Sage Publications; 1998.</p><p>2. Broome ME. Integrative literature reviews for the development</p><p>of concepts. In: Rogers B, Knafl K, eds. Concept Development</p><p>in Nursing. Philadelphia, Pa: WB Saunders; 2000:231-250.</p><p>3. Polit DF, Beck CT. Nursing Research: Principles and Methods.</p><p>7th ed. Philadelphia, Pa: JB Lippincott Co; 2004.</p><p>4. Beyea SC, Nicoll LH. Writing an integrative review. AORN.</p><p>1998;67:877-880.</p><p>5. Stevens K. Systematic reviews: the heart of evidence-based</p><p>practice. AACN Clin Issues. 2001;12:529-538.</p><p>6. American Psychological Association. American Psychological</p><p>Association Publication Manual. 5th ed. Washington, DC: The</p><p>American Psychological Association; 2001.</p><p>7. Ganong LH. Integrative reviews of nursing research. Res</p><p>Nurs Health. 1987;10:1-11.</p><p>8. Sackett DL, Haynes RB. Compliance With Therapeutic Regi-</p><p>mens. Baltimore, Md: Johns Hopkins University Press; 1976.</p><p>9. Cooper HM. Integrating Research: A Guide for Literature</p><p>Reviews. 2nd ed. Newbury Park, Calif: Sage; 1989.</p><p>10. Bandura A. Social Foundations of Thought and Action: A Social</p><p>Cognitive Theory. Englewood Cliffs, NJ: Prentice-Hall; 1986.</p><p>11. Chisholm MA, Vollenweider LJ, Mulloy LL, et al. Renal</p><p>transplant patient compliance with free immunosuppressive</p><p>medications. Transplantation. 2000;70:1240-1244.</p><p>12. Downs SH, Black N. The feasibility of creating a checklist</p><p>for the assessment of the methodological quality both of ran-</p><p>domized and non-randomized studies of health care interven-</p><p>tions. J Epidemiol Comm Health. 1998;52:377-384.</p><p>13. Evidence Report/Technology Assessment: No. 47. Systems</p><p>for Rating the Quality of Individual Articles. AHCPR Web</p><p>site. Available at: http://www.ahcpr.gov/clinic/epcsums/</p><p>strengthsum.htm#Rating. Accessed April 24, 2004.</p><p>14. Evidence Report/Technology Assessment: No. 47. Systems</p><p>for Rating the Quality of Individual Articles: Summary.</p><p>AHCPR Web site. Available at: http://www.ahcpr.gov/clinic</p><p>/epcsums/strength2.htm. Accessed April 24, 2004.</p><p>15. Estabrooks C, Goes V, Thiel E, Pinfold P, Sawka C, Williams</p><p>I. Decision aids: are they worth it? A systematic review. J</p><p>Health Serv Res Policy. 2001;6:170-182.</p><p>Russell3_05pgs.qxd 2/15/05 10:10 AM Page 13</p>