ABSTRACT

Objective: To translate the Understanding COPD (UCOPD) questionnaire into Portuguese, adapt it for use in Brazil, and assess its reliability. Methods: The UCOPD questionnaire consists of two sections, designated section A and section B. Section A comprises 18 items divided into three domains: "About COPD", "Managing Symptoms of COPD", and "Accessing Help and Support". Section B includes five questions regarding patient satisfaction with the educational component of pulmonary rehabilitation programs. The UCOPD questionnaire was applied twice on the same day by two different raters (with a 10-min interval between applications) and once again 15-20 days later. The Wilcoxon test was used in order to compare the scores among applications. Reliability was assessed by the intraclass correlation coefficient and Bland-Altman plots. Results: The study sample consisted of 50 COPD patients (35 men; mean age, 65.3 ± 7.91 years; mean FEV1, 36.4 ± 16.2% of the predicted value). Inter-rater intraclass correlation coefficients for section A total scores and domain scores ranged from moderate to high. Section A scores and domain scores had no significant differences regarding test-retest reliability (p < 0.05). The test-retest and inter-rater Cronbach's alpha coefficients for section A total scores were 0.93 and 0.86, respectively (p < 0.001). There were no floor or ceiling effects. Conclusions: The Brazilian Portuguese version of the UCOPD questionnaire is reliable.

Keywords: Pulmonary disease, chronic obstructive; reproducibility of results; Health knowledge, attitudes, practice.

RESUMO

Objetivo: Traduzir o questionário Understanding COPD (UCOPD) para o português, adaptá-lo para uso no Brasil e avaliar sua confiabilidade. Métodos: O questionário UCOPD é dividido em duas seções, denominadas seção A e seção B. A seção A compreende 18 itens, divididos em três domínios: "Sobre a DPOC", "Manejo dos Sintomas da DPOC" e "Acesso a Ajuda e Suporte". A seção B inclui cinco itens referentes à satisfação do paciente com o componente educacional dos programas de reabilitação pulmonar. O questionário UCOPD foi aplicado duas vezes no mesmo dia por dois observadores diferentes (com 10 min de intervalo entre uma aplicação e outra) e mais uma vez 15-20 dias depois. O teste de Wilcoxon foi usado para comparar a pontuação obtida em cada aplicação. A confiabilidade foi avaliada por meio do coeficiente de correlação intraclasse e de gráficos de dispersão de Bland-Altman. Resultados: A amostra foi composta por 50 pacientes com DPOC (35 homens; média de idade = 65,3 ± 7,91 anos; média do VEF1 = 36,4 ± 16,2% do valor predito). Os coeficientes de correlação intraclasse referentes à análise interobservador da pontuação total da seção A e seus domínios variaram de moderados a elevados. A pontuação obtida na seção A e em seus domínios não apresentou diferenças significativas quanto à confiabilidade teste-reteste (p < 0,05). Os coeficientes alfa de Cronbach de teste-reteste e interobservador referentes à pontuação total da seção A foram de 0,93 e 0,86, respectivamente (p < 0,001). Não foram observados efeitos de piso ou teto. Conclusões: A versão em português do Brasil do questionário UCOPD é confiável.

Palavras-chave: Doença pulmonar obstrutiva crônica; Reprodutibilidade de testes; Conhecimentos, atitudes e prática em saúde.

INTRODUCTION

Education is one of the key components of pulmonary rehabilitation programs (PRPs) for patients with COPD. In recent years, it has received greater attention because it can teach patients how to cope with their disease and because it can increase the likelihood of their adopting self-management strategies.(1)

Education involves activities that encourage patients and their families to learn more about and, consequently, gain a deeper understanding of the disease, thus improving patient self-efficacy.(1,2) Education plays an important role in promoting behavioral changes,(2,3) which are necessary because patients may not actively engage in appropriate behaviors which could improve their health outcomes.(4) This might be due to a lack of understanding of the importance of appropriate behavior or to a lack of disease-related self-efficacy. Educational interventions can change these outcomes and have proved to be effective in patients with COPD, being associated with improvement in self-management skills,(2) i.e., improved medication use, increased ability to manage disease exacerbations, and increased ability to achieve disease management goals.(5) However, despite its importance, education is seldom evaluated in PRPs because there are only a few instruments available for this purpose. The Understanding COPD (UCOPD) questionnaire,(4) the Bristol COPD Knowledge Questionnaire (BCKQ),(6) the Lung Information Needs Questionnaire (LINQ),(7) and the Mount Sinai Hospital Questionnaire(8) have been developed to assess patient knowledge of COPD. The UCOPD questionnaire stands out because, in addition to evaluating patient understanding of COPD, it assesses self-efficacy, use of self-management skills, and patient satisfaction with a given PRP.(4) However, there is currently no Brazilian Portuguese version of the UCOPD questionnaire. Therefore, given the importance of an instrument that can evaluate the effect of the educational component of PRPs, the objective of the present study was to translate the UCOPD questionnaire into Brazilian Portuguese and determine the reliability of the Brazilian Portuguese version of the questionnaire.

METHODS

Fifty patients with COPD referred to the Núcleo de Assistência, Ensino e Pesquisa em Reabilitação Pulmonar of the Universidade do Estado de Santa Catarina, located in the city of Florianópolis, Brazil, were included in the study. The inclusion criteria were as follows: having a clinical diagnosis of COPD confirmed by spirometry in accordance with the Global Initiative for Chronic Obstructive Lung Disease (GOLD) criteria(9) and self-reporting the ability to read in Brazilian Portuguese.

Patients unable to understand the questionnaire or to follow instructions were excluded (a Mini-Mental State Examination score of 18/19 for patients without previous formal education and a score of 24/25 for patients with previous formal education).(10) In addition, patients with any other severe or limiting respiratory or nonrespiratory disease were excluded. The local research ethics committee approved the study (CAAE protocol no. 11603112.1.0000.0118), and all participants gave written informed consent. The relevant measurement properties of the UCOPD questionnaire were evaluated in accordance with the recommendations of the Consensus-based Standards for the Selection of Health Status Measurement Instruments.(11) The authors of the original version authorized the cross-cultural adaptation of the UCOPD questionnaire.

The cross-cultural adaptation protocol was as recommended by Guillemin et al.(12) First, the English version of the UCOPD questionnaire(4) was translated into Brazilian Portuguese by two independent bilingual translators who are fluent in English and native speakers of Brazilian Portuguese, one of whom had no specific knowledge of health. A summary of the translations was made by a translation review committee, comprising the first author of the original questionnaire, the translators, and health professionals. That version was subsequently back-translated by a health professional who is a native speaker of English and fluent in Portuguese. The back-translator had previously had no contact with the questionnaire.

That first version was applied to 8 patients with COPD in order to identify uncertainties and difficulties regarding the text. Afterwards, issues raised by those patients were discussed by the review committee, and a consensus was reached. The final version of the instrument did not require cross-cultural adaptations or changes in the original structure. Although the name of the questionnaire was translated into Portuguese, a decision was made to keep the original, English-language abbreviation of the name (i.e., UCOPD) in the Brazilian Portuguese version of the questionnaire so as to facilitate recognition of the instrument. Appendix 1 (Erro! A referência de hiperlink não é válida.) shows the final version of the translated questionnaire.

The Brazilian Portuguese version of the UCOPD questionnaire was tested for inter-rater and test-retest reliability. On the first day, the questionnaire was applied twice by two raters (R1 and R2) in the following order: first by R1 and 10 min later by R2. After 15-20 days, it was reapplied by R2.(13)

The UCOPD questionnaire(4) consists of two sections: section A has 18 items in three domains: "About COPD", "Managing Symptoms of COPD", and "Accessing Help and Support"; section B has five questions regarding patient satisfaction with the educational component of the PRP. The answers to each question are indicated on a ten-centimeter visual analog scale with numerical intervals per centimeter.

The scores for the domains and the total score for section A range from 0% to 100%. The higher the score, the better the understanding, self-efficacy, medication use, and satisfaction. To calculate the scores for each domain, the scores for the individual questions in the domain are added up, divided by the maximum score for the domain, and multiplied by 100. The maximum scores for the domains, as well as total scores for sections A and B, are as follows: About COPD domain (questions 1-7), 70; Managing Symptoms of COPD domain (questions 8-14), 70; Accessing Help and Support domain (questions 15-18), 40; section A total score (questions 1-18), 180; and section B total score (questions 1-5), 50.(4)

For sample characterization, participants underwent spirometry (EasyOne® spirometer; ndd Medical Technologies, Zurich, Switzerland) in accordance with the American Thoracic Society/European Respiratory Society standards.(14) The predicted values were obtained from the equations proposed by Pereira et al.(15) The spirometric and multidimensional GOLD classifications(9) were used in order to stratify the severity of COPD. The Brazilian Portuguese versions of the Saint George's Respiratory Questionnaire (SGRQ)(16) and the COPD Assessment Test (CAT)(17) were applied to all participants, who were also evaluated regarding their physical activity in daily life (PADL) for 12 h on two consecutive week days.(18)

Data are presented as mean ± standard deviation and 95% confidence interval. We used the Wilcoxon test to compare the scores of the UCOPD questionnaire between the applications. The mixed two-way, single-measure intraclass correlation coefficient (ICC) and respective 95% CIs were used in order to analyze the reliability of UCOPD questionnaire, whereas the Cronbach's alpha coefficient was used to analyze the internal consistency of the questionnaire. The classification used for the ICC was as follows: low reproducibility, ICC < 0.40; moderate reproducibility, ICC ≤ 0.75; and high reproducibility, ICC > 0.75.(19)

Bland-Altman plots were used in order to represent the agreement between the UCOPD questionnaire scores, whereas the Spearman's correlation coefficient was used in order to assess the correlation of UCOPD questionnaire scores with SGRQ and CAT scores, as well as with the level of PADL. The standard error of measurement and the minimum detectable difference (MDD) were calculated as described by Terwee et al.(20) For the analysis of the floor and ceiling effects, the proportions of occurrence of the minimum (0%) and maximum (100%) scores for section A of the UCOPD questionnaire were used.(20) The significance level was 5%. Data analysis was performed with the IBM SPSS Statistics software package, version 20.0 (IBM Corporation, Armonk, NY, USA), and graphics were created using the GraphPad Prism program, version 5.0 (GraphPad Inc., San Diego, CA, USA). In order to estimate the sample size, in accordance with the Consensus-based Standards for the Selection of Health Status Measurement Instruments(21) recommendation, 50 patients were selected (good sample size). Sample size calculation was also based on an expected ICC of 0.50 for moderate reliability,(19) α = 0.05, and β = 0.10, yielding a sample size of 38 patients.(22)

RESULTS

Fifty-five patients with COPD were enrolled in the study. Of those, 2 were excluded for not completing the protocol and 3 because of disease exacerbation during the protocol. Therefore, 50 patients completed the protocol (35 males). Table 1 presents the characteristics of the patients.
 



The level of education varied among the patients: 7 (14%) had completed college; 4 (8%) had not completed college; 17 (34%) had completed high school; 2 (4%) had not completed high school; 4 (8%) had completed elementary school; 15 (30%) had not completed elementary school; and 1 (2%) had never attended school.

There were no differences in test-retest reliability scores for section A total score and its domains. There were significant differences in About COPD domain scores between R1 and R2 (Table 2). Table 2 shows the scores for section A and its domains, as well as test-retest and inter-rater ICCs, which ranged from satisfactory to excellent. None of the patients scored 0% or 100%.
 



In the agreement analysis, inter-rater Cronbach's alpha coefficients for section A total score and its domains (About COPD, Managing Symptoms of COPD, and Accessing Help and Support) were 0.93, 0.94, 0.83, and 0.94, respectively (p < 0.001 for all).

In 11 of the 18 items in section A, ICCs were higher than 0.75 (p < 0.001). Items 3, 7, 8, 9, 13, 14, and 15 showed satisfactory reproducibility (an ICC of 0.54-0.68; p < 0.001). Of those items, 4 belonged to the Managing Symptoms of COPD domain.

With regard to test-retest reliability, the Cronbach's alpha coefficient for section A total score was 0.86 (p < 0.001), and those for About COPD, Managing Symptoms of COPD, and Accessing Help and Support domain scores were 0.83, 0.76, and 0.87, respectively (p < 0.001). The ICCs were higher than 0.75 in 8 of the 18 items (p < 0.001). However, ICCs ranged from 0.43 to 0.70 for items 3, 4, 7, 8, 9, 10, 12, 14, 15, and 17. Five of the 7 items in the Managing Symptoms of COPD domain had ICCs lower than 0.75. The standard error of measurement was 6, and the MDD was 16.6.

Figure 1 shows the agreement between R1 and R2. There was a wide variability among UCOPD questionnaire applications, which was more pronounced in the test-retest analysis (Figure 2).
 

 




UCOPD questionnaire domain scores correlated weakly with CAT and SGRQ scores, as well as with the level of PADL. UCOPD questionnaire section A scores correlated with the total SGRQ score (r = −0.38; p = 0.007) and SGRQ "impacts" domain scores (r = −0.46; p = 0.001), as well as with sitting (r = −0.33; p = 0.024), standing (r = 0.33; p = 0.023), and walking times (r = 0.30; p = 0.04). UCOPD questionnaire About COPD domain scores correlated with the total SGRQ score (r = −0.30; p = 0.033) and SGRQ impacts domain scores (r = −0.35; p = 0.014), as well as with sitting time (r = −0.32; p = 0.027). UCOPD questionnaire Managing Symptoms of COPD domain scores correlated with SGRQ impacts domain scores (r = −0.35; p = 0.014), whereas UCOPD questionnaire Accessing Help and Support domain scores correlated with the CAT score (r = −0.30; p = 0.042), the total SGRQ score (r = −0.42; p = 0.003), SGRQ "activity" domain scores (r = −0.33; p = 0.019), SGRQ impacts domain scores (r = −0.48; p = 0.013), sitting time (r = −0.38; p = 0.004), standing time (r = 0.42; p = 0.003), and walking time (r = 0.35; p = 0.013), as well as with a level of PADL ≥ 3 metabolic equivalents (r = 0.33; p = 0.021). No correlations were found between FEV1 and the total UCOPD questionnaire score or between FEV1 and UCOPD questionnaire domain scores (values of p = 0.24-0.88).

DISCUSSION

The major finding of the present study is that the UCOPD questionnaire is reliable and able to reflect quality of life, health status, and PADL in patients with COPD in Brazil. The test-retest reliability and the inter-rater reliability of the total score for section A of the UCOPD questionnaire were excellent. The reliability of the domains ranged from satisfactory to excellent, the ICCs being lowest for the Managing Symptoms of COPD domain and the test-retest reliability analysis.

In the study of development and validation of the UCOPD questionnaire,(4) the authors found excellent reliability of the domains and section A total score. Interestingly, the Managing Symptoms of COPD domain had the highest ICC among the UCOPD questionnaire domains (ICC = 0.92; 95% CI: 0.81-0.97). Some factors can explain why the results of the present study differed from the aforementioned results. O'Neill et al.(4) included only patients who had a good understanding of written English, and the questionnaire was self-administered. Given that most of the elderly individuals in Brazil have low levels of education,(23) the UCOPD questionnaire was completed by interviewing the participants. Although this is permitted according to the UCOPD questionnaire application instructions, the low education level of the majority of the study participants (32% of whom had not completed elementary school) could have affected patient understanding of the Managing Symptoms of COPD domain items and, consequently, contributed to reducing its reliability. However, it was decided not to exclude those patients from the sample. Had they been excluded, the sample would no longer have been representative of the Brazilian population, the external validity of the instrument therefore being compromised.

The reliability of section B of the UCOPD questionnaire was not analyzed in the present study or in the original study.(4) Section B of the UCOPD questionnaire has been reported to have high internal consistency and concordance.(4) In the present study, section B was administered to 7 patients, all of whom had completed a PRP, and the answers were the same for all items. It is worth mentioning that section B is aimed at evaluating patient satisfaction with the education sessions; it is not part of the evaluation of patient knowledge and self-efficacy.

Given the lack of a previously validated instrument to assess knowledge and self-efficacy in COPD patients in Brazil, it was impossible to test the concurrent validity of the UCOPD questionnaire. A correlation of 0.303 (p = 0.002) has been obtained between the English-language version of the UCOPD questionnaire and the BCKQ.(4) The association with other outcomes, however, has yet to be tested. Despite weak correlations, it appears that patients with limited knowledge of COPD have impaired quality of life, increased sitting time, and reduced time spent in active postures (walking and standing). It appears that the more patients know about their disease and the more confident they feel about how much they know about it, the more active they become and the better their quality of life is. However, this has yet to be confirmed. Nevertheless, this reinforces the hypothesis that increasing patient knowledge and self-efficacy is an important strategy to promote long-term adherence to active and healthy behaviors.(1,24,25) In addition, this suggests that the UCOPD questionnaire is capable of reflecting other important outcomes, reinforcing its validity.

Although the educational component of PRPs is currently focused on promoting knowledge, self-care, and self-management, as well on increasing self-efficacy, there are only a few instruments that can be used in order to evaluate some or all of these outcomes: the UCOPD questionnaire, the BCKQ,(6) and the LINQ.(7) These instruments were developed in England and are reliable and valid; they can be used in order to assess patient response to educational programs(6) and PRPs.(26) However, to the best of our knowledge, there are currently no data regarding the translation and cross-cultural adaptation of these instruments in Brazil. In addition, neither the BCKQ nor the LINQ address patient self-efficacy; they are used in order to assess patient knowledge. In contrast, in addition to assessing patient knowledge, the UCOPD questionnaire assesses patient self-efficacy and patient satisfaction with the educational component of PRPs.(4)

Self-efficacy is now regarded as one of the major outcomes to be assessed in patients participating in a PRP, because it appears to be associated with adoption and maintenance of active and healthy behaviors.(1) It is defined as the confidence that an individual has in their ability to deal with a specific task,(3) such as successfully managing their disease. Unlike the BCKQ or the LINQ, the UCOPD questionnaire addresses how confident patients are that they know what COPD is; that they can recognize an exacerbation; that they know when to seek medical attention; that they know how to use their COPD medication; and that they know how to exercise, among other questions. Therefore, it is an interesting tool that can aid in the identification and development of strategies for patients who, despite having good knowledge of their disease, are not confident about applying what they know or being able to manage their health, a lack of confidence that might result in nonadherence to treatment.

Despite cultural differences between Brazil and Northern Ireland, the Brazilian Portuguese version of the UCOPD questionnaire required no major adjustments. The final version of the questionnaire is the same as the first translated version, and its back-translation was approved by the first author of the original questionnaire. The Brazilian Portuguese version of the UCOPD questionnaire was found to be reliable, having no floor or ceiling effect. This finding is important because a floor/ceiling effect could compromise the discriminatory ability of the questionnaire or the detection of change over time/after an intervention.(20)

One potential limitation of the present study is the time elapsed between the two applications of the questionnaire (i.e., 10 min for inter-rater reliability analysis and 15-20 days for test-retest reliability analysis). Although one of the assumptions of the reliability test procedure is that the interval between applications should be short but long enough to avoid respondent memory bias,(27) we found no specific recommendations regarding how long that interval should be. Although the results of the present study were satisfactory, our interval between applications was longer than that in the original study. Therefore, it is possible that patient clinical status changed during that time interval, patient responses being influenced by that. Another possible limitation is that we did not test the responsiveness of the Brazilian Portuguese version of the UCOPD questionnaire to a PRP or other interventions. In addition, there are currently no data such as cut-off points and minimal clinically important difference to assist in interpreting the results of the UCOPD questionnaire. However, this was outside the scope of the present study, further studies therefore being required. Our finding of an MDD of 16.6 reflects the lowest intrapersonal variation; therefore, changes above this point showing a value of p < 0.05 can be considered "real".(20)

To the best of our knowledge, this is the first study designed to develop a Brazilian Portuguese version of the UCOPD questionnaire, which is a reliable questionnaire to assess patient knowledge of COPD and patient self-efficacy in managing the disease. Our findings can inform clinical practice, allowing the evaluation of the aforementioned outcomes and contributing to the development of strategies to improve them. In addition, they allow the evaluation of the results of educational programs.

In summary, when translating an outcome measure tool for use in a different country, it is important to ensure that the properties remain robust. The present study demonstrated that the Brazilian Portuguese version of the UCOPD questionnaire is reliable, has internal consistency, and has no floor or ceiling effects that might hinder its use in COPD patients in Brazil. In addition, the questionnaire is able to reflect health status, quality of life, and PADL in such patients. Further studies are needed in order to determine whether the Brazilian Portuguese version of the UCOPD questionnaire is responsive to PRPs.

REFERENCES

1. Spruit MA, Singh SJ, Garvey C, ZuWallack R, Nici L, Rochester C, et al. An official American Thoracic Society/European Respiratory Society statement: key concepts and advances in pulmonary rehabilitation. Am J Respir Crit Care Med. 2013;188(8):e13-64. Erratum in: Am J Respir Crit Care Med. 2014 Jun 15;189(12):1570. https://doi.org/10.1164/rccm.201309-1634ST
2. Bourbeau J. The role of collaborative self-management in pulmonary rehabilitation. Semin Respir Crit Care Med. 2009;30(6):700-7. https://doi.org/10.1055/s-0029-1242639
3. Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84:191-215. https://doi.org/10.1037/0033-295X.84.2.191
4. O'Neill B, Cosgrove D, MacMahon J, McCrum-Gardner E, Bradley JM. Assessing education in pulmonary rehabilitation: the Understanding COPD (UCOPD) questionnaire. COPD. 2012;9(2):166-74. https://doi.org/10.3109/15412555.2011.644601
5. Poureslami I, Kwan S, Lam S, Khan NA, FitzGerald JM. Assessing the effect of culturally specific audiovisual educational interventions on attaining self-management skills for chronic obstructive pulmonary disease in Mandarin- and Cantonese-speaking patients: a randomized controlled trial. Int J Chron Obstruct Pulmon Dis. 2016;11:1811-22. https://doi.org/10.2147/COPD.S105408
6. White R, Walker P, Roberts S, Kalisky S, White P. Bristol COPD Knowledge Questionnaire (BCKQ): testing what we teach patients about COPD. Chron Respir Dis. 2006;3(3):123-31. https://doi.org/10.1191/1479972306cd117oa
7. Hyland ME, Jones RC, Hanney KE. The Lung Information Needs Questionnaire: Development, preliminary validation and findings. Respir Med. 2006;100(10):1807-16. https://doi.org/10.1016/j.rmed.2006.01.018
8. Kamel H, Wolkove N, Baltzan MA. Disease related knowledge of patients with COPD. Chest. 2002;122(Suppl 4):144S.
9. Initiative for Chronic Obstructive Lung Disease--GOLD [homepage on the Internet]. Bethesda: Global Initiative for Chronic Obstructive Lung Disease [cited 2017 Apr 1]. Global Strategy for the Diagnosis, Management and Prevention of COPD 2016. Available from: http://www.goldcopd.org/.
10. Lourenco RA, Veras RP. Mini-Mental State Examination: psychometric characteristics in elderly outpatients [Article in Portuguese]. Rev Saude Publica. 2006;40(4):712-9.
11. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol. 2010;10:22.
12. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46(12):1417-32. https://doi.org/10.1016/0895-4356(93)90142-N
13. Streiner DL, Norman GR. Health Measurement Scales: A practical guide to their development and use. 4th ed. New York: Oxford University Press; 2008. https://doi.org/10.1093/acprof:oso/9780199231881.001.0001
14. Miller MR, Hankinson J, Brusasco V, Burgos F, Casaburi R, Coates A, et al. Standardisation of spirometry. Eur Respir J. 2005;26(2):319-38. https://doi.org/10.1183/09031936.05.00034805
15. Pereira CA, Sato T, Rodrigues SC. New reference values for forced spirometry in white adults in Brazil. J Bras Pneumol. 2007;33(4):397-406. https://doi.org/10.1590/S1806-37132007000400008
16. Sousa TC, Jardim JR, Jones P. Validação do Questionário do Hospital Saint George na Doença Respiratória (SGRQ) em pacientes portadores de doença pulmonar obstrutiva crônica no Brasil. J Pneumol. 2000; 26(3):119-28. https://doi.org/10.1590/S0102-35862000000300004
17. Dodd JW, Hogg L, Nolan J, Jefford H, Grant A, Lord VM, et al. The COPD assessment test (CAT): response to pulmonary rehabilitation. A multicentre, prospective study. Thorax. 2011;66(5):425-9. https://doi.org/10.1136/thx.2010.156372
18. Pitta F, Troosters T, Spruit MA, Probst VS, Decramer M, Gosselink R. Characteristics of physical activities in daily life in chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2005;171(9):972-7. https://doi.org/10.1164/rccm.200407-855OC
19. Fleiss JL, Levin B, Paik MC. Statistical methods for rates and proportions. 3rd ed. New York: John Wiley & Sons; 2004.
20. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34-42. https://doi.org/10.1016/j.jclinepi.2006.03.012
21. Terwee CB, Mokkink LB, Knol DL, Ostelo RW, Bouter LM, de Vet HC. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21(4):651-7. https://doi.org/10.1007/s11136-011-9960-1
22. Hulley SB, Cummings SR, Browner WS, Grady D, Hearst N, Newman TB. Delineando a pesquisa clínica. Porto Alegre: Artmed; 2006.
23. World Health Organization. World report on disability: World Health Organization; 2011.
24. Fahy BF. Promoting long-term benefits of pulmonary rehabilitation: the role of reducing the impact of respiratory exacerbations. Clin Chest Med. 2014;35(2):429-37. https://doi.org/10.1016/j.ccm.2014.02.007
25. Busby AK, Reese RL, Simon SR. Pulmonary rehabilitation maintenance interventions: a systematic review. Am J Health Behav. 2014;38(3):321-30. https://doi.org/10.5993/AJHB.38.3.1
26. Jones RC, Wang X, Harding S, Bott J, Hyland M. Educational impact of pulmonary rehabilitation: Lung Information Needs Questionnaire. Respir Med. 2008;102(10):1439-45. https://doi.org/10.1016/j.rmed.2008.04.015
27. Bolarinwa OA. Principles and methods of validity and reliability testing of question-naires used in social and health science researches. Niger Postgrad Med J. 2015;22(4):195-201. https://doi.org/10.4103/1117-1936.173959