Subscribe to our RSS Feed Chinese Medicine Times Facebook Fan Page Chinese Medicine Times Twitter Page Chinese Medicine Times Linkedin Page
Chinese Medicine Times

Clinical Studies and Randomized Controlled Trials in Chinese Herbal Medicine: A Historical and Contemporary Review - Part One

by Zhaoxiang Bian and David Moher

Introduction

The usage of complementary and alternative medicine in the Western world is becoming more popular and seems to have gathered momentum, especially in the past decade [1,2]. This could be due to the fact that there are still no satisfactory methods within the conventional medicine paradigm for dealing with many complicated illnesses such as cancers, cardiovascular diseases, functional diseases, and so on. Traditional Chinese Medicine (TCM) is a major component of complementary and alternative medicine [3]. When researchers and practitioners focus their attention on this traditional health care system with an attempt to extract useful hints and find ways to compensate for the inadequacy of orthodox or conventional medicine, they are bound to be overwhelmed by the immense amount of TCM classics and literatures, elusive theories, and clinical study reports. Undoubtedly the most important consideration for medical treatments is efficacy—one that is derived from the most stringent scrutiny of clinical tests and evaluations. After all, clinical study is the real litmus test for determining the safety and efficacy of any interventions in any health care systems, be they orthodox, conventional, alternative, or complementary medicine.

 

From ancient times to the present day, TCM practitioners have been working tirelessly on evidence gathering and have conducted many clinical studies in the course of their clinical practice. Some of these studies have been recorded in the classics as well as the current medical literature. Consider the situation of clinical trials with Chinese herbal medicine (CHM) in ancient times and today. How about the credibility of these trials? What is the future development of the clinical trial within CHM? Up to now, there has been no thorough review in this field. The primary objective of this article is to provide an overview of the clinical study of CHM from the past to the present. The secondary objective is to analyze the quality of the research methodologies used in clinical trials of CHM. Finally, the key issues and major problems of clinical trials of CHM and our recommendations will be identified and elaborated.

History of the Clinical Study of Chinese Herbal Medicine

To date, little information is available regarding the practice of TCM before the writing of the Yellow Emperor’s Classic of Medicine (Huang Di Nei Jing, compiled between 200BCE and 100CE), which is the most ancient and authoritative Chinese medical text found. In this classical work, health and illness are described as natural phenomena subject to investigation and observation, with therapeutic modalities discussed, including CHM, acupuncture, diet, and exercise. From then on, a colossal amount of medical literature recorded the results of ancient clinical studies, such as The Compilation of Clinical Case Studies (Zhen Jie) [4], which is the earliest clinical case study report, written in the Western Han dynasty, Discussion of Cold-induced Disorders (Shang Han Lun) [5] written in the Han dynasty, and Case Histories from the Matrix of Clinical Patterns (Lin Zheng Zhi Nan Yi An) [6] written in the Qin dynasty. Almost all of these records were written in the form of the case study. Nowadays, commonly conducted and reported clinical studies include descriptive study, such as case reports or case series; cohort study involving retrospective and prospective investigations; and controlled study with or without randomized and quasi-randomized trials. Although the concept of using a comparison group did appear about 900 years ago in Chinese medical literature (as recorded in the Atlas of Materia Medica (Ben Cao Tu Jing) originally published in 1061 [7]), it is evident that almost all clinical trials recorded in ancient CM literature are in the form of case study reports. In a milestone study involving two persons testing the efficacy of ginseng, it was found that the person who did not consume ginseng was short of breath sooner than his counterpart who had taken ginseng. Though the aspect of comparison existed, the study was only a rare exception to the mainstream studies in which CM practitioners’ evaluated efficacy based on observation of pre- and post-treatment symptoms in the same patient rather than comparing symptom changes in patients of treatment and controlled groups.

 

Why had most of the CM clinical studies been conducted in the form of the case study, and what contributed to this format lasting for more than 3000 years? Although no comprehensive study in this area has been conducted to answer the question, it could be inferred by examining the essence of Chinese medicine theory and its common treatment model.

Treatment based on syndrome differentiation is the cornerstone of the theory and philosophy of CM, which advocates individual treatment and emphasizes the fine-tuning of treatment in accordance with the change of symptoms in the patient. Hence, treatment strategies for different patients of the same disease and treatment methods applied during the different stages for the same disease could be different. That being the case, the concept of conducting controlled trials in clinical studies would naturally have been inhibited over the history of CM. This may explain why even though comparison studies in CM did appear briefly about 900 years ago; it was but a shooting star across the sky of CM history and failed to develop into a mainstream approach. Clinical case study reports eventually became the mainstream approach, as subsequently reflected by the voluminous classics from the Yellow Emperor’s Classic of Medicine (Huang Di Nei Jing) to Case Histories from the Matrix of Clinical Patterns (Lin Zheng Zhi Nan Yi An).

Case study reports record information about the efficacy of CHM for specific diseases, and no doubt they can be a useful reference to orthodox medicine. A typical case in point is the new drug discovery of artemisinin from qing hao (Herba Artemisiae annuae) [8], which has taken reference from the CM classic A Handbook of Prescriptions for Emergencies (Zhou Hou Bei Ji Fang ) of the Jin dynasty [9], in which the fact was recorded that fresh cold ground qing hao liquid can cure malaria. Nevertheless, drawbacks in clinical case study reports are evident. There could be interference in the judgment of the efficacy of CHM from factors such as the development process of diseases and the patients’ subjective feelings toward the treatment. Hence, results recorded in CM case study reports cannot be readily generalized and applied. Based on the hierarchy of evidence in the theories of evidence-based medicine [10], the strengths of case studies or case reports are weaker than those of retrospective cohort studies, prospective cohort studies, non-randomized controlled studies, and randomized controlled studies. The detailed hierarchy of evidence is shown in Figure 1.

Figure 1. Hierarchy of clinical efficacy study research evidence.

(This figure was originally published in JICM 2006, 4, 455-466, cited with approval from the JICM.)

The strongest evidence for clinical study regarding the efficacy of one intervention comes from randomized controlled trials (RCTs) and systematic review that performs quantitative statistical pooling of results from included RCTs when they are sufficiently homogeneous. It should, however, be noted that the concept of RCTs in medicine is relatively recent. The first RCT began in 1946 and was conducted to test the efficacy of immunization against whooping cough [11]. Since the results of that study were not reported until 1951, the first published RCT was in fact organized by D’Arcy Hart and Daniels to examine the efficacy of streptomycin for treating pulmonary tuberculosis [12]. In CHM, the first RCT was reported in 1983, which examined the effects of Buxux harlandii on coronary heart disease [13]. Numerous RCTs in CHM have been conducted and published in the past three decades [14]. No doubt, RCTs will be the major clinical trial modality for evaluation of the efficacy of CHM in the future.

Quality of Research Methodology in Clinical Trials of Chinese Herbal Medicine

Well designed RCTs have been widely recognized as being able to provide the strongest evidence of the effectiveness of health care interventions [10]. The credibility of evidence generated from such activities depends on the quality of methodology of RCTs, which includes trial design, conduct, and final reporting. Our previous study showed that the common problem with the RCTs in CHM is the poor quality in methodology and reporting [15].We analyzed 11 systematic reviews on CHM, which were published in the Cochrane Library database from January 2000 to July 2005, and appraised the quality of 167 RCTs that have met the basic inclusion criteria of respective systematic reviews. The 11-item quality checklist modified from the revised Consolidated Standards of Reporting Trials (CONSORT) statement with two CHM-specific items (herb preparation form and quality control of herbs) was used for quality assessment. The results showed that the quality of an overwhelming majority of these RCTs was very poor. All these trials included statements about the interventions, objectives, primary outcome design, statistical methods, and herb preparation form. Although 163 (97.6%) trials reported inclusion criteria, exclusion criteria were only reported in 26 (15.6%) trials. Fewer than 10% of trials had clear statements of the random allocation sequence generation methods, and only 2.4% mentioned allocation concealment. The vast majority of trials (86.8%) were open-label, while only 13.2% used blinding. Almost half (45.5%) of the trials administered CHM intervention in the form of tea or decoction. Only one trial (0.6%) reported a sample size calculation, and one (0.6%) discussed quality control of the CHM intervention. The breakdown of the quality of each item is listed in Table 1 [15]. Without a doubt, if the overall quality of CHM trials is to be improved, each step and item of the trial design has to be strictly observed and adhered to.


Item

Stated

Percentage %

1

Participants

Inclusion criteria

Exclusion criteria

163

26

97.6

15.6

2

Interventions

167

100

3

Objectives

167

100

4

Outcomes

Primary outcome

Secondary outcome

167

4

100

2.4

5

Sample size

Sample size calculation

1

0.59

6

Sequence generation

14

8.4

7

Allocation concealment

4

2.4

8

Implementation

Who generated the allocation sequence

Who enrolled participants

Who assigned participants to their groups

0

0

0

0

0

0

9

Blinding (masking)

Double blinding with clear explanation

Double blinding without clear explanation

Single blinding with clear explanation

Single blinding without clear explanation

0

13

5

4

0

7.78

2.99

2.4

10

Form of herbs

167

100

11

Quality control of herbs

1

0.6

 

Table 1. Breakdown of all items related with methodology.

(This table was originally published in JICM 2006, 4, 120-129, cited with approval from the JICM,)

Selection of Participants

Participant selection must ensure that all subjects are equal in terms of targeted diseases. It is necessary to set up inclusion and exclusion criteria for enrolment, with the inclusion criteria aim, to define the target patients, whilst the exclusion criteria weed out subsets of the target population that may undermine the validity of the results. Failure to specify inclusion and exclusion criteria may undermine the internal consistency of the clinical study. There is a possibility that patients who meet the inclusion criteria but also exclusion criteria will be enrolled, with baseline differences in their health status perhaps affecting anticipated response to treatment. On the other hand, overly restrictive exclusion criteria will limit generalization of study results. Therefore participants must be properly selected to ensure all of them have the same baseline so that trial results can reflect the difference of intervention.

As for the RCTs in CHM, one important issue should be emphasized when the inclusion and exclusion criteria are set: participants may be selected according to the disease of conventional medicine, or syndrome base of TCM, or an integrated version of both. One proposed method to tackle this problem is to set the criteria based on the objective of a clinical trial. If the aim of this trial is to verify the intervention to the disease, then the disease will be the base to set the criteria, especially the inclusion criteria. If the objective is to verify the intervention for the syndrome, then the syndrome will be the base for setting the criteria. If the aim is for the syndromes of a specific disease, an integration of syndromes and disease is necessary to set the criteria.

Sample Size Calculation

Our previous studies showed that the few RCTs that reported calculating the sample size did not record the method and details used to perform this calculation, thus severely limiting their usefulness [15]. It is necessary that one trial has a proper sample size; sample size should be calculated before the beginning of the clinical trial and calculation methods should be clearly documented and reported.

Proper sample size is necessary to make sure, on the one hand, that it is large enough to enable the investigators to observe the differences in the efficacy among tested interventions. On the other hand, it must not be so large as not to waste resources, especially the time spent in recruiting and screening potential participants, and to avoid unduly exposing too many participants to an experimental intervention. However, if a sample size is too small, the researchers might not be able to detect the presence of differences between different interventions (i.e. type II error or false negative), and the study might be a waste of resources and potentially unethical. The calculation method of sample size could be obtained from references [16, 17]. The sample size should be calculated before implementing the clinical trial.

It is not proper to start a trial without a target number of participants. Furthermore, the calculation method of sample size should be presented in the report so that readers can ascertain the origin of the number.

Randomization

Randomization is a convincing method to assign participants to different aims on weighing possible known and unknown interferences, so as to avoid systematic bias. The quality of the randomization process, especially of the randomized allocation sequence generation, determines the quality of the result from RCT. Our previous study showed that just 8.4% (14/167) of trials with CHM provided clear statements about sequence generation, while 91.6% (153/167) merely stated that participants were randomized with no explanation of how the allocation sequences were generated [15]. Absence or no adequate explanation of sequence generation results in readers being unable to judge whether proper trial methods were used. Allocation sequence could be generated either by simple (unrestricted) randomization or stratified randomization [18, 19]. The sequence generation method of randomization will determine the scientific accuracy and credibility of RCTs. Therefore, strictly following every step in randomization, and clearly reporting these steps, are crucial to RCTs in CHM.

Allocation Concealment

In ensuring the success of randomization, allocation concealment is essential because random sequence generation merely means that the group assignment sequence of a participant is randomly generated, but it does not ensure that the allocation sequence is consistently followed during implementation. In order to allocate the participants to balanced treatment groups, the sequence should be concealed to all persons involved in the study, such as investigators and participants. Otherwise, selection bias may occur—treatment assignment is not genuinely random and an imbalance in prognostic factors may occur between treatment groups. Previous studies have shown that inadequate or unclear allocation concealment can exaggerate clinical effects by up to 40%, especially in poorly conducted trials [20], and it may also cause greater heterogeneity in results [21]. Hence, proper randomization should involve both random sequence generation and complete implementation of the sequence to minimize bias. The reason for the poor rate of randomization concealment (and/or the reporting of it) in RCTs in CHM is still unknown, but some researchers believe that with blinding (double blinding in particular), allocation concealment is not necessary. Nevertheless, it should be noted that blinding and allocation concealment are different [22]. Blinding prevents participants and other personnel involved in the study from determining to which group the participants should be assigned (which will lead to ascertainment bias) and it safeguards the sequence after allocation. On the other hand, allocation concealment prevents selection and confounding biases; it safeguards the assignment sequence before and until allocation. Commonly used methods to conceal allocation include calling a central coordination office for each patient assignment at the time the patient is present for study inclusion, using sequential numbering, opaque sealed envelopes or numbered bottles or containers [23]. In RCTs in CHM, not enough attention has been given to allocation concealment. The researchers should understand its importance and implement it properly in clinical trial.

Blinding

The aim of clinical trials is to explore the objective efficacy of the target intervention. To maintain objectivity it is essential to prevent results from being interfered with, diluted, or misled by subjective preferences (bias) of the participants, investigators, or assessors [24]. In this regard, blinding is the golden standard for clinical trial design and should be applied even to trials with objective assessment indices. Previous reports have shown that trials that were not double-blinded yielded over-exaggeration of treatment effect compared with trials in which authors reported double blinding (odds ratios exaggerated, on average, by 17%) [25]. Therefore, it is essential for blinded tests to be conducted. Our previous study showed that the quality of blinding in RCTs in CHM is far from satisfactory [15]. Only 13.2% (22/167) of trials were described as blinded, and among them more than half claimed to have been double-blinded, yet showed no details on how this had been implemented. To improve the situation, more attention should be given to blinding in clinical trials. This is especially true for trials in CHM in which efficacy is largely subjective.

A challenge for CHM trials to conduct blinding is the preparation form of the intervention herbs, which may be administered in the form of tea, tablets, capsules, or decoction (extract of a crude drug made by boiling or simmering herbs in water, which usually has a stronger effect than tea or infusion). Due to their characteristic odour, flavour, and colour, the CHMs may be easily detected and identified by participants and/or clinicians, thus hampering successful blinding. Ideally, interventions for different treatment groups should be given in the same form with the same route of administration, as is common in conventional medicinal drug trials. A double dummy trial design could be used to overcome the above-mentioned challenge, whereby each participant receives two preparation forms (e.g., tea and capsule), only one of which contains the active treatment, while the other contains a placebo.

Contact Details

Zhaoxiang Bian, PhD
Associate Professor
School of Chinese Medicine
Hong Kong Baptist University
Kowloon Tong, Hong Kong
bzxiang@hkbu.edu.hk

This review is based on our previous four papers published in Zhong xi yi jie he xue bao, 2006. We thank the journal for permitting us to conduct this review based on these four papers and the cited tables therein. Edited by Michael McCarthy, Stephen Birch, et. al. (eds.), Thieme Almanac 2008: Acupuncture and Chinese Medicine, Thieme: Stuttgart–New York; 2008.

References

1. Kessler RC, Davis RB, Foster DF, Van Rompay MI, Walters EE, Wilkey SA, Kaptchuk TJ, Eisenberg DM. Long term trends in the use of complementary and alternative medical therapies in the United States. Ann Intern Med. 2001, 135(4):262-268.

2. Tindle HA, Davis RB, Phillips RS, Eisenberg DM. Trends in use of complementary and alternative medicine by US adults: 1997-2002. Altern Ther Health Med, 2005, 11(1):42-49.

3. WHO traditional medicine strategy 2002-2005 (Document WHO/EDM/TRM/2002.1) Geneva, World Health Organization 2002.

4. Si Ma Qian, Grand Scribe’s Record, Beijing Publishing House, The People’s Republic of China , 2004.

5 Zhang Zhong Jing, Discussion of Cold-induced Disorders (Shang Han Lun), He Bei Science &Technology publishing Company, HeBei, The People’s Republic of China , 1994.

6. Ye Ting Shi, Case Histories from the Matrix of Clinical Patterns (Lin Zheng Zhi Nan Yi An), Shanghai Science & Technology publishing Company, Shanghai, The People’s Republic of China , 2000.

7. Su S. Atlas of Meteria Medica. Hefei: Anhui Science & Technology Publishing House, 1995.

8. Pharmacopoeia Commission of the Ministry of Public Health of PRC, Chinese Pharmacopoeia, Chemical Industry Press, The People’s Republic of China , 2005, P137

9.Ge Hong, A Handbook of Prescriptions for Emergencies (Zhou Hou Bei Ji Fang), TingJin Science &Technology publishing Company, The People’s Republic of China , 2000.

10. Sackett DL, Straus SE, Richardson WS, et al. Evidenced-based medicine: how to practice and teach EBM. 2nd ed. London: Churchill Livingstone, 1997

11. Prevention of whooping-cough by vaccination; a Medical Research Council investigation. Br Med J, 1951, 4721: 1463-1471.

12. Medical Research Council Streptomycin in Tuberculosis Trials Committee. Streptomycin treatment for pulmonary tuberculosis. BMJ, 1948, ii: 769-782.

13. Analysis of the effect of Buxux harlandii on coronary heart disease by double blind method. Report of 110 cases. J Tradit Chin Med, 1983, 24(5): 37-40.

14.Katz DL, Williams AL, Girard C, Goodman J, Comerford B, Behrman A, Bracken MB. The evidence base for complementary and alternative medicine: methods of Evidence Mapping with application to CAM. Altern Ther Health Med. 2003; 9(4):22-30.

15. Bian ZX, Li YP, Moher D, Liu L, Wu TX, Miao JX, Song L, Dagenais S. Improving the Quality of Randomized Controlled Trials in Chinese Herbal Medicine, Part I: Clinical Design and Methodology. Zhong Xi Yi Jie He Xue Bao, 2006,4(2):120-129.

16. Schulz KF, Grimes DA. Sample size calculations in randomized trials: mandatory and mystical. Lancet, 2005, 365(9467): 1348-1353.

17. Friedman LM, Furberg CD, DeMets DL. Fundamentals of clinical trials. 3rd ed. New York: Spring Verlag, 2005.

18. Schulz KF, Grimes DA. Unequal group sizes in randomized trials: guarding against guessing. Lancet, 2002, 359(9310): 966-970.

19. Schulz KF. Grimes DA. Generation of allocation sequences in randomized trials: chance, not choice. Lancet, 2002; 359(9305): 515-519.

20. Kjaergard LL, Villumsen J, Gluud C. Quality of randomized clinical trials affects estimates of intervention efficacy. 7th Annual Cochrane Colloquium Abstracts. Rome: 1999.

21. Juni P, Altman D, Egger M. Systematic review in health care: Assessing the quality of controlled trials. BMJ, 2001, 323(7303): 42-46.

22. Schulz KF, Grimes DA. Allocation concealment in randomized trials: defending against deciphering. Lancet, 2002, 359(9306): 614-618.

23. Schulz KF. Assessing allocation concealment and blinding in randomized controlled trials: Why bother? ACP J Club, 2000, 132(2): A11-A12

24. Schulz KF, Grimes DA. Blinding in randomized trials: hiding who got what. Lancet, 2002, 359(9307): 697-700.

25.Schultz KF, Chalmers I, Hayes RJ, et al. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA, 1995, 273(5): 408-412.

Payment methods

| | | |

This site and contents are copyright 2006 - 2012 ©

is the trade name of CMT Integrated Health Ltd, , , , , . Registered in England and Wales No. 6528121. VAT No. GB 941 4574 19.