Part 2

D. A new concept of peer review

Mindful of the GIGO-principle, editors of most high-class scientific journals ensure that all research papers, reviews and important clinical articles are submitted to a peer-review process before they are considered for publication. This eliminates the more obvious mistakes in data, or its interpretation.

In conventional peer-review, 2-4 referees examine the manuscript before its acceptance as suitable for publication. These people usually are experts in the field. They may question (as devil's advocates) any or all aspects of the article: the experimental design, ethical issues, the validity of the experimental licenses and patient-consent, the statistics, interpretation of the data, the validity of references cited, and (most of all) the conclusions reached. The referees return their comments to the editor, who compiles a list of comments and questions for the author(s). The editor will not publish the article until all of these are answered to the satisfaction of the expert referees.

In spite of the conventional peer-review process, prestigious journals publish many papers that prove subsequently to be quite incorrect or misleading. Using email/Internet to reach more referees than before, an improved pre-publication assessment could extend the present system. However, the old problems will remain. The Journal Editor must still find good referees who will devote the time to a detailed assessment of the paper, and return it quickly. Also some incorrect articles will get through the system. Therefore, so that they have a more meaningful "validity score" on each paper of interest to them, clinicians and young researchers need a much more effective system of peer-review to help them to structure their reading lists.

Internet technology offers the possibility of a unique, dynamic peer-assessment by specific categories of readers. Hundreds, if not thousands, of online readers could make a post-publication assessment. That system, based on an instant score, assigned under different assessment categories, and by "defined groups" of readers, would be the most relevant of all peer-reviews.

Each online article could contain an "Assessment Feedback Form" to allow two levels of online feedback by each reader who wishes to comment. (a) Immediate feedback (assessment 1) would be made immediately after the paper is read. (b) Considered feedback (assessment 2) would be made at any time (days to months later) after the reader has had time to reconsider the quality of the paper (its value to him/her), after having read other authorities in the area.

Both forms of feedback would be dynamic; i.e. updated constantly as new readers feedback their assessments under (a) and (b), above. Instead of going to the journal of publication, the assessments could be sent to an agreed International Organisation. This could be the World Health Organisation (WHO). The assessments could also be sent simultaneously to other organisations, such as designated national authorities, or the Food and Agriculture Organisation (FAO).

The software behind the database could calculate a PRES ranking (Professional Reader Evaluation Score) and update it on the Master Version, a copy of which is accessed via the WWW. This ranking could be made dynamic by updating the mean scores daily or weekly, as follows.

1. The USER would CLASSIFY his/her expertise in "Comment Boxes", as follows:
1. Research Leader working on that research area
2. Research Postgraduate working in that research area
3. Research Graduate working in that research area
4. Professional User (clinician, surgeon etc) who uses information in that area
5. Professional Academic (Professor, Lecturer) with no special expertise in the area
6. Professional General Reader with no special expertise in the area
7. Other (student, or non-expert in this area, etc)

2. The CLASSIFIED USER would then assign an evaluation score from 0 to 100 (where 0=worthless or grossly incorrect, 50=average and 100=superb) to each of the following aspects of the paper: (a) adequacy of the introduction/review/discussion; (b) materials and methods (design of experiment or observation); (c ) statistical analysis; (d) accuracy of interpretation of results; (e) conclusions (basic message or content):

	Score
a. Introduction / review / discussion	(0 to 100)_
b. Materials & Methods / Design	(0 to 100)_
c. Statistical Adequacy	(0 to 100)_
d. Accuracy of Interpretation of the Results	(0 to 100)_
e. Overall Scientific Value of Conclusion	(0 to 100)_

3. The User would submit the assessment data to the Master Version of the database. By dynamic links between the article and the designated organisations, the current (most recently updated) assessment scores could be assigned online to the article, and to the abstracting databases. These assessment scores could be part of the download to the user's Personal Computer for storage and retrieval later.

4. The Server of the Master Version would calculate and display the updated assessments as follows:

Professional Reader Evaluation Scores (PRES, range 1-100) by Reader Category (today's date)

	Leader	Postgrad	Grad	User	Acad	Gen	Other	ALL
Count	34	213	355	251	223	377	45	1,498
Intro/Rev/Disc Score	55	62	77	67	50	77	80	67
M&M / Design Score	76	89	84	54	57	87	100	78
Statistics Score	66	88	67	75	80	100	76	79
Interpretation Score	97	94	96	95	94	99	72	92
Conclusion Score	69	77	83	76	84	87	57	76
MEAN SCORE	73	82	81	73	73	90	77	78

From such tables, readers in each user category could see at a glance how their peers have ranked the paper. A peer-assessment system such as this also would allow searches of full-text journal articles, and of abstract databases, to include search fields for specific assessment scores. This would allow users to specify that they wanted abstracts (or full text) of articles with scores =/> those which the user may specify. This would allow automatic elimination of all articles from searches if their relevant assessment scores fall below a user-specified value.

Potential for abuse: Such a dynamic poll would have great implications for researchers, clinicians and users! De facto, it would allow user-assessment scores to be calculated for publications from all of the following: (a) individual authors; (b) individual journals; (c) individual laboratories or institutes, and (d) individual medical modalities (including AP and TCM). If it becomes the norm, it will have the most profound implications for those categories (a to d, above). For example, if papers published on AP research receive low mean scores for experimental design, AP researchers (in general) will have to reconsider their designs very seriously.

Any online polling- or assessment- system has some risk for bias, or for vested interests to manipulate scores up or down. For example, mediocre scientists (or laboratories), afraid of scoring badly, could "boost" their assessment scores by rallying every friend and acquaintance to assign maximum scores in all categories of the assessments of their papers. Similarly, rival groups of scientists could try to downgrade their competitors by assigning low scores to their papers. The system would not be able to identify a person with multiple email addresses (for example a work- and a home- address). For example, the same person may have different identifiers before the "@ sign" in the email address (for example, joebloggs@tinet.ie and joebloggs@grange.teagasc.ie. Also, different persons with different email providers may have the same identifier before the "@ sign" (for example jsmith@aol.com and jsmith@tinet.ie).

Policing of the PRES system: Proper planning and good programming would eliminate the worst of the potential abuses. Assessments would be accepted only from validated email addresses and duplicate addresses would be eliminated after the first entry in the initial assessment, and in the second (longer-term) assessment. Senders would be asked to identify their expertise in the area. Overall, this would be one of the nearest ways to have genuine dynamic international peer review.

Will each abstract on a future version of Medline include "Dynamic Assessment Scores, classified by reader Type"? This would make searches much more efficient by eliminating the dross! A researcher may then enter a desired assessment score in the search query. The search engine would then eliminate papers that fail to meet the desired score-criteria, and locate those papers with excellent scores, as predetermined by the researcher!

E. A new concept to assess clinical therapies: the International Therapeutic Forum

Why do practitioners prescribe the specific therapies that they use daily? Apart from the constraints of the availability of medicinal agents(s) and the ability of the client to pay for them, most practitioners prescribe the best therapies that they know for the diagnosed problem(s). In other words, the key factors in deciding the specific therapy are professional knowledge (especially of diagnostics and therapeutics) and personal belief (that the prescription will give satisfactory results). In any medical system (WM or TCM) the practitioner's knowledge and belief are conditioned by his/her culture and limited by his/her education and by ongoing access to credible information.

No practitioner can honestly claim 100% clinical success in all clinical conditions. Practitioners who value self-esteem and reputation, as well as the welfare of their clients, try to improve their clinical success in problematic conditions. The role of continuing education (peer-contact, refresher courses, conferences, seminars, email discussion groups, Internet Sites, etc) in enhancing therapeutic knowledge and skill has been discussed above. A new concept, as yet probably untried, offers great potential. That concept is the International Therapeutic Forum.

International Therapeutic Forum: Modern IT offers the possibility of a centralised international database specifically dedicated to the recording of clinical success rates to specific therapies in specifically diagnosed or confirmed clinical conditions.

Access to the password-protected database would be by application only. Initially, applicants would have to prove their bona fides as regards name, address, verified email address, current professional license, and category of practice. Those criteria would be updated annually.

On acceptance to the List, each member would be issued with a unique password that, when linked with his/her verified email address, would act as a positive identifier for that reporter. That identifier would be recoded to another unique ID for display on the database.

Members would be requested for an annual update (which could be done online) of each clinical condition that they wished to report. Input reporting fields, and output fields (readable by all Members) would be:

1. Reporter's ID
2. Clinical condition diagnosed
3. Diagnosis confirmed definitively or not). On entry of that field, the user would be queried to specify the nearest synonymous diagnosis if the original entry did not match a predetermined list of diagnoses, or to add his/her entry as a New Diagnosis (with full clinical details)
4. Main forms of therapy used (with treatment protocol, including details of complementary therapies)
5. Number of cases of that condition treated per year
6. Number of cases in each outcome category ( 0= worse, or no improvement; 1=slight improvement; 2=good improvement; 3=great improvement; 4=clinically cured)
7. Mean days from presentation to assessment for (f above)

The following table represents possible entry data for 6 different Members (MA to MF) for AP treatment of thoracolumbar disease in dogs:

a	b	c	d	e	No. cases in each outcome category					Mean days from presentation to assessment for category
ID	Diag	?	Therapy (see detail)	N/yr	0	1	2	3	4	0	1	2	3	4
MA	TLDD	Y	AP	15	1	1	1	1	11	12	18	16	17	18
MB	TLDD	?	AP	44	6	7	5	11	15	14	17	17	21	18
MC	TLDD	N	AP	32	1	1	3	4	23	16	23	20	22	20
MD	TLDD	Y	AP	9	0	0	0	1	8			17	20	19
ME	TLDD	Y	AP	81	6	1	2	5	67	14	18	17	19	18
MF	TLDD	?	AP	14	0	1	1	2	10		20	18	17	20
ALL	TLDD		AP	195	14	11	12	24	134	14	19	18	19	19

Before addition to the database, the entry data would be checked automatically for mathematical errors. For example the number of cases listed under Outcome Categories 0-4 must = the number in column e, or if a zero value is recorded in an outcome category, only a blank value would be allowed in the corresponding column under "Mean days from presentation .". The Reporter would be alerted to correct the data before online submission. On submission of correct totals, the data would be added to the database.

The user could sort the database by any or all of its entry fields, allowing grouping and output of selected subsets of data by field. The software behind the system could transform the data for a given diagnosis, and output the selected results as follows:

a	b		c	d	e	% cases in each outcome category						X days (pres to assess) for category
ID		Diag	?	Therapy (see details)	N/yr	0	*	**	***	****	3+4	0	*	**		***	****
MA		TLDD	Y	AP	15	6.7	6.7	6.7	6.7	73.3	80.0	12	18	16	17		18
MB		TLDD	?	AP	44	13.6	15.9	11.4	25.0	34.1	59.1	14	17	17	21		18
MC		TLDD	N	AP	32	3.1	3.1	9.4	12.5	71.9	84.4	16	23	20	22		20
MD		TLDD	Y	AP	9	0.0	0.0	0.0	11.1	88.9	100.0			17	20		19
ME		TLDD	Y	AP	81	7.4	1.2	2.5	6.2	82.7	88.9	14	18	17	19		18
MF		TLDD	?	AP	14	0.0	7.1	7.1	14.3	71.4	85.7		20	18	17		20
ALL		TLDD		AP	195	7.2	5.6	6.2	12.3	68.7	81.0	14	19	18	19		19

Usefulness of the database:

a. Such a database would highlight successful therapies and allow members to see how their clinical success rates compare with those of their peers for the same clinically diagnosed conditions. If Member MB has a particularly poor success rate in a specific condition (for example TLDD in dogs), and sees that Members MD and ME have much better success, Member MB could check the therapeutic details listed by Members MD & ME. He might find, for example, that he has ceased treatment too early (as above), or that the AP points used were very different to those used by the apparently successful therapists. Further email queries on technical aspects could be sent via the WebMaster to Members MD & ME.

b. Such a database would also highlight unsuccessful therapies, be they conventional drugs, herbal formulas, acupuncture formulas, etc.

c. A specific section of the database could be dedicated to online recording of adverse reactions and iatrogenic disorders. Various national bodies retain a register of such reactions, but practitioners would have more faith in an international database maintained and secured by a recognised international body.

d. Such a database has potential for abuse by vested interests. Commercial interests could try to boost or downgrade specific therapies or drugs. Fraudulent practitioners could use it to falsely inflate their success rates for self-advertising. Revenue Commissioners might be interested to use it to estimate potential income of listed practitioners. Regulatory Agencies might wish to monitor the use of unlicensed or illegal substances, etc. Thus, very high levels of database security, strict professional honesty in the claims made, and legal protection of the members for data shared there, would be essential elements for the success of this concept.

Discussion and Conclusions

IT and multimedia can provide rapid answers to specific technical queries, and solve more problems that depend on expert information. Given access to a phone and a modern computer, a practitioner in a rural outpost can cheaply communicate by email with colleagues around the globe. Direct access to medical and TCM databases, whether online or on CD ROMs, etc, allows instant access to the most up-to-date information. Self-study by distance learning will become a powerful educational medium in the near future. Modern IT also offers a marvellous opportunity to integrate eastern and western medical knowledge in a way never possible before.

Aside from the costs of buying modern hardware and paying annual subscriptions and phone-bills, two groups of workers (professionals in developing nations and isolated field workers and clinicians) would benefit greatly from modern IT. These groups would have access to the same technical information as those from wealthy industrialised countries. Depending on the quality of the people, access to the data would probably increase their therapeutic effectiveness and productivity (D Jaggar, Pers. Comm., 1998).

Modern IT is a global phenomenon. Its main benefit is rapid global communication. This can be on any subject from specific research topics, to precise methodology of expert techniques, to circulation of clinical results or problems, etc. Graphic images [radiographs, CAT scans, ultrasound scans, colour slides (for example of histopathological slides, plant leaves, tongue colour etc) can also be sent to expert colleagues for assessment via the Internet.

Online dynamic Peer-Assessment and the International Therapeutic Forum are still concepts only. Will some international group develop these concepts? Relatively free of political intervention and corruption by vested interests, they offer unique ways to allow the ultimate in peer-assessment of published information, and reporting of useful, useless and harmful therapies via the Internet.

However, as stated in the introduction, Change is the most fundamental Law of TCM. Most people inherently tend to resist change, or to fear the unknown. Also, technophobia, the fear or distrust of technology, is commonplace. Lazy or older people, or people unsure of their bona fides, their terminology or logical thinking processes, are reluctant to commit themselves to any searchable medium, especially digital media. Such media are unforgiving in recording, and ease of recall, of errors or fraudulent statements.

Problems to be overcome: Some problems remain to be overcome before IT can be a real force in precipitating integration of eastern and western medicine:

1. A major problem to be overcome is the tendency amongst certain sections of the experts in TCM to be defensive and secretive about their methods, including their AP- and herbal- formulas. In the past year, many discussions on the Professional Acupuncture List confirm that there is distrust, lack of respect, and even antipathy between certain sections of the expert TCM community and the "western medicine" community. The "Them and Us Syndrome" stems partly from fear of encroachment by one group into the other's professional area. This is inherently a "Turf-War" mentality; it is based on fear of losing income to the "other side". However, part of the problem is also a genuine belief that the therapy offered by the opposing group may harm patients, or may delay the instigation of "proper treatment". Uncertainty about the validity of their expertise is another cause of secrecy. People who are unsure of their facts or theory usually want to keep their lack of knowledge to themselves. If a fool stays silent, he or she may pass unrecognised as a fool for longer. Academic western science has no place for "secrets". An essential part of academic medicine is that all methodology is described accurately. Also, all medications licensed for medical use in most developed countries must be shown to be safe and effective, and must pass stringent quality-control regulations.
2. Another fear is that digitised media will make us mentally lazy. Many high school and third level students in the west can not do simple mental arithmetic since the advent of calculators, etc. Wisely used, IT need not lead to mental laziness. In fact, the reverse may be the case; IT offers the greatest learning possibilities ever known.
3. Some experts in TCM claim that their complex knowledge can not be simplified, translated, or documented in equations or detailed descriptive terms. This is the ultimate cop-out for mentally lazy or fraudulent people. It is surely possible to define the basic terminology, basic action and interaction of the theory and Laws of TCM. It is also possible to document the basic methodology for selecting remedies for diagnosed problems.
4. De facto, if something can not be described in accurate, repeatable detail, it can not be taught to the masses. Some aspects of TCM (and of medicine as practised by gifted western healers) can be classed as "paranormal-", "intuitive-" or "energetic-" medicine. Examples are clairvoyance, clairaudience, out-of-body experience, telepathy/distant hypnosis, kinesiology, fengshui, dowsing, qigong, telekinesis, homeopathy, etc. These areas must remain outside the realm of modern science, until it can evolve further to understand their laws. The ancient teaching system (Master/Guru to disciple, on a one-to-one basis, or in small experiential groups) is probably the best way to transmit this knowledge; it generally can not be taught (or learned) by standard distance-learning methods. As paranormal laws and theories, and decision-making systems are (literally) "up in the air", they probably are not suitable for consideration for coding on modern IT media at this time.
5. You all can help!: Though there are some encouraging attempts to use the internet and expert software databases as a way to provide in-depth, integrated knowledge on AP and TCM to western professionals, there is much more that could be done in these areas. I ask you all, especially those of you from the east, or from western organisations with expertise in AP and TCM, to encourage your national governments and universities to develop these areas. We need the highest-calibre translation of oriental (especially Chinese and Japanese) traditional medical terminology, diagnostics and therapeutics to be integrated (by expert national or international bodies) and incorporated into massive databases, accessible by the most user-friendly software. With today's technology, it should be possible for a vet or doctor with little knowledge of AP or TCM to interact with an expert guiding menu to supply all the clinical assessment data which the database needs to make a TCM diagnosis. Having made the diagnosis, the database could return the most appropriate treatment programmes, using AP, or TCM, or both. Development of such facilities would be of huge importance in the rapid uptake of oriental medicine by professionals in western nations.

It is possible today to encode much (but not all) of the richness of AP / TCM theory, methodology and practice on modern IT media. It would take a huge amount of concerted work by TCM experts, helped by western experts, to undertake this task. If international bodies, such as the WHO, can convince national Chinese and Japanese authorities of the value of this work, it could be done within 5-10 years. There would be a price to pay for full access to this wealth of information. Are we in the west prepared to pay that price? For the welfare of our clients and patients, and for the advancement of an integrated medical science, I dearly hope so.

References

• Felt B (no date) Paradigm's Computers and Oriental Medicine Forum at http://www.paradigm-pubs.com/html/forum.html
• Lin JH, Rogers PAM & Yamada H (1998) Integration of ancient and modern medicine towards a sustainable system of animal production and medical care. Proceedings of 24^th IVAS (International Veterinary Acupuncture Society) Annual Congress, Taiwan, August 1998.
• Wiseman N & Ye F (1998) A Practical Dictionary of Chinese Medicine. Paradigm Publications, Brookline, Mass, USA, 945 pp.

 

Appendix 1:	Useful Web Sites on acupuncture and traditional Chinese medicine
Appendix 2:	JAMA Editorial - May 1, 1996