CHAPTER 3

DEVELOPING THE EXPERT DIAGNOSTIC SYSTEMS

Introduction

Florida's warm subtropical climate and relatively high annual rainfall is conducive to the production of a wide range of subtropical and tropical fruit crops. It is well known that pest damage, plant diseases, as well as disorders are some of the major factors which limit the productivity of these crops. In order to manage these problems and so limit their negative impact on crop production, correct symptom identification and rapid diagnosis is vital. An integrated approach involving chemical, cultural and biological control methods is often utilized within fruit crop management programs in Florida.

Florida producers, however, sometimes rely heavily on chemical control measures (J. H. Crane, Univ. of Florida, personal comm.). Pesticides may sometimes be used without proper identification of the causal agent. Sometimes symptoms of a disease or disorder are mistakenly attributed to another, resulting in unwarranted pesticide or fertilizer use (J. H. Crane, Univ. of Florida, personal comm.). Yet many of the diseases, disorders and pest damage can be identified easily, with the availability of appropriate resources, by specific and distinctive symptoms appearing on different morphological regions of a plant such as the leaves, small branches, trunk or fruit. For example, with some commonly reoccurring disease and pest problems, an experienced producer may be able to identify the causal agent and apply correct management recommendations. However, with other problems, an experienced expert is often required to provide diagnoses of diseases, disorders and pest damage because producers are unfamiliar with the symptoms. Unfortunately, most experts are rarely able to allocate adequate time to the assistance of all diagnostic requests.

Historically, major state wide extension programs have provided diagnostic services and pest management information to support growers and county extension faculty to minimize problems of misdiagnosis and unnecessary agrochemical use. Currently, information is disseminated primarily via county and commodity oriented meetings, on-site grove visitation, as extension fact sheets, publications, circulars and handbooks or through the FAIRS CD-ROM series (Johnson and Beck, 1986). Producers increasing awareness of commercial computer applications in horticulture has also led to the request for the development of advanced computer technology with applications in horticulture. As a result, these conditions have accelerated the need for the development of interactive computer-based expert diagnostic systems to assist growers and extension faculty to diagnose diseases, disorders and pest damage.

Consequently, we proposed to develop two interactive computer-based diagnostic expert systems, one for citrus crops and another for six tropical fruit commodities including avocado, carambola, lychee, mango, papaya and 'Tahiti' lime. These diagnostic systems are designed to perform at the level of knowledgeable human experts but can be easily understood and utilized by extension faculty and commercial producers as a valuable tool to assist them in the field diagnoses of diseases, disorders and pest damage of subtropical and tropical fruits. In addition, the diagnostic systems are intended to be integrated and distributed on CD-ROM with the Florida Agricultural Information Retrieval System (FAIRS) which is a comprehensive electronic library, currently consisting of thousands of documents from major programs within the University of Florida's Institute of Food and Agricultural Sciences (IFAS) (Johnson and Beck, 1986; Beck et al., 1994).

Materials and Methods

The methodology used to develop the diagnostic systems was divided into four major areas including Knowledge Acquisition, Knowledge Representation, User Interface, Design and Testing.

Knowledge Acquisition

Since an expert system is primarily based on reasoning methods that reflect expert human logic, each diagnostic system was developed based on the diagnostic reasoning process and expert knowledge acquired from a single expert in the Horticultural Sciences Department. Dr. James J. Ferguson, Citrus Extension Specialist, participated in the development of the citrus diagnostic system (CIT·Xpert) and Dr. Jonathan H. Crane, Tropical Fruit Crop Extension Specialist, served as the primary expert for the development of the tropical fruit crops diagnostic system (TFRUIT·Xpert).

The first procedure was to identify and select the domain of each diagnostic system. Since expert systems are best designed for very narrowly defined domains, it was determined that only diseases, disorders and pest damage problems which occur in the field would be addressed. As a consequence, post-harvest diseases and disorders were omitted.

Secondly, the reasoning method of diagnosing problems was identified by the two participating experts independently. It was agreed that the best reasoning strategy was to separate specific symptomology by major morphological regions of the tree including the roots, trunk, major limbs and scaffold branches, small branches and twigs, leaves, blossoms, fruit and the entire tree (APPENDIX A). For CIT·Xpert, a further breakdown was required for two morphological regions, the roots (fibrous feeder, crown roots, and main lateral roots, entire root system) and the trunk sub-regions (rootstock, budunion, scion, entire trunk) (APPENDIX B). TFRUIT·Xpert utilized a simpler diagnostic reasoning process based solely on morphological regions. Broad symptom classes were then identified such as change in appearance, change in shape and presence of insects and in some instances sub-classes were utilized to distinguish between unique symptomology such as brown, black, white, or yellow in color. Specific symptoms were finally placed into one or more of these symptom classes or subclasses based on morphological regions or subregions.

Thirdly, each participating expert authored a summary document for each causal agent within the domain (APPENDIX C). These summary documents were developed to provide the user with a concise review of the major symptoms for each causal agent and to provide a source of specific symptoms to generate the symptom list as well as a source of hypertext links to additional IFAS extension documents. A total of 91 summary documents were prepared for CIT·Xpert and 180 documents for TFRUIT·Xpert. Each summary document contains at least a single paragraph and up to a full length page of text which includes a detailed listing and description of the symptoms by morphological region, and pertinent information such as susceptible cultivars or the identification of specific symptoms which can be commonly confused with other causal agents, pest damage or disorders. The common name and scientific name of the causal agent or pest, as well as the seasonal occurrence of the disease, disorder and pest is also available. The content of these summary documents was based both upon the personal training of the participating expert's and field experience as well as information extracted from current IFAS extension publications, current literature, and peer reviews.

After the summary documents were completed, a comprehensive list summarizing the visual symptoms within each individual systems domain was generated. Table 3.1 shows a partial symptom listing of the CIT·Xpert facts. For each individual case or problem, the causal agent (goal) was defined as the hypothesis whereas the supporting facts were defined as symptoms which could be visually observed and easily verified.

In order to provide the user with on-screen color images to illustrate symptomology, it was necessary to acquire 35 mm slides for each identified symptom. Images were provided in 35 mm color slide format by the experts as well as numerous other sources. A review of existing 35 mm slides from the Horticultural Sciences Department collection was also completed as well as a search for slides from IFAS researchers and faculty. An attempt to obtain a slide image for each of the 269 symptoms detailed for CIT·Xpert and the 400 symptoms for

Table 3.1 Developing the symptom list from the summary documents.

Goal(s) or hypotheses	Symptoms
Root Rot	Vertical bark splitting near base of trunk
Cold Damage	Vertical bark splitting near base of trunk
Growth Splits	Vertical bark splitting near base of trunk
Citrus Snow Scale	Vertical bark splitting near base of trunk
Heart Rot	Emergence of fungal fruiting bodies from the bark
Fire Ant Damage	Bark girdling
	Oozing of gum
Foot Rot	Bark girdling only at the soil level
	Oozing of gum

TFRUIT·Xpert was made, however in many instances slides detailing specific symptoms either did not exist or could not be obtained. Every effort was made to obtain and select slides of the highest quality from the limited supply which was available. None of the slides were taken by professional photographers, and as a result there were common problems with image quality, specifically with images which had been duplicated, including inadequate depth of field, poor contrast or poor color representation. In the case of visual nutrient deficiency symptoms for three of the tropical fruit crops included in the domain for TFRUIT·Xpert (carambola, lychee and papaya) there was little information and no images available. As a consequence, an additional sand culture nutrient omission greenhouse experiment was designed and completed to obtain up-to-date 35 mm images and symptomology of certain nutrient deficiencies. This experiment will be described in detail in the next chapter.

All acquired 35 mm slides were processed digitally by one of two methods, either by using Kodak Photo CD technology in which 35 mm slides were sent to a Dale Laboratories in Hollywood, FL and processed onto special compact discs or by manually scanning each image by using a AGFA color scanner. A total of twelve Kodak Photo CD disks were processed containing more than 1000 images. Each Kodak Photo CD can contain up to 110 images stored at 5 different resolutions. Following digitization, each slide image was reviewed on a 17 inch VGA+ color computer monitor and manipulated using Adobe Photoshop 3.0 software to improve the quality and content of each image. Only the best quality images were selected for inclusion into the diagnostic systems. All images were stored as high-resolution color images in a "PCX" format using 8 bit color - 640 x 480, (256 colors) pixel resolution. Since the resulting "PCX" files do not reproduce an image of the same quality as the original slide, the difference being the resolution and color representation, many slides were of inadequate quality and were not included in the image database. In many instances, symptoms could not be fully represented by a single photograph. Consequently, two to four images were occasionally combined into one composite image in order to create a representative full screen color image showing a certain range of symptom expression. The entire digitization process was performed on a Power Macintosh computer system designed specifically for scanning 35 mm slides. At the conclusion of the scanning all selected electronic images where transferred to and stored on a PC-based system. A full screen color "PCX" image generally required less than 300 kilobytes of hard disk storage space.

Knowledge Representation

Several informal discussions were conducted with the primary and secondary experts to determine the best strategy for representing the knowledge base. Initially a rule-based, dichotomous key and matrix approach were identified as plausible methods. However, upon closer examination the rule-based approach was determine to be inappropriate due to the large size of the diagnostic systems domains. We determined that a rule-based approach would be too difficult and cumbersome to verify and to periodically update.

Dichotomous or decision trees are a graphic representation of the reasoning pathways. In some instances decision trees can be a useful tool to the knowledge engineer for representing knowledge. Nodes on the tree represent subgoals with possible values for each node form each pathway emanating from each node. The decision tree approach, however, was also rejected based on the large domain size and excessive number of pathways which could be searched, as well as the fact that the user interfaces would have to be developed independently for each node and could not be easily generated. Additionally, sometimes the user cannot reply "Yes" or "No" to a query and consequently, the diagnostic process is stopped and doesn't lead to a final conclusion. This widely recognized fault of decision tree approaches was found to be an unsuitable characteristic to provide rapid diagnoses.

As a result, the best search strategy was determined to be a matrix approach in which the logic could be more easily represented, verified, validated and periodically updated or expanded even to include new fruit crop commodities. Initially a flow-chart was developed based on the diagnostic reasoning process that the experts followed in reaching diagnoses of field symptoms. In attempt to assist in the development process, a simple prototype using HyperCard 2.1 was developed to mimic their reasoning process and then presented to the experts to determine the feasibility of the proposed methodology. Once the matrix methodology was agreed upon by a consensus, the primary experts began authoring the summary documents, detailing the specific symptomology for each causal agent.

The next development phase included identifying specific symptoms from the summary documents and editing the symptom lists by placing similar symptoms or supporting facts and causal agents together which occurred within the same morphological region of a tree (Table 3.2). These supporting facts from the symptom list were then subsequently assembled into a tabular form using Microsoft Word 5.1 for the Macintosh® with the column headings being the symptoms and the row headings as the causal agent or goal. When a fact was identified as a symptom of a specific goal an "x" was placed in each corresponding cell of the table to represent the relationship. An example of a portion of a table developed for part of the CIT·Xpert is shown in Table 3.2.

A two, three or four layer hierarchy was developed depending upon specific symptomology. The software which generated the interface was limited to only a four level hierarchy. The first level of the table was based solely upon the major

Table 3.2 Microsoft Word table generated from the symptoms list.

Goal(s) or hypotheses	Trunk (symptoms)
	Rootstock
	Growth Abnormality	Fungal Growth	Girdling		Gumming
	Vertical bark splitting near base of trunk	Emergence of fungal fruiting bodies from the bark	Bark girdling	Bark girdling only at the soil level	Oozing of gum
Root Rot
Cold Damage
Growth Splits
Citrus Snow Scale
Heart Rot
Fire Ants
Foot Rot
	Image1.pcx	Image2.pcx	Image3.pcx	Image4.pcx	Image5.pcx

morphological regions of a tree previously mentioned. The second level required symptoms to be grouped into one of several morphological sub-regions such as rootstock, budunion or scion. A third level required further breakdown into symptom classes such as growth abnormality, fungal growth, girdling, gumming, death, decay and/or sloughing of bark, lesions and/or cankers. And finally the final fourth level of the hierarchy included the actual detailed description of the symptom. A C++ computer algorithm was then used to read the WordPerfect 5.1 tables to extract the tables text and values and generate a matrix as shown in Table 3.3.

Table 3.3 Matrix generated from a Microsoft Word table.

Root Rot	1	0	0	0	0
Cold Damage	1	0	0	0	0
Growth Splits	1	0	0	0	0
Citrus Snow Scale	1	0	0	0	0
Heart Rot	0	1	0	0	0
Fire Ant Damage	0	0	1	0	1
Foot Rot	0	0	0	1	1

User Interface Design

The final step involved constructing a graphic user interface to direct the user to a rapid diagnosis using as non-technical a terminology as possible, so that a 'layperson' could easily understand and respond to the on-screen symptoms descriptions. It should be noted that the development of the interface of each system was not the responsibility of the knowledge engineer and was beyond the scope of this thesis investigation. The development of the interface was primarily the responsibility of Dr. Howard Beck, Agricultural Engineering Department and generated using Borland C++ 3.0, an object-oriented programming language. The application of C++ provided a class library for generating interface features such as buttons, windows, hypertext displays, pop-up menus, and dialog boxes. An example of a screen display illustrating the user interface features available under MS-Windows is shown in Appendix D. Several other screen displays were also developed by a graphic artist using Adobe Photoshop 3.0. These displays included a welcome screen (APPENDIX D), an acknowledgments screen, an about screen, a help screen and a menu screen depicting the six tropical fruit crop commodities as well as a screen illustrating the morphological regions of each fruit crop tree (APPENDIX A). Each diagnostic system was developed for both MS-DOS 5.0 and MS-Windows-based computer operating systems and can be operated on 386 or 486 PC-based system with a VGA+ standard color monitor and CD-ROM. Currently, all county extension offices in Florida have the requisite hardware for delivering such a system.

Validation

Validation of the symptoms of each disease, disorder and pest damage occurred periodically throughout the development of the diagnostic systems by the knowledge engineer. Symptoms of causal agents were also collectively reviewed by an accordance of prominent experts not directly involved with the design of the diagnostic programs. Numerous other experts were involved with validation of images expressing symptomology and served as reviewers of the summary documents as well. Changes in symptom descriptions, slides and summary documents recommended by the reviewers was completed and incorporated into each diagnostic system. Printouts of text generated from the C++ algorithm were also reviewed by the knowledge engineer to insure the systems would provide credible diagnoses. Misspellings and inconsistencies were corrected and all "PCX" images were reviewed to ensure they were properly linked to described symptoms and correctly formatted.

Verification

Prior to transferring the prototype programs to individual CD-ROMs, each diagnostic system was certified to be correct by the knowledge engineer by randomly going through possible pathways and verifying that the conclusion matched the data acquired from the matrix. Although further verification was desired, problems of access to the primary experts, time constraints, hardware incompatibilities and resource limitations prohibited extensive verification. Further verification by the primary experts would have permitted the experts to challenge the systems, and to better understand the way the systems functioned and performed. Possibly additional refinement of the diagnostic systems may have occurred had the limitations been minimized. As a consequence the experts did not have the opportunity to thoroughly review the diagnostic systems prior to field testing. The systems, in their present state, are first generation prototypes. Further validation and field testing are recommended before distributing and placing the systems into service.

Testing

Each diagnostic program was transferred to a separate CD-ROM and independently distributed to two testing sites, namely the Citrus and Research Education Center (CREC), Lake Alfred, FL and the Tropical Research and Education Center (TREC), Homestead, FL. The purpose of the testing was to compare the capability of subjective human diagnosis versus the computer-based diagnostic expert systems, and for the TFRUIT·Xpert system to compare the efficiency among the three groups to determine if the diagnostic system could improve non-experts diagnostic capabilities. A secondary purpose was to test the user interface.

At the beginning of the testing, an evaluation questionnaire was given to each user which requested basic information about the experience of each participants, familiarity with computers and the diagnosis of diseases, disorders and pest damage of citrus or tropical fruit crops (APPENDIX E). Comments on the design of the graphic user interface, the diagnostic reasoning process and the content of the summary documents were also requested. Each user was given a brief explanation of the function and operation of each system and was encouraged to ask questions and provide input at any time during the testing session.

The second section of the testing was designed to evaluate the ability of the diagnostic systems to assist a user in making correct diagnoses of specific diseases, disorders and pest damage. The second section consisted of two separate evaluation components. First, each participant was asked to identify 10 to 12 randomly selected real-world samples obtained from nearby orchards. Identifications were based on their personal subjective knowledge and experience without assistance of the diagnostic system. In the second part, each participant was then asked to use the expert diagnostic systems to diagnose the same 10 or 12 randomly selected real-world samples. Each participant was also asked to indicate the level of certainty of their diagnoses by circling a number corresponding to their confidence based on two Likert-type scales (Meister, 1986). A Likert-type scale that uses a verbal continuum of response anchors can be used to rate the response of the participants (Meister, 1986). Likert scales are commonly used to measure attitudes in the field of psychology (Likert, 1932). Figure 3.1 presents a typical Likert-type scale used in the testing. Testing of each participant generally took one to two hours. Based on the results of the testing, recommendations for improving the programs were identified and revised versions of the diagnostic programs will be completed prior to their final distribution on CD-ROM.

Strongly Agree

Somewhat Agree

Neutral

Somewhat Disagree

Strongly Disagree

Figure 3.1 Likert-type scale that uses a verbal continuum of response anchors.

Results and Discussion

System Design

Each system was developed using a diagnostic/prescriptive paradigm in which the user is asked to identify symptoms or recognizable characteristics of a unknown problem by morphological region of a tree. The diagnostic systems require the user to make several decisions by identifying the region(s) within a tree which are expressing unique symptoms, classes, and identifying unique symptoms which are being expressed by specific symptom classes (APPENDIX F). As each choice from a menu is selected using a mouse, the diagnostic system prompts the next window to appear until the user selects a symptom or set of symptoms and decides to proceed with the search query (APPENDIX G). Color images which illustrate visual symptoms are also available to the user (APPENDIX H). These images can be viewed, by using a mouse and activating an icon, to assist the user in clarifying text descriptions of symptoms or by visually matching identified symptoms. The final goal is to generate either a single conclusion or a list of possible conclusions that successfully match the symptoms selected by the user. At the end of the consultation session a user executes the search for the consultation by using a search query generated by the diagnostic system. The query activates the search function and rapidly reaches either a specified conclusion, a set of conclusions, or possibly admits it cannot reach a conclusion based on the information provided by the user (APPENDIX I). Optionally, the user can specify additional or omit criteria for searching, and a search query will reach a new conclusion.

In the instance when a conclusion is reached, the list of identified causal agents can be displayed. The user can also subsequently browse the summary documents for each conclusion, and view detailed information which may assist the user in confirming the conclusion before utilizing the hypertext function and linking to the more detailed IFAS extension documents which sometimes contain control recommendations (APPENDIX J). A total of 56 IFAS publications are available for CIT·Xpert and 75 for TFRUIT·Xpert. All summary documents and extension documents can be printed. The user can return to previous menus allowing for multiple diagnostic sessions without exiting or restarting the program.

CIT·Xpert System Testing

Twelve outside male experts participated in the testing procedure. Unfortunately, due to problems in coordinating participation from growers and county extension agents, the citrus testing procedure was based primarily upon IFAS research or extension specialist participation. Fifty percent of the 12 participants were 50 to 59 years of age, 48% were 40 to 49 years of age while only 8% were 30 to 39 years of age. Ten participants had completed a Ph.D. program while the remaining 2 had completed a masters degree program. Two participants were county extension agents, 2 were extension faculty specialists, 6 were CREC research faculty and one was a citrus consultant/commercial producer.

Thirty-three percent had 5 to 10 years of experience, 8% had 10 to 15 years of experience, 59% had greater than 20 years of experience. Their responses to the degree of familiarity with diagnosing diseases and disorders included 8% extremely familiar, 25% very familiar, 16% generally familiar, 33% characterized themselves as vaguely familiar, and 8% were not familiar.

When asked about how often their management, research or teaching time included work in the area of diagnosing citrus diseases, disorders and pest damage problems, 8% responded always, 50% often, 25% occasionally, and 17% rarely.

Computer Experience

Eighty-three percent owned or had access to a 386 or 486 personal computer with a mouse and CD-ROM and 67% owned or had access to a laser printer. When questioned which of the following responses best described their training in the use ofcomputers 17% have never received training, 50% primarily taught themselves, 25% primarily received training from someone else and 8% received training during higher education.

Hands-on Evaluation

Each participant to completed a diagnosis of the ten real-world test case scenarios during the allotted test period (Table 3.4). The largest obstacle was the ability of the participant to carefully examine and correctly identify the primary symptom of each case and to exclude other nonrelated symptoms. Participants were instructed not to overlook the possibility of several causal agents occurring together. In some instances this

Table 3.4 Real-world cases presented to participants to evaluate CIT·Xpert.

Case #	Fruit crop commodity
	Morphological region	Disease, disorder or pest
1-1	Citrus Fruit	Fruit Splitting
1-2	Citrus Leaves	Citrus Snow Scale
1-3	Citrus Leaves	Katydid
1-4	Citrus Roots	Citrus Root Weevil
1-5	Citrus Leaves	Zinc Deficiency
1-6	Citrus Leaves	Citrus Leafminer
1-7	Citrus Leaves	Aphids
1-8	Citrus Leaves	Magnesium Deficiency
1-9	Citrus Fruit	Scab
1-10	Citrus Twigs/Stem	Caribbean Black Scale

resulted in experts jumping to a conclusion based on field case histories and not proceeding through the system rationally and following steps in diagnosing a disease or disorder. Experts were able to successfully diagnose an average of 84.2 percent of the cases correctly and 10.8 percent incorrectly. Five percent of the diagnoses were uncertain based solely upon their subjective knowledge (Table 3.5). When the same experts utilized CIT·Xpert, the percent of correct identification decreased slightly by 7.5 percent to 76.7 percent correct, the incorrect responses increased 3.4 percent to 14.2 percent, and uncertain responses increased 4.2 percent to 9.2 percent (Table 3.5). The results of the testing indicated the performance of CIT·Xpert in correctly identifying the causal agent was only slightly lower than the performance of the experts without using the aid of the diagnostic expert system. In contrast, the ability of the computer-based diagnosis increased slightly with reference to the percent of incorrect diagnoses and decreased in reference to uncertain responses. These difference were not considered to be significant.

Table 3.5 Percent responses to the diagnosis of each real-world test case for CIT·Xpert.

	Subjective responses			Computer-based responses
Case #	correct %	incorrect %	uncertain %	correct %	incorrect %	uncertain%
1-1	92y	-	8.3	83.3	-	16.7
1-2	100.0y	-	-	100.0y	-	-
1-3	91.7y	8.3	-	100.0y	-	-
1-4	100.0y	-	-	91.7y	8.3	-
1-5	33.3	58.3y	8.3	-	58.3y	41.7
1-6	100.0y	-	-	75.0y	16.3	8.3
1-7	83.3y	8.3	8.3	75.0y	16.7	8.3
1-8	83.3y	8.3	8.3	66.7y	33.3	-
1-9	66.7y	25.0	8.3	83.3y	8.3	8.3
1-10	91.7y	-	8.3	91.7y	-	8.3
	84.2yz	10.8z	5.0z	76.7yz	14.2z	9.2z

z Mean percentages
y Highest response given

Interface Design/Ease of Use

The majority of the participants strongly agreed that the on-screen type was easy to read and the appearance of the graphic illustrations were good as well as the selection of the screen colors (Table 3.6). Only 42% agreed the system was easy to navigate while 58% indicated the diagnostic expert system was easy to use (Table 3.6). Seventy-five percent agreed the system seemed to be accurate, 75% agreed they could make a diagnosis quickly whereas 83% indicated they should be able to find information rapidly by using CIT·Xpert (Table 3.6).

Content of CIT·Xpert Diagnostic Expert System

Nearly all of the participants agreed the information in the diagnostic expert system was accurate and current (Table 3.7). Seventy-five percent responded the supplementary

Table 3.6 Percent responses to the questions regarding interface design and the ease of use of CIT·Xpert.

Question number	Strongly agree	Somewhat agree	Neutral	Somewhat disagree	Strongly disagree
E.18	75z	8	17	0	0
E.19	67z	33	0	0	0
E.20	50	50	0	0	0
E.21	17	25	33	17	8
E.22	8	50z	17	17	8
E.23	8	67z	0	17	8
E.24	33y	42z	25	0	0
E.25	33	50z	17	0	0

z Response most given.

summary documents were helpful in making a diagnosis (Table 3.7). In the absence of an expert to provide technical assistance regarding diagnostic capabilities, 92% indicated the diagnostic expert system could effectively guide the process (Table 7.0). When asked what the maximum amount in dollars that they would be willing to pay for the CD-ROM 1 responded $1 - 25.00, 3 replied $26-50.00, 3 - $51-75.00 and 3 - $76-100.00. When questioned how often the diagnostic system should be updated and released, 1 participant responded every 6 months, 3 responded every year, 3 responded every two years, 3 responded every five years, and 1 responded as needed being assessed by a committee. Finally, 75% of the participants indicated they would be willing to serve on an editorial board whose function would be to review new information, documents or images before they would be added to the diagnostic expert system.

TFRUIT·Xpert System Testing

Ten participants took part in the testing procedure of the TFRUIT·Xpert diagnostic system and these were separated into three independent groups which included 4 commercial growers, 3 county extension agents and 4 IFAS specialists or "experts". Although 1 individual from the "expert" group was actually employed as a county extension agent, the individual possessed a Ph.D. and more than 20 years of experience and was therefore considered as an "expert" for the purpose of the testing procedure.

Table 3.7 Percent responses to questions regarding informational content of CIT·Xpert.

Question number	Strongly agree	Somewhat agree	Neutral	Somewhat disagree	Strongly disagree
E.26	17	58z	25	0	0
E.27	33	67z	0	0	0
E.28	33	42z	25	0	0

z Response most given.

Twelve real-world samples were collected from nearby orchards, and included two samples for each of the six fruit crop commodities. The twelve real-world cases presented to participants to evaluate TFRUIT·Xpert system are presented in Table 3.8. The following discussion is a summary of the results of the testing.

Commercial Growers

Four commercial growers participated and included 1 female (40-49 years of age) and 3 male commercial growers whose ages ranged between 25-29 and one being over 50 years of age. With the exception of 1 grower who had 5-10 years experience involving the diagnosing of diseases and disorders of subtropical and tropical fruit crops, all of the remaining growers had less than 5 years experience. Only two indicated they were very familiar with identifying visual field symptoms. Two growers had received BS. degrees while the remaining two had completed a high school education. Only two growers had received prior training in the use of computers and owned or had access to a personal computer with a CD-ROM which could run the diagnostic system. Only one grower had access to a laser printer required for printing summary or extension documents.

The results of the testing indicated that when the growers relied solely upon their subjective knowledge and experience they were able to correctly identify 29 percent of the real-world cases, they misidentified 31 percent of the real-world cases and could not make a diagnosis for 40 percent of the cases (Table 3.9). In contrast, when the growers utilized the TFRUIT·Xpert system they correctly identified one-half of the cases, misidentiifed 46 percent, while the remaining 4 percent of cases could not be diagnosed (Table 3.9).

Table 3.8 Real-world test cases presented to participants to evaluate TFRUIT·Xpert.

Case #	Fruit crop commodity
	Morphological region	Disease, disorder or pest
2-1	Carambola Leaves	Iron Deficiency
2-2	'Tahiti' Lime Leaves	Citrus Leafminer
2-3	Lychee Leaves	Nitrogen Deficiency
2-4	'Tahiti' Lime Fruit	Stylar-end Breakdown
2-5	Lychee Leaves	Iron Deficiency
2-6	Papaya Fruit	Anthracnose
2-7	Avocado Leaves	Powdery Mildew
2-8	Carambola Fruit	Wind Stress/Mechanical Damage
2-9	Mango Leaves	Zinc Deficiency
2-10	Avocado Fruit	Scab
2-11	Mango Leaves	Anthracnose
2-12	Papaya Leaves	Papaya Ringspot Virus

County Extension Agents

Three IFAS extension agents participated in the testing including 2 females (50-59 years of age) and 1 male (40-49 years of age). All of the county extension agents had received higher formal academic training than that of the growers but were not as qualified as the "Experts". All had completed a four year bachelors of science college degree with 2 possessing a masters degree. One had 5 to 10 years of experience in diagnosing tropical fruit disease, disorders and pest problems, while the remaining had less than 5 years experience. Their responses to the degree of familiarity with diagnosing diseases and disorders included 1 generally familiar and the remaining 2 were vaguely familiar. When asked about how often their management, research, or teaching time included work in the area of diagnosing plant diseases, disorders and pest damage problems, 1 responded often and 2 indicated occasionally. All extension agents responded that they either owned or had access to a 386 or 486 personal computer with a mouse and CD-ROM and all 3 also had access to a laser printer. When questioned which of the following responses best described their training in the use of computers, 1 have never received training, while the remaining 2 others had primarily taught themselves.

The results of the testing indicated that when the extension agents utilized their subjective knowledge and experience they were able to correctly identify 67 percent of the real-world cases, they misidentified 22 percent of the cases and could not make a diagnosis for 11 percent of the cases (Table 3.9). In contrast, when the extension agents used the TFRUIT·Xpert system, they correctly identified 61 percent of the cases, misidentiifed 33 percent, while 6 percent of the cases could not be diagnosed (Table 3.9).

Table 3.9 Percent responses to the real-world test cases presented to each test group for TFRUIT·Xpert.

Test Groups	Growers
	% correct diagnosis	% incorrect diagnosis	% uncertain
Subjective diagnosis	29	31	40
TFRUIT·Xpert diagnosis	50	46	4
% Change	+21	+15	-36
	County Extension Agents
Subjective diagnosis	67	22	11
TFRUIT·Xpert diagnosis	67	33	6
% Change	-6	+11	-5
	"Experts"
Subjective diagnosis	70	30	0
TFRUIT·Xpert diagnosis	64	36	0
% Change	-6	+6	-
	Groups Combined
Subjective diagnosis	55.3	27.7	17
TFRUIT·Xpert diagnosis	58.3	38.3	3.3
% Change	+3	+9.4	-13.7

Experts

The three participating experts were all male with 2 being older than 60 years of age and 1 being 30 -39 years of age. All experts had completed a Ph.D. degree. The results of the testing showed that when the experts used their subjective knowledge and experience they were able to correctly identify 70 percent of the real-world cases, misidentified 30 percent of the cases (Table 3.9). In contrast, when the experts used the TFRUIT·Xpert system they correctly identified 64 percent of the cases, misidentified 36 percent of the cases (Table 3.9). In neither the subjective nor computer-based diagnoses did the expert fail to reach a conclusion.

Interface Design/Ease of Use

Based on the results of the testing the majority of the participants strongly agreed that the on-screen type was easy to read (70%) and the appearance of the graphic illustrations were good (60%) as well as the selection of the screen colors (60%) (Table 3.10). Sixty percent also strongly or some what agreed the system was easy to navigate as

well as indicated the diagnostic expert system was easy to use (Table 3.10). One-half of the participants agreed the system seemed to be accurate, however only 40% agreed they could make a diagnosis quickly. One-half of the participants also indicated they should be able to find the information rapidly by using TFRUIT·Xpert (Table 3.10).

Table 3.10 Percent responses to the questions regarding interface design and the ease of use of TFRUIT·Xpert.

Question number	Strongly agree	Somewhat agree	Neutral	Somewhat disagree	Strongly disagree
E.18	70z	10	0	10	10
E.19	50z	30	0	0	20
E.20	60z	20	0	0	20
E.21	30z	30z	10	0	30z
E.22	20	40z	10	10	20
E.23	20	30z	30z	10	10
E.24	20	20	40z	0	20
E.25	30z	20	20	10	20

z Response most given.

Content of TFRUIT·Xpert Diagnostic System

With the exception of only 1 individual, the participants agreed the information in the diagnostic expert system was accurate and current (Table 3.11). Eighty percent responded the summary documents were helpful in making a diagnosis. When asked if in the absence of an expert to provide technical assistance, the diagnostic expert system can effectively guide the process, 30% strongly agreed while 60% responded "Somewhat agree" (Table 3.11). When asked what the maximum amount in dollars that they would be willing to pay for the CD-ROM 1 responded $1 - 25.00, 4 replied $26-50.00, 1 replied $51-75.00 and 3 replied $76-100.00 and 1 suggested the CD-ROM be distributed free of charge. When questioned how often the

Table 3.11 Percent responses to the questions regarding informational content of TFRUIT·Xpert.

Question number	Strongly agree	Somewhat agree	Neutral	Somewhat disagree	Strongly disagree
E.26	20	60z	10	0	10
E.27	60z	30	0	0	10
E.28	60z	20	10	0	10

z Response most given.

diagnostic system should be updated and released, 4 responded every year, 3 responded every two years, 2 responded every three years, and 2 responded every five years. Finally, six of the ten participants indicated they would be willing to serve on an editorial board whose function would be to review new information, documents or images before they would be added to the diagnostic expert system.