Development of standardized evaluation tools for assessing clinical competence of third year medical students rotating in the Department of Pediatrics, Dela Salle University Medical Center - Caparas, Delia S.
Assessment of clinical competence is generally based on observed performance of skills. In the present setup of the Department of Pediatrics, Dela Salle University College of Medicine, evaluation consists of a single tool accomplished by only one rater, the preceptor. The existing evaluation tool provided variable and incomplete data which was not effective in identifying the competencies of the students. The criteria used were generalized, non-measureable and un-standardized with emphasis on the cognitive component. This study was done to identify the overall competencies of the students, assessing both the product and actual performance of skills encompassing the three domains of learning. The objective is to formulate a valid, reliable, generalizable and practical evaluation tool utilizing the appropriate raters in order to come up with a fair and objective manner of assessing the clinical competence of third year students who rotated in the Department of Pediatrics. This is a descriptive developmental study conducted to three sets of respondents ?60 third year medical students, 20 preceptors (Pediatric Consultants) and 20 patient's caretakers. They were chosen by purposive sampling as they were manning the wards during the data collection procedure of the researcher. Four sets of evaluation tools were constructed using rating scales. Field testing was done to correct ambiguous items. The data was then subjected to tests of validity, reliability, generalizability and practicality to come up with standardized assessment tools. Actual collection of data was done from October to December 2000. A thorough review of the official curricula and appropriate documents were reviewed together with other literature. Major constructs of clinical competence were identified and included: ?eliciting pediatric history?, ?performing appropriate physical examination?, ?formulating a logical diagnosis based on gathered data and physical examination?, ?formulating differential diagnosis?, ?selecting appropriate diagnostic work-up?, ?justifying a chosen therapeutic plan of management? and ?interpersonal skill.? Further analysis showed that year level 5's emphasis is on the formulation of primary and differential diagnosis. These constructs were broken down into specific behavioral items and were distributed according to the proportional weight of each construct earlier mentioned. The rating scales were developed and the most appropriate raters were identified. They referred to the self and peer because there is active interaction among students and the length of exposure when they work as a group during the first session. On the other hand, the preceptor's assessment is based mainly on the discussion during the second session. The cognitive areas, primarily the formulation of primary and differential diagnosis, are best rated by the preceptors being the content experts, the emphasis being on the higher hierarchy of learning. The noncognitive areas are assessed jointly by self, peer and caretakers. These include communication skill, listening ability and feeling of empathy. To establish construct and content validity, the completed rating scales for preceptors, patient, self and peer were presented to 5 content experts in Pediatrics. Through this Delphi technique, the instruments were revised thrice upon the recommendation of the experts. To determine interrater reliability, the data was subjected to statistical analysis utilizing kappa-statistic measure of agreement set at p<.05 level of confidence. Results showed a significant degree of agreement among the raters given to the students. This further indicates a strong interrater reliability in the responses of the raters. Results show a fairly acceptable level of agreement. The tools are generalizable based on the homogeneity of the subjects and the sample size which is more than the statistically required number. It is also practical because of the ease of administration and scoring, economical and user-friendly. This study shows that the evaluation of clinical competence of students involves several raters who will assess different areas of competency encompassing the three domains of learning. The results can be used to provide feedback to both students and faculty to improve clinical evaluation of students performance during ward work.