Effect of Feedback on Direct Observation of Procedural Skills in Estimation of RBC Count among First Year Medical Students

Background: Formative evaluation has been effective in promoting active learning and a better future performance in students after immediate personalized feedback. Aim and Objectives: To analyze the effect of “Direct observation of procedural skills” (DOPS) assessment and immediate feedback for first year MBBS students in estimation of RBC Count. Methodology: The study was done on first year MBBS students (n=120) during the Physiology practical class. Checklist for estimation of RBC count (15 steps) was prepared by six faculty members. The formative practical test was divided into two practical sessions. During the first session, each faculty observed 4 students performing RBC estimation one by one (DOPS) and the details were recorded in the checklist. Similarly in five practical classes 120 participants were evaluated and immediate feedback was given to them for two minutes. In the second session the same procedure was followed and results were compared. Students perception about the assessment process (DOPS) was documented by feedback questionnaire as per Likert scale. Results: The mean practical marks of the participants before feedback was 8.32 ± 2.25 and after feedback was 10.18 ± 1.81 (P<0.05). 92% of the study participants felt DOPS in RBC estimation was useful to them Conclusion: DOPS assessments worked well in the student’s practical laboratory setting by serving as an effective educational tool. Immediate personalized feedback to the students has resulted in reasonable skill development both subjectively and objectively.


Introduction
In India, Physiology is taught in the first year MBBS (Bachelor of Medicine and Bachelor of Surgery) course. The aim of teaching Physiology practical's is to facilitate better understanding of the principles of Physiology and to inculcate the basics of clinical medicine in the future doctors 1 .
According to Bloom, psychomotor domain deals with acquisition of physical abilities, motor skills, manipulation of materials or acts requiring neuromuscular coordination. This domain is also referred to as the "domain of practical skills". 2 DOPS (Direct observation of procedural skills) is a preferred assessment tool in a student's skills-lab, which helps to furnish high performance of clinical skills and it works well in the undergraduate setting 3 . DOPS ensures that a particular skill is performed correctly in stepwise manner according to the prescribed guidelines using an agreed checklist. DOPS is useful for ongoing educational and training purposes as it determines the student's level of knowledge, skills and attitude.
A single practical examination at the end of an academic year (summative evaluation) does not fully assess a student and there is no scope for improvement. There arises a need for formative evaluation periodically along with constructive feedback.
While doing estimation of RBC count there are high chances for errors for example, sucking of too much of blood or diluting fluid may lead to abnormal increase or decrease in RBC value respectively. Similarly, undercharged or overcharged chamber may result in erroneous RBC estimation 4 . A Strong mentoring system for practical sessions accompanying comprehensive assessment program is a key to improve the professional competence.
Feedback is considered as a vital approach to facilitate student's development in order to monitor, evaluate and regulate their own learning 5 . Feedback can be considered as constructive in the process of learning if it is delivered immediately and in a sensitive manner 6 . Appropriate feedback during education helps learner to identify the deficiencies in their learning and correct them towards achieving the objectives at the end of the program.

Rationale:
There are very few studies in Indian literature about formative assessment, DOPS in improving hematology skills of first year medical students. The present study was done to assess the student's practical skills and to assess the effect of evaluation using DOPS while performing RBC count in Physiology laboratory.

Aims and objective:
1. To analyze the effect of immediate feedback on hematology practical examination.

2.
To assess the effect of formative evaluation using DOPS during the first year Physiology practical.

Materials and Methodology
This Single arm interventional study was done at the department of Physiology-after obtaining Institutional Ethical Committee clearance. Participants (n=120) for the study were recruited from first MBBS after their voluntary consent.
Checklist with fifteen questions for estimation of RBC count was prepared with reference from standard Physiology practical manual 7 and validated by all the teaching faculty in the Physiology department. Six faculty members willing to participate were chosen. Faculties discussed among themselves the steps to be demonstrated as per the checklist.
The formative practical test was divided into two practical sessions. During the first practical session, 6 faculties demonstrated the RBC estimation (one faculty for four students). After demonstration, each faculty observed a student performing RBC estimation for half an hour on one to one basis (DOPS) and the details were recorded in the checklist. Students were awarded one mark for each correct step. Similarly in five practical classes 120 participants were evaluated and immediate feedback was given to them . In the second session the same procedure was followed and results were compared.
Feedback questionnaire (using five-point Likert scale) was provided to all the participants to find out their perception regarding the assessment process (DOPS) and the need for feedback in hematology practical's.

Statistical Analysis:
Pretest and post test scores of the participants were analyzed by students paired t-test using SPSS 24-0 version. The feedback score was measured using proportion and percentages. The check list had 15 items and the students were assessed on dichotomous scale.  Table 1 shows that the mean practical marks of the participants before feedback was 8.32 ± 2.25 and after feedback was 10.18 ± 1.81. A statistically significant difference in marks of pretest and posttest were seen in the study participants. Difference between the pretest and posttest marks is 2.25 in boys and 1.34 in girls. Though the pretest and posttest marks were more in girls, improvement ie., percentage gained in marks was more observed in boys (24.40%) than girls (11.71%). Table-2, shows that 92% of the study participants felt DOPS in RBC estimation is useful. 94% of the students were of the opinion, that the process was made clear and the task was properly explained. 91% of the students agreed that immediate personalized feedback was more effective and helped them to identify their competency gaps during the session. 92% of them perceived that the feedback sessions after DOPS motivated them for further learning. 92% of them felt that the instructor effectively directed and stimulated the student.

Discussion
The aim of the practical Physiology curriculum is to provide the students with planned educational experience on Physiological principles that will enable them to be a good practicing physicians 1. According to Millers pyramid of clinical competence, the domains at which a trainee might be assessed are, knows (recall the procedure in order), knows how (ability to describe the need for the individual steps), shows how (demonstration of skills in a controlled setting), Does (performance integrated in to practice). For each of the above levels, the student can apply the knowledge and principles to the skill performed 8 .
Though formative assessments are popular in medical education, data to establish their educational benefits are lacking 9 . In this study psychomotor objective is supervised by DOPS clinical tests. The process of observing the students during the RBC estimation and awarding marks for the steps involved, renders the gap analysis, ie., what is expected and what has been done. This is the important facet of DOPS which distinguishes it from other forms of assessments, where the results are only scrutinized.
The students' performance in the session 2 showed a greater improvement, which clearly depicts the satisfactory improvement in the student's competency. Our study findings were supported by Wigton and Boehler, who had documented that the immediate feedback has been shown to improve students' performance in medical schools. 10,11 Hattie and Heleb also had proved that immediate feedback is more effective for student learning 12 .
As per table 2, DOPS assessments motivated the undergraduates for performing practical indubitably. The present curriculum is not likely to change soon, so there is a need to identify those lacunae in hematological skills and correct them. During OSPE sessions only certain part of the experiment can be assessed, but in DOPS all the individual steps in an experiment are given equal importance.
Strength of our study is that confidentiality and privacy was maintained while giving feedback which may lead to futuristic educational gains. Limitations of this study are 1 . There is no comparison group 2 . There are chances of observer bias as same faculty members are assessing the students 3 . The verbal feedback cannot be always be retained in the listeners memory and sometimes may lead to misunderstanding.
Self-realization and reflection are the two fundamental steps to receive feedback and to work on it 13 . This technique (DOPS) helped the students to fill the competency gap and thereby improve their practical skills. This will take Physiology from bench to bedside and truly inculcate the basics of clinical medicine in tomorrow's physicians of first contact. 14

Conclusion:
From this study we infer that structured teaching and DOPS with assessment and feedback provides a better learning environment to the students. It helps to assess the student's skill and helps to identify the competency gap. Therefore, formative evaluation using DOPS with feedback results in better learning of practical skills than routine traditional practical teaching process.
Funding Source: Self. 14. Paralikar S, Shah C, Faculty perceptions of the strengths, weaknesses and future prospects of the current medical undergraduate experimental physiology curriculum in Gujarat, India Indian J Pharmacol 2015 59(1):109-116.