Students often learn from their errors on exams, why is it that only students who fail or borderline pass are privileged to see their exam?
Why are students allowed to see their exam only once?
- These restrictions are in place to support an exam sequestration policy. In turn, this policy serves the primary goal of the exams: confidence on the part of the faculty that our students are qualified to proceed to the next phase of their education based on having learned the material already presented to them. Ultimately, this confidence must extend to the target of all this education, your patient. Access to prior exams almost invariably leads to learning based on the content of old exams. The second, and laudable goal of exams—as a learning experience is, indeed sacrificed somewhat for this primary goal. Clearly, the construction of totally new exams each year, without sequestration, would obviate this problem, but creates new one in that ‘Untested’ questions are often poor or ambiguous questions that need to be dealt with after the exam. Perhaps more important, open access to the exams led to hoarding of them by some students creating two groups of students: the haves and the have-nots. Finally, reliance on old exams for studying does not create the solid knowledge base necessary for USLME passage Practically speaking, we know these issues are real because we have not always had a sequestration policy.
Why are students not allowed to view their failed exam less than two weeks before the “make-up” exam?
- In a well-structured assessment process, a student would never see a specific question twice so memorizing the answer to a specific question wouldn’t help. But most students believe there will be similar questions on the same topic so memorizing the answer to one question will help in answering some future question. This may even be true but it only serves the purpose of passing an exam and does not advance mastery of the subject. The purpose of the exam review is to look beyond the topics see if there is a pattern to why these questions were answered incorrectly. Perhaps they are all related to a specific set of lectures or perhaps they are all related to mathematical formulas or perhaps they all involve 3D spatial reasoning? This is a oader objective that will help a student master the subject as well as avoid bad study habits that may have an effect on mastery of other subjects. If a student can’t discern these patterns in one review of the exam, they will not discern them in multiple reviews. Meanwhile, multiple reviews would only reinforce the idea of memorizing specific questions, an approach that we believe is counter-productive.
When reviewing the exam, if there is a question about the validity of an answer, what steps can a student take to insure that the answer is correct?
- As mentioned above, the purpose of the review is to help correct systematic problems with a student’s approach to studying the material. A review just before the make-up reinforces the question memorization approach that we feel is counter-productive. Furthermore, if a student has delayed studying for a make-up until 2 weeks before, they have some more basic problems than just mastering material for one course. By having a deadline we are encouraging borderline pass students to be more organized in their approach to studying.
What measures does the Teaching Office take to avoid “human error” or “ambiguity” in scoring?
- If you believe there is an error in the validity of an answer, report this to the person who is administering the review. They will report it to the Teaching Office and the problem will be investigated. It is of interest to note that to date, no investigations have uncovered any problems in the administration or grading of an exam.
Why are some exams riddled with typographical errors?
- Extensive testing has shown that in thousands of questions scanned, the automatic scoring machine has never substituted one answer for another. That leaves four types of errors to consider: blank answers, unclear answers, answer-key errors, and student ID errors.
The two most common errors related to the automatic scoring machine are answers that are not penciled in heavily enough or answers that have been changed without sufficient erasure. These show up in the scoring process as blank and “?” answers. All scantron sheets with blank and “?” answers are pulled and examined to see if we can determine what the students intent was. Usually it is pretty clear what the problem was. In this case, a staff member either pencils in or erases the problem mark and rescans the sheet. In any case where there is some question of what the student intended, three staff members will consult. Every examination is analyzed and item analysis report is given to faculty members. If there has been an error in transcribing the answer key, this usually becomes obvious as a question will be identified as extremely hard (more than 50% get the question wrong) or ambiguous (a question is answered incorrectly by students with high scores and correctly by students with low scores). In reviewing this report, faculty will often find other questions that are deemed ambiguous and they can be thrown out or allowed to have more than one correct answer. Faculty can also use the ambiguity sheets to identify problematic questions. The item analysis is also reviewed by the Assistant Dean for Educational Informatics before exams are made public. All of the above errors are relatively rare and less than once a year an error will be caught after the grades have been made public. In contrast there is essentially always at least one student who incorrectly enters his or her ID number. When this happens, staff must pull that sheet and manually match the computer record to the correct student. If this is done incorrectly, the result is that a student who took the test does not see his/her grade. A student who did not take the test may also potentially see this grade but since they didn’t take any test, they wouldn’t know to look. Nor could they have any way of knowing whose grade they were seeing even if they saw it. These errors get corrected when students report a problem seeing their grade.
Why are some make-up exams scheduled months away to the actual course?
- Only the course leaders can completely answer this question. The office staff proofread for question numbers, page numbers, and, as far as possible, correct letter choices. They can’t proof read the spelling or sense of the questions since this requires expert knowledge in the subject. Proofreading is boring. Writing exams is boring. There is no doubt that proofreading exams is doubly boring. This is not to excuse the errors but to at least explain them. In a multiple line line sentence, it can be hard to detect a word that repeats at the end of one line and the beginning of the next (e.g. like the word “line ” on the previous two lines). In the same way it is easy for entire questions to repeat without the faculty noticing. Again, this doesn’t excuse the problem but only explains it. As bad as the typographical errors are on some examinations, this has actually been improved since the sequestration policy was put into effect.
Why are some make-up exams scheduled close to the end of the course?
- The answer to this question is closely related to the next question.
Can I have knowledge of how many students failed a course exam?
- The schedule of make-up exams is complicated by the need to accommodate students who are taking both first and second year courses in the same year and the desire to give borderline passing students as much of a chance to perform well as possible. The ideal time for a make-up is neither too close nor too distant from the course. But it must also not fall to close to regular exams being given in either year. And in most cases, make-ups in different courses have to be spread out from each other. If we knew which students would need to take which make-ups, a more coherent schedule course be devised, but there is need to schedule make-ups in advance before the regular course exams have been given. We try to place “mega-course” make-ups with more space between them and other exams. This generally means we can give one mega-course make-up after the winter eak (NSHB), one after the spring eak (MCFM), and the others have to wait till summer (Anatomy, ID). Anatomy has additional complications if a practical is being given and this is usually the longest gap between course exam grading and make-up administration (the Preventive Medicine make-up is given at the beginning of the second year but this is because the final exam is often not graded until late July).
10. What is the purpose of having Pass/Fail system when in the end, students and the Office of Student Affairs see the actual grades?
- We have found that disclosing the number of exam failures in a course to students does not, in and of itself, speak to the factors that may have uniquely influenced their exam performance. If a course leader should disclose to the students the number of failures, it is his or her prerogative. Because there is much more collaborative study in medical school, a large shift in the mean and standard deviation of an exam may reflect patterns of class study or patterns of teaching rather than the intrinsic ease or difficulty of an exam. For example, we have found alternating pattern of performance on exams for what is essentially the same course with the same exam. Our theory is that in one year, most students pass and tell the next year’s students “This course has an easy exam.” The next cohort doesn’t study as hard and more students fail and do poorly and they tell the next cohort “This is a killer course. You better study hard.” These kinds of rumors have more to do with mean fluctuations than almost anything else. We do provide the exam mean and standard deviation. Since the passing level is either 65 or 1.5 standard deviations below the mean you can calculate how many standard deviations 65 is. If it is less than 1.5 standard deviations then roughly 5% of the class has failed. If 65 is more than 1.5 standard deviations below the mean then less than 5% (possibly none) of the class has failed.
If I fail one exam, do I automatically become ineligible for AOA?
- Course leaders employ different methods of evaluating course grades. Assessing the extent to which a student’s academic performance is impeding his/her progress in Basic Sciences is one of the roles of Office of Student Affairs (OSA). The Office of Student Affairs relies on information that bespeaks a student’s difficulties, e.g., academic performance on written and practical exams as well as psychosocial factors. Attendance points for just showing up, although may enable a student to “Pass”, they do not measure academic performance. Being able to monitor a student’s low grades is useful information. A pattern of high grades is less informative. The difference between a student who consistently gets 80’s and 70’s is marginal.
If I fail an exam, does this mean I will get dismissed?
- No. There are many factors that can contribute to exam performance. AOA candidates are determined on a case-by-case basis by the Office of Student Affairs.
Will clerkship directors have access to my actual grades, to include record of my exam failure?
- No. As outlined in the student by-laws, “Failing three exams provokes ‘advisory status’ for a student who will be formally counseled by the Office of Student Affairs.” Four or five failed exams provokes ‘review’ status and further counseling. The sixth exam failure triggers a ‘probationary status’ and provokes appearance before the Committee on Student Promotions and Professional Standards. Seven exam failures are grounds for dismissal. Eight failed exams requires that a student be dismissed.
When I apply for residency, will my ‘F’ be on my transcripts?
- No. Clerkship Directors and Course Leaders have access only to grades in their own courses. No transcript information is available to faculty.
- The school recognizes that moving from college to medical school represents a substantial shift study habits and tries to allow for that by having a very lenient approach to how grades are represented on the transcript for first year courses and then a progressively less lenient approach for second and third year grades. A course that is initially failed but then passed on the make-up is represented as a “P”. A course that is failed initially and on the make-up is represented as an “F” but the “F” on first year courses will be replaced by a “P” if the course is passed in the subsequent year. Second year failures (after make-up) are not erased, but the eventual passing grade is added. For a complete understanding of what will appear on the transcript, it is necessary to read the By-Laws on the Committee on Student Promotions and Professional Standards.
For more information regarding the Basic Examination Review Policy, please click here