Method In Medical Education

Published Date: 02 Nov 2017

Abstract

Introduction

Learning in a very crucial process while assessment is an indispensible part of it. Assessment plays a crucial role in both training and education. Students understand best when all learning activities and related events are chosen carefully in order to enhance the skills they have already acquired or to challenge them to learn new skills. Assessing students gives teachers and tutors an overview of students understanding of what they have learnt and what the students failed to understand comprehensively. This enables them to plan for more learning programs or to streamline future activities in order to address the weaknesses. They are also able to review whether the appropriate teaching methods are in use and the units that need to be revisited. Therefore, assessments can be defined as the process of evidence collection to gauge the level of understanding of students and to provide a locus from which the teaching methods are readjusted and furnished.

This shifts the focus to what the students are able to demonstrate or what they understood, rather than just listing what was covered in class. Assessment is a continuous process aimed at: establishment of measurable and clear expected outcomes as a result of student learning, ensuring that each and every student has sufficient opportunities to affect those outcomes. Assessment also gathers, analyzes and interprets evidence acquired to decide on how well the learning matches with the expected outcomes. Finally, assessments use the resultant information obtained to understand and work on how to improve the student’s performance and understanding in a particular area. In higher education, assessment forms a central element in overall quality of learning and teaching.

This assessment in higher education helps in shaping the experience of students thus, influencing their behavior. So, for proper assessment to be carried out, some steps have to be followed so as to ensure that the whole process is successful. First of all, the examiner has to clear learning outcomes, then check whether these outcomes align with the curriculum, after that, the examiner should develop an appropriate assessment plan. This includes choosing the most reliable and adequate method of assessment. After the assessment, the data obtained should be put together and then utilized to improve the program. Since assessment is a continuous process, routine examination of the process should be done and any arising matters addressed adequately. To ensure this process is successful, assessments should be accurate, useful, fair, truthful, systemized, cost effective and ethical.

Types of assessment methods

Since assessment is a non avoidable, end product chore used to evaluate, box and measure students, it is therefore advisable to adopt student assessment methods that are innovative and beneficial to students. These approaches to evaluate the students must be effective so that they can produce desired outcomes. Generally, assessment methods can be classified into three major groups. These include:

Summative assessment

These are the most commonly used forms of assessment. Under this we have tests, final projects and examinations aimed at measuring growth and achievement in part of the students. The teacher here measures the understanding of the learning goals. This provides important information and data to the teachers regarding the progress of the students.

Diagnostic assessment

This form or method of assessment is used in determining the mastery of new skills and knowledge and any future content in terms of knowledge.

Formative assessment

This method of assessment is used to gauge the student’s progress through out a lesson or unit.

Under these three broad categories, there exist various methods and ways of evaluating students. These include:

Cases and open problems

In this method of assessment, students carry out an intensive and detailed analysis of a given specific example.

Computer based assessment

Under computer based assessment, students use computers in a bid to support their assessments

Essays

This includes work written by the students whereby they try out various arguments and ideas and then try to support these ideas and arguments using evidence.

Learning logs

These vary widely and they may range from structured forms based on a given task to unstructured accounts of each day.

Mini practicals

These involve a series of short and comprehensive practicals undertaken usually under timed conditions. Assessment may also occur under authentic settings in cases of assessment of practical skills.

Modified essay questions

Under this method, sequences of questions related to a certain case are asked. After answering one question, more information is provided on the same and more questions related to the case asked.

Orals

This includes verbal examination and interaction between the examiner and the examinee.

Objective structured clinical examinations

In this method, the students are measured under exam conditions based on their reaction to a given series of practical, short, real life situations.

Portfolios

These are systematic collections of work or educational products which are collected over a given period of time. They may include a large collection of assignments or certain reflections on critical issues.

Poster sessions

This is where results from an investigative project are displayed on posters.

Presentations

These are oral reports on investigative activities or projects

Problems

These ones measure a given application, analyze it and come up with problem solving strategies.

Group projects and strategies

This involves assessment by a lecturer or tutor on the results of a group work of students.

Report forms and questionnaires

This represents one or more questions that have been answered and presented together.

Reflective practice assignments.

These assessments measure the capacity of the students to evaluate and analyze experiences in regard to research evidence and theories.

Reports on practicals

These are written methodically so as to account for a practical done by the students.

Self assessed questions which are based on open distance learning

This is more of a learning method than an assessment one. In this method, the assessment instruments are self administered and are meant for specific purposes like providing feedback on performance of the student. It also provides prescription and diagnosis recommendations unlike the others where it is either pass or fail.

Short answer questions

These are short answer questions used to measure application of knowledge, analysis evaluative and problem solving problems.

Simulated interviews

These are majorly useful for assessment of oral communication skills.

Single essay examinations

These are aimed at assessing students on a given topic. They are done within a specified timed period like three hours.

Work based assessment

This form of assessment employs a wide variety of methods to assess the student. These include use of log books, projects, portfolios, and structured reports from the mentors or supervisors.

Multiple choice questions

This involves series short structured questions with multiple answers whereby the examinee is supposed to either pick the best answer or state whether true or false for all choices.

This article will take a detailed look at multiple choice questions as a method of assessment of students, how it is applied to assess the students, its advantages and disadvantages, how to come up with proper multiple choice questions, the impact this method has on learning especially on the medical students and what is special in using it in the medical field especially as a tool of examining the patients.

Multiple Choice Questions

As stated earlier, this is a form of assessment whereby the students or examinees are required to select or mark the best answer out of a given set of choices. This form of testing is among the most frequently used in learning institutions to test the students. Also, it finds application in election processes, whereby the voters are required to select their preferred candidates, policies or parties.

The structure of multiple choice questions

Multiple choice questions mainly consist of two parts: the stem which is essentially the question itself and a set of choices from which the examinee is supposed to choose the right or best possible option. The stem forms the first part of the structure. It presents the whole item or question as the problem which needs to be solved, or a question raised which needs to be answered or a statement which is incomplete and needs to be completed. The options given in the multiple choice questions are possible answers from which the examiner is expected to choose. One of them is the most appropriate while the others serve as distracters or incorrect choices. This is different from those whereby different answers can be keyed in as correct options.

In this assessment, the correct answer usually earns a given number of points in relation to the total mark while the incorrect answers or choices do not earn one any marks. In other cases, the test may award some credit for any unanswered questions, or sometimes penalize the student for choosing the wrong options hence, discouraging them from guessing. For example, some deduct half a mark for every wrong option given. In cases or advanced items like some applied knowledge items, the stem of the question may consist of quite a number of parts. The stem may have ancillary or extended material like a case study, vignette, or a graph, a detailed description of a number of elements, and a table related to the main question. Anything can be included to the stem as long as it proves to be necessary in ensuring the utmost authenticity and validity to the mentioned item. The stem usually has a lead in question which explains how the examinee should answer. For example, what pathogen is likely to be responsible for the disease?

An example of a multiple choice question

If a=2 and b=4. What is b - a?

A. 7

B. 2

C. 4

D. 3

E. 1

A well drafted multiple choice question usually avoids wrong distracters. This makes the question have some sense when being read with the correct answer as well as with the distracters.

Advantages of multiple choice questions

Versatility

This method of assessment is suitable for use in a wide range of subject matters and therefore can be employed to measure different educational objectives. This method is also adaptable to many levels of outcomes from simple remembering of things to complex levels of reasoning. It helps to measure the students’ ability to analyze things, apply learned principles to new situations, understand principles and concepts, differentiate between facts and opinions, interpret various relationships arising, interpret graphs and charts, decide whether a given set of information is right or wrong, make inferences and solve problems given.

This method of assessment has various advantages. If the examinees are offered proper training, then quality of the work is assured. Hence, this makes it a very effective way of assessing students. If the examinees are given proper instructions on the by which the examination format works, and any arising matter the test corrected, then performance in that test will be better. On repeated assessment using this method, its reliability has been proven to actually improve with use of large numbers of questions in a given test, with great care taken in case specificity and good sampling. This increases overall reliability of the test.

Another advantage of multiple choice questions is the ability to assess over a large area of the curriculum. Since no explanations are required in this method of assessment, a considerably larger area of the syllabus can be covered. The teacher thus gets a chance to see whether the students understood well the areas covered.

Another advantage of this method of assessment is that it reduces the workload of the marker. Here, the answer is either right or wrong, and there are no explanations for the choice given by the students. This makes marking the scripts an easier job compared to those assessment methods whereby a lot of explanation is required. In regard to this point, the reliability and accuracy of marking is considered to be high as only one choice is correct.

Compared to other methods of assessment, multiple choice questions usually require less amount of time to administer. This in turn provides a more comprehensive form of evaluation and assesses the extent of knowledge of the examinee. To increase the efficiency of this method, online examinations delivery softwares can be used. This in turn provides a good platform whereby the student’s knowledge is gauged by this "good measuring tool." They are also more affordable when testing a large group of students.

Multiple choice items give themselves to the growth and enhancement of the assessment objectives. Therefore, if the author is not properly trained, then the questions may be subjective in a way. This is because this style of assessment does not usually require the tutor to interpret the given questions. The students are graded depending on the selections they make during the exam. This in turn lowers any likelihood of teacher bias when marking the papers. Also, any factors irrelevant to the material being assessed do not play any part in the assessment. So the candidates get to be graded on his or her knowledge in the topic being assessed.

Finally, good clarity of the answers in ensured by informing the examinees on how to select their choices. This may be through ticking the correct choice or circling it, thus achieving more clarity compared to other methods of assessment. In overall, multiple choice questions offer the strongest prediction of the overall performance of the students in comparison with other methods of assessment.

Disadvantages of multiple choice questions

The number one disadvantage of this method of assessment is the scanty types of knowledge that can be assessed using this method. The questions in this method are adapted best for testing lower order or well defined skills. Higher order reasoning or problem solving skills are assessed better by use of essay tests or short answer questions. Another disadvantage is a likelihood of ambiguity in the student’s interpretation of the questions asked. Therefore, care must be taken to ensure that the test maker gives relevant information that cannot lead to misinterpretation and confusion on the part of the examinee. In comparison to the other methods of assessment, multiple choice questions have been associated with the term "guessing."

This describes a situation whereby the examinees attempt to guess the correct choice instead of determining it. In the other methods, a free response usually allows the examinee to argue from his or her point of view and this may give him or her some credit for the argument. Also, in connection to the above point, this method of assessment does not offer the students a chance to give further information on the question even if they have sufficient knowledge regarding the same. Hence, no credit is given having correct and more information and if they happen to select a wrong answer, then an assumption is made that they do not have sufficient information on the question asked. In free response scenarios, the students get a chance to demonstrate their partial understanding of the unit under question and thus, they receive credit for this.

Another disadvantage of this method of assessment is that the examinee who does not have the capacity to answer a particular question can randomly select an answer from the choices given and still have an opportunity of getting that question right.

This has become a common practice for students who do not have any time left during the examination. This results in random guess work hoping that some of these random answers may be correct and therefore, earn some marks for the student. This also can be countered by coming up with rules like penalizing every wrong answer by subtracting some marks. This will discourage the students from guessing.

Also, any ambiguous framing of questions may result in confusion of the students taking the examination. Most, if not all multiple choice questions allow only one answer. That one answer may enclose other previous options. However, some examiners may be unaware of this and therefore expect the student to choose a number of answers without permission from the examiners. Hence, care should be taken to ensure validity of the test items.

Another disadvantage is that this method of assessment does work always for all types of questions. Thus, some questions are better answered using other methods of assessment.

Also, designing questions in this method of assessment can be tedious and time consuming.

On the other hand, multiple choice questions usually encourage students to memorize very discrete items of given information rather than clearly understanding the topic. Another disadvantage of this method of assessment is that it restricts creative students from expressing their ideas. This limits the student’s ability to articulate and furnish information, exhibit thought process, organize his or her personal thoughts, give original ideas, provide good examples, and perform a given task.

How to construct proper multiple choice questions

Poorly constructed multiple choice questions yield bad scores that are of less value therefore, inappropriate to use as a way of assessing the student’s achievement. Most poorly crafted multiple choice questions are characterized by one of the following frailties: they try to measure objectives for which they are not well suited for, some may have clues for the right answer while some are ambiguously worded. Thus, well crafted questions are those defined in adherence to set rules and guidelines to avoid such weaknesses as the ones mentioned above. Below are the guidelines for coming up with better multiple choice questions.

Ensure each item is meant to assess a single objective

Questions not written with specific objectives in mind end up evaluating trivial material or content, or sometimes unnecessary objectives that have little or no educational value to the student. Studies have shown that well crafted items based on specific objects make the questions more homogenous and easier to comprehend (Baker, 1971)

Make sure each item is based on a problem stated in the stem.

The stem is usually the main foundation of any item. Once the student has read the stem, then he or she is able to exactly know what the problem is and what is expected to be done in order to solve it. In case the examinee has to infer on what is expected to solve the problem, then that item will surely measure the ability of the student to make inferences from any vague descriptions rather than her or his achievement of the objectives of the course. Thus, the examiner should ensure that there are no incomplete sentences in the stem of the item.

Give as much information as you can on the item, but try as much as possible to avoid any irrelevant material.

Use clear and concise terms so as to avoid repetition of phrases or words especially in the alternatives. Such material should be put in the stem of the item so as to decrease the burden of reading, and this makes the problem clearer, hence easier to handle. Addition of irrelevant material to the stem of the item may also end up confusing the student more thus, this should be avoided. This may also decrease the reliability and validity of the resultant test scores therefore, undermining the credibility of the examiners.

The stem should be stated in a positive form

In negatively worded items, the examinee is asked or instructed to point out the exception among the options, or choose the incorrect option or the least correct option. These items are used frequently because it is easier to construct them. The examiner only needs to come up with a distracter instead of positively worded items. However, positive items prove more appropriate when measuring the attainment of educational objectives taught.

The alternatives should be worded concisely and clearly

Clear and proper wording reduces confusion when the student is doing the paper. On the other hand, concise wording decreases the burden associated with reading unnecessary and irrelevant materials. Thus, the student finds it easier to read and comprehend the item.

The alternatives should be kept mutually exclusive

Some alternatives may overlap thus creating undesirable situations. This should be avoided at all costs. On the other hand, some overlapping options may easily be identified as possible distracters; hence a lot of care should be taken. Also, if one of the overlapping alternatives includes the correct option, then it means there may be other alternatives which can be defended successfully as being among the correct options. Therefore, when coming up with the items, a lot of care should be taken so that only one option is correct.

The alternatives should be kept homogenous in terms of content.

If the alternatives comprise of a combination of a number of statements which are related to the stem, but not related to each other, then the task of the student becomes needlessly confusing. Therefore, the examiner should aim at coming up with alternatives which are parallel especially in content so as to aid the item in presenting the problem in a clear cut manner, that is also in line with the specific objectives taught by the teacher or lecturer.

The alternatives should be kept free from clues which may point out the correct response

Most often, poorly crafted items or questions contain clues which help the students who do not have an idea on which option is the correct one. They therefore start by eliminating the incorrect options, and by doing so, they increase their chances of nailing the correct answer. When left with few options, their chances of guessing wisely are highly increased. Also, these poorly crafted options sometimes contain clues which make it even easier to guess from the few remaining options. In some cases, this is done to measure the cleverness of the students in finding out clues within the set items. This specifically does not measure how well the students have understood the specific objectives taught by the teacher. To avoid giving clues in the stem, the following should be done: the grammar of each and every alternative should be kept in line with the stem. Studies done on this method of assessment have shown that grammatical clues often make it easier to come up with the correct answer from a given set of options. Another way of avoiding more clues is to try and keep all the alternatives in a parallel form. If the answer to a given question is worded in a given way, and the distracters are also worded differently, then the student may notice that notice that difference quickly and give the correct response. Thus, the alternatives should be kept parallel to the item to avoid making it easier. Another way of avoiding to give further clues is to make sure that all the alternatives are same in length. In most cases, students assume that alternatives which are shorter or longer are answers to the asked question, and this assumption is made without any valid reason. Therefore, to avoid this, all the alternatives should have roughly the same length. Studies have shown that questions are much easier when the correct option is longer than the existing distracters when all alternatives have similar length. Another way of avoiding clues is to avoid verbatim phrasing and textbooking. This is a situation whereby the answer to a given question has been copy pasted from certain pages of a given text book. This makes it easier for the students to identify the phrasing used rightly out of familiarity, choose the correct answer than reason it out of understanding. Another way of avoiding to give clues is by avoiding using specific determiners. When words such as always, never, and only are part of the distracters so as to make the alternatives false, then these act as clues therefore, alerting the students. Another way to avoid clues is to avoid including key words in the alternatives. When a phrase or a word is found in one of the options given, and also, the same phrase or word is found in the stem, then this may serve as a clue to the students, thus making it easier to identify the correct answer. Studies have shown that questions are easier when the key word in the stem is also found in one of the alternatives. This strongly indicates that this alternative is most likely the correct one. Another way to avoid clues is by using plausible distracters. Those students who did not understand the objectives off the topic well will find it hard to choose the correct answer if the distracters are made to look as if they are correct. Distracters that are unrealistic and humorous do not add any value and go a long way in increasing the chances of the student guessing the correct answer. Therefore, plausible distracters should be created in any item to make it more interesting. These distracters can be created in several ways. They can be created through use of common misconceptions. These are common among students and they usually perceive them to be correct but indeed, they are not. So, including these ones among the alternatives will go a long way in providing good alternatives that are not easy to determine the correct ones. The examiner can also create his or her own plausible distracters. These alternatives created should be plausible so as not to make it easier for the student who has not achieved the objectives of the subject. Care should be taken not to make them subtle for students who have achieved the intended objectives.

Avoiding alternatives stating "none of the above" or "all of the above"

The above two alternatives find a lot of use especially when the examiner has a problem of coming up with enough options or alternatives. Such examiners lay a lot of emphasis on the quantity of the distracters rather than the quality of the same. Therefore, continued use of the above phrases as alternatives tends to decrease the effectiveness of the set questions. Studies have shown that use of "none of the above" tends to decrease discrimination within the item itself while use of "all the above" has been found to be non conclusive. Thus, the examiners should try as much as possible to avoid using the above phrases.

Make sure you use many functional distracters that are feasible

Functional distracters are mainly chosen by students who have not clearly achieved the objectives of the topics or subjects taught. They are also ignored by the students who have achieved the required objectives. These distracters can be classified into the following classes: nonfunctional, functional, and dysfunctional. Whether a distracter is functional or not, it can be easily identified through proper analysis of the item. Generally, multiple choice questions usually contain 2 to 4 distracters. Many tutors assume that the greater the number of distracters, the lesser the chance of guessing the correct answer. This assumption made by many tutors proves to be true only when the existing distracters are functional and the existing multiple choice questions contain only one nonfunctional distracter. This distracters do not really work well and the entire question may fair on well even if those distracters were absent. It is usually advisable that the examiner designs good distracters even if they are few. It is not usually necessary to put few good distracters and then add poor ones. This is because these poor ones will undermine the credibility of these good ones and this in turn makes the question easier to answer even by means of guess work. Therefore, uniformity should be ensured and unnecessary distracters avoided. Studies have shown that there is usually little discrimination, difficulty or reliability of the test score in questions containing less distractors like two to four.

Ensure that only one answer is correct or the best in each question

When there are two or more alternatives that can be successfully defended to be the correct choice, then answering these questions becomes a hard task keeping in mind that only one answer is correct, and since you do not know what the lecturer had in mind, then narrowing down to the intended option becomes a hard nut to crack. Such uncertainty proves to be a problem especially in items which have a variety of best answers. In this case, more than one option is correct, but only one of all those alternatives should be the best. In such cases, if the relevant authorities do not agree on which alternative is the best, then that question should either be cancelled or revised and new alternatives given. Therefore, to ensure this does not come up, the examiner should carefully go through the whole paper and see whether there exists such a problem.

The answers should be placed in alternative positions with approximately equal chances in random order.

Many examiners have the habit of not placing the correct alternatives either in the last, or in the first position. They prefer putting them in the middle. However, some students do know this tendency and end up making right guesses because they have narrowed to a few options. Also, in some cases, the students are able to identify a given pattern probably on the positions of the correct choices. Once this pattern is noted, then guessing or choosing the next correct alternative becomes an easier task to perform. This usually occurs in cases of clever students who have not prepared themselves adequately for the paper. Thus, when they notice that pattern, their work becomes much easier and through this clever move, they have a higher chance of scoring good marks. The easier method of randomization of answer positions is to make sure that the alternatives are arranged in a certain logical order. The best order usually depends on the nature of the alternatives in a given item. Thus, that logical order can either be sequential, alphabetical or numerical. Studies have indicated that questions prove to be easier when a given outline or logical order is violated. Therefore, a lot of care should be taken when deciding on the positioning of the correct choices. The examiner should ensure that their positions do not at all give even a single clue on which one is the correct alternative.

The items should be laid out in a consistent and clear manner.

Test questions or items that are well formatted not only bring minimum or no confusion, but also enable students to take less time in answering the questions. This also makes grading of the easier especially when done by hand. The following should be done so as to the layout of the items is improved: the examiner should provide clear instructions and regulations to be followed or adhered to during the test, use the vertical format as it provides the best option for presentation of alternatives, and avoid page changing in middle of a given item. This makes it difficult to locate the correct options or alternatives. Hence, for a given item to have good standards, the alternatives should be arranged in a consistent and clear manner.

Ensure use of proper punctuation, spelling and grammar

While setting the questions, the examiner or any relevant body that undertakes the task should ensure that correct grammar, spelling and punctuation are used. This ensures that the guidelines and the questions are self evident. This also reduces any ambiguity within the whole item and by doing so; the student gets to take the test with a lot of seriousness.

Do not use any unnecessary or difficult grammar

Use of difficult and unnecessary grammar may hinder the reading ability of some students and hence bar them from achieving the objectives for which that item was meant for. As a result, students who are poor readers may fail to achieve the objectives or receive lower marks. Therefore, the persons or bodies tasked with the responsibility of setting the exam should ensure that there is no use of technical or difficult language. Simplifying the language used in the items makes the questions easier to comprehend.

The examiner should do analysis of the effectiveness of every item especially after every administration

Analysis of the items provides an excellent way of checking the effectiveness of the test given periodically. It usually identifies any items that do not function well hence, enabling revision to be done on these items. If revision cannot be done, then it is advisable that they are removed from the test, or the instructions of the paper revised.