Tag Archives: testing

Direct and Indirect Test Items VIDEO

Developing direct and indirect test items for language assessments.

Advertisements

Tips for Developing Tests

Assessment is a critical component of education. One form of assessment  that is commonly used is testing. In this post, we will look at several practical tips for developing tests.

Consider the Practicality

When developing a test, it is important to consider the time constraints, as well as the time it will take to mark the test. For example, essays are great form of assessment that really encourage critical thinking. However, if the class has 50 students the practicality of essays test quickly disappears.

The point is that the context of teaching moves what is considered practical. What is practical can change from year to year while adjusting to new students.

Think about the Reliability

Relibility is the consistency of the score that the student earns. THis can be affected by the setting of the test as well as the person who marks the test. It is difficult to maintain consistency when marking subject answers such as short and answer and or essay. However, it is important that this is still done.

Consider Validity

Validity in this context has to do with whether the test covers objects that were addressed  in the actual teaching. Assessing this is subject but needs to be considered. What is taught is what should be on the test. This is easier said than done as poor planning can lead to severally poor testing.

The students also need to be somewhat convince that the testing is appropriate. If not it can lead to problems and complaints. Furthermore, an invalid test from the students perspective can lead to cheating as the students will cheat in order to survive.

Make it Aunthentic 

Tests, if possible, should mimic real-world behaviors whenever possible. This enhances relevance and validity for students. One of the main problems with authentic assessment is what to do when it is time to mark them. The real-world behaviors cannot always be reduced to a single letter grade. This concern is closely relates to practicality.

Washback

Washback is the experience of learning from an assessment. This normally entails some sort of feedback that the teacher provides the student. the feedbag they give. This personal attention encourages reflection which aides in comprehension. Often, it will happen after the testing as the answers are reviewed.

Conclusion

Tests can be improved by keeping in mind the concepts addressed in this post. Teachers and students can have better experiences with testing by maintaining practical assessments that are valid, provide authentic experiences as well insights into how to improve.

Understanding Testing

Testing is standard practice in most educational context. A teacher needs a way to determine what level of knowledge the students currently have or have gained through the learning experience. However, identifying what testing is and is not has not always been clear.

In this post, we will look at exactly what testing as. In general, testing is a way of measuring a person’s ability and or knowledge in a given are of study. Specifically, there are five key characteristics of a test, and they are…

  • Systematic
  • Quantifiable
  • Individualistic
  • Competence
  • Domain specific

Systematic

A test must be well organized and structured. For example, the multiple choice are in one section while the short answers are in a different section. If an essay is required there is a rubric for grading. Directions for all sections are in the test to explain the expectations to the students.

This is not as easy or as obvious as some may believe. Developing a test takes a great deal of planning for the actual creation of the test.

Quantifiable

Test are intended to measure something. A test can measure general knowledge such as proficiency test of English or a test can be specific such as a test that only looks at vocabulary memorization. Either way, it is important for both the student and teacher to know what is being measured.

Another obvious but sometimes mistake by test makers is the reporting of results. How many points each section and even each question is important for students to know when taking a test. This information is also critical for the person who is responsible for grading the tests.

Individualistic 

Test are primarily designed to assess a student’s individual knowledge/performance. This is a Western concept of the responsibility of a person to have an individual expertise in a field of knowledge.

There are examples of groups working together on tests. However, group work is normally left to projects and not formal modes of assessment such as testing.

Competence

As has already been alluded too, tests assess competence either through the knowledge a person has about a subject or their performance doing something. For example, a vocabulary test assesses knowledge of words while a speaking test would assess a person ability to use words or their performance.

Generally, a test is either knowledge or performance based.  it is possible to blend the two, however, mixing styles raises the complexity not only for the student but also for the person who s responsible for marking the results.

Domain Specific

A test needs to be focused on a specific area of knowledge. A language test is specific to language as an example. A teacher needs to know in what specific area they are trying to assess students knowledge/performance. This not always easy to define as not only are there domains but sub-domains and many other ways to divide up the information in a given course.

Therefore, a teacher needs to identify what students need to know as well as what they should know and assess this information when developing a test. This helps to focus the test on relevant content for the students.

Conclusion

There is art and science to testing. There is no simple solution to how to setup tests to help students. However, the five concepts here provides a framework that can help a teacher to get started in developing tests.

Test Blueprint

Developing assessments is often difficult. Teachers wonder if they have covered all the material, they have to think about how to assess the students, and they need to consider reaching them at different levels of thinking. This is not in anyway easy for most teachers.

One way to deal with these problems is through the use of what is called a test blueprint. A test blueprint is a map of the objectives that are assessed on the test as well as a map of the different levels of learning that each question addresses. Below is an example. You can click on the image to make it bigger.

Objectives-page0001 (1)

The example above is a test on curriculum development. There are four objectives that are assessed on this test and they are.

  1. Develop aims from the needs assessment.
  2. Develop goals
  3. Develop standards
  4. Develop objectives

These four objectives are assessed at one of three levels of Bloom’s taxonomy. These levels are

  1. Knowledge
  2. Comprehension
  3. Application

For these test, I am looking at how my students perform these four objectives at the knowledge, comprehension, and application levels of Bloom’s Taxonomy.

For each level of Bloom’s Taxonomy, I asked 1 or 2 questions related to my objectives. This number is then totaled at the far right. For example, for objective one, developing aims from needs assessment, I have 1 true and false question at the knowledge level, 1 multiple choice question at the comprehension level, and one long essay question at the application level. This gives me a total of 3 questions that focus for the “developing aims from the needs assessment objective.”

These three questions for objective 1 represent 15% of the total questions on the test. In other words, not only do I know how many questions I asked in each level for this objective, I also know how much of the total test is represented in my first objective. This helps in maintaining a balanced test as sometimes teachers give too much weight to one specific piece of information.

Not only can you know how many questions you ask about each objective, you can also know how many questions you have for each level of Bloom’s taxonomy. For example, looking at the chart, you can see that there are 6 knowledge questions which represent 30% of the total questions on this exam. Again, such information helps in maintaining a balanced exam.

At the very bottom of the chart, you can also find out how many of each type of question you asked on the exam. For example, there are 6 True and false questions and 8 multiple choice questions for a percentage of 30% and 40% of the total questions. This also helps with balance. You can make sure that different forms of questions are an appropriate percentage of the total exam.

The test blueprint helps teachers to develop balance exams. Objectives, question type, and the level of the questions can all be taken into account when developing an assessment.