Category Archives: curriculum evaluation

Evaluation Models Part III: Eisner’s Criticism Model

In the last few posts, we have looked at evaluation models that come from a scientific approach. In this post, we look at evaluation from a humanistic perspective.

Curriculum evaluation models that come from a humanistic approach see evaluation as more qualitative than scientific models. Humanistic evaluators believe that the process of assessing curriculum is too complex and messy to rely so heavily on quantitative methods. There should be fewer numbers and more written descriptions  The purpose is not to answer questions completely but to add to the conversation by exposing additional questions.

One example of an evaluation model that adheres to the principles of humanistic approaches is Eisner’s Criticism Model.

Eisner’s Criticism Model

Eisner’s Criticism Model is used for evaluating a new curriculum and not necessarily to examine a curriculum that is established. This model has four steps as follows

  1. Describe
  2. Interpret
  3. Evaluate
  4. Identify themes

The steps are mainly self-explanatory. First, evaluators describe the setting and curriculum of the study. Second,  the evaluators explain the reasons for the new curriculum by explaining the need. Third, The evaluators present their understanding of the value of the new curriculum. Fourth, various themes are identified within the curriculum that are meaningful.

This four-step process involves qualitative activities for data collection. Evaluators might participate in classes, observe classes or other activities, analyze student work,  use video, photos, interviews, of teacher and students in action. The goal is to notate what is happening but also what may not be happening in the data. This involves written narrative much more the numerical summaries.

In order to complete such an evaluation, an individual needs to have an expert knowledge in education. Eisner refers to such individuals as connoisseurs. These are people who know what to look for as well as how to value and appreciate what is happening.

Conclusion

Although not nearly as common as scientific models such as Stufflebeam or Stake humanistic models such as Eisner’s Criticism model has its place in education.  For smaller schools or smaller programs, this model is an excellent way to go beyond just numbers and to describe the effectiveness of a curriculum with words. However, for larger schools or programs it is challenging to use such an approach when there is so much happening that needs to be summarized succinctly.

Evaluation Models Part II: Stufflebeam’s Model

In the last post, we began a discussion on evaluation models of the scientific variety. In this post, we conclude are look at scientific evaluation models by examining Stufflebeam’s CIPP model. Daniel Stufflebeam is a famous educator and evaluator of curriculum.

Stufflebeam’s model is called the CIPP model which stands for

  • Context
  • Input
  • Process
  • Product

Context

Context is about studying the context in which the curriculum is used. The purpose is to assess what is happening and examine why needs of the stakeholders may not be met. Common activities at this stage include the following.

  • Physical environment
  • Philosophical foundations of the curriculum
  • Determine stakeholders
  • Background of the context

Of course, there are other activities that could take place but these just provide some examples

Input

Input is about determining whether there are adequate resources to conduct an evaluation. Evaluators look at the goals and strategies of the evaluation. If there are concerns, other approaches may be suggested in order to evaluate the curriculum.

In many ways, this is when the methodology of the evaluation is checked rigorously. Common questions assessed at this level include the following

  • Do goals and objectives align?
  • Are teaching strategies suitable?

Process

Whereas context and process take place before implementation,  process evaluation takes place during implementation. At this stage, evaluators examine if the plan is actually happening in the classroom. Evaluators look for defects in the implementation or use of the curriculum. Whatever problems are identified, strategies are developed to address them.

Normally, process happens at a pilot stage before a larger implementation. For example, the algebra teachers might experiment with a new teaching approach before using it in the entire math department at a school. In many ways, process evaluation is formative evaluation.

Product

The product stage involves collecting data to make a decision about the curriculum. Information is gathered to see how well the curriculum is meeting objectives. From this decision are made. The product stage is really a type of summative evaluation.

Conclusion

Stufflebeam’s model involves analyzing the context and the appropriateness of the curriculum to this context. From there, a formative and summative evaluation is used to assess the impact of the curriculum among the stakeholders. This practical model is useful for educators who are seeking ways to examine the various programs under their care.

Evaluation Models Part I: Stake’s Congruence-Contingency Model

Evaluation models are used in curriculum as a process for assessing the appropriateness of a curriculum for a context. As with approaches to curriculum evaluation, evaluation models can be divided into scientific and humanistic models. For the next few post, we will look at scientific models of curriculum evaluation. Our first example is Robert Stake’s Congruence-Contingency Model.

Congruence-Contingency Model

Stake’s model of curriculum evaluation is more than just an evaluation process. Stake’s model also looks at the development of the curriculum. When using this model, it is necessary to compare the developed curriculum with what actually happens in the classroom.

There are six key terms, broken down into two groups of three, that we need to know in order to understand Stake’s model and they are as follows.

Development Stage

  • Potential prerequisites
  • Potential Curriculum
  • Potential results

Evaluation Stage

  • Prerequisites applied in context
  • Evaluation of operational curriculum
  • Actual results

Prerequisites

The prerequisites is another way of saying “before” or the state of the context before the intervention of teaching. This includes student’s attitude, motivation, prior academic performance, teacher characteristics, and more. In the development stage, the teachers need to identify what are some potential prerequisites that may impact learning. In the evaluation stage, the evaluators determine what prerequisites actually impact the curriculum. In other words, there is a comparison of what was anticipated and what actually was the case in terms of the prerequisites.

Potential & Operational Curriculum

Potential curriculum is the “dream” curriculum that is developed. It includes everything that the teachers want to do. The Operational curriculum is what was actually used. There is normally a discrepancy between the two as it is difficult to cover all of the material and use all of the activities. The evaluation will examine the difference between these two aspects of curriculum as another criterion for assessing the quality of the curriculum.

Potential vs. Actual Results

Potential results are what the teachers hope to see as a result of the use of the curriculum. Actual results are the real performance of the students. The difference between the potential or desired results and actual results is another indicator of the quality of the curriculum in Stake’s model.

Conclusion

Stake’s Model provides evaluators with an opportunity to compare the desired outcome with the actual outcome. The benefit of this is that it is the curriculum developers that set the criteria of evaluation. All the evaluators do is determine if the curriculum performed in a manner that is consistent with the ideas of the developers.

Approaches to Curriculum Evaluation Part II: Intrinsic vs Payoff Approach

In the last post, we looked at scientific vs humanistic approach to curriculum evaluation. In this post, we continue the discussion by examining the intrinsic vs payoff approach to curriculum evaluation

Intrinsic Approach

The intrinsic approach is used to assess the overall quality of a curriculum. This involves looking at the various components of curriculum design such as the scope, sequence, articulation, balance and other aspects. Aspects of curriculum development are also assessed which means examining the teaching methods, content, and learning experiences. However, initially, at least, the criteria of evaluation is not determined but emerges after the process begins.

The intrinsic approach not only examines the value of the curriculum but also how well a given curriculum reaches its goals and objectives. This involves collecting some form of data whether quantitative or qualitative. As such, most evaluators normally approach evaluation with some of the characteristics of the intrinsic approach.

Payoff Approach

In the payoff approach, clear evaluation criteria are set from the outset. Normally, evaluators look at the impact of the curriculum on its stakeholders, which often includes, students, teachers, parents, and administrators. This approach to evaluation is among the most popular in education because of the clear criteria which makes data collection smooth and efficient.

Which Approach to Use?

The intrinsic approach may be most useful when it is unclear exactly what the stakeholders want to know. In other words, it is useful for exploratory purposes. Nobody is sure where they are going and the intrinsic approach helps to setup a map of strengths and weaknesses within a program. From there, other approaches can be used to refine the evaluation if necessary.

The payoff approach is best when the evaluation team knows exactly what it wants to know. Clear evaluation questions/criteria has been set and it is only a matter of answering the questions or assessing the level at which the curriculum meets the criteria by collecting data.

As such, the purpose is not to declare one approach superior to the other but to keep in mind the context when deciding which tool to use.

Approaches to Curriculum Evaluation Part I: Scientific vs Humanistic Approach

Curriculum evaluation is the process of collecting data in order to make decisions about the curriculum in question. Curriculum can mean a host of things. It could refer to a particular subject such as 7th grade reading, it could refer to a particular grade such as 8th grade in general, it can also refer to an entire school such as elementary or secondary school. As such, one aspect of curriculum evaluation to consider is the scope or what is being evaluated.

There are meaning different approaches or ways of seeing curriculum evaluation. For whatever reason, approaches to curriculum evaluation are always explained in extremes. In this post we will look at the following approach to curriculum.

  • Scientific vs Humanistic Approach to Curriculum Evaluation

Scientific Approach

The scientific approach is probably the oldest approach to curriculum evaluation as it dates from modernism and the emphasis on the scientific method of the 19th to 20th century. This approach to curriculum evaluation focuses on using quantitative data generate by the learners. This allows for statistical analysis. Furthermore, the results are compared in order to determine the level of success. This comparison is at the heart of decision-making when this approach is employed.

There are natural issues with such a heavy emphasis on numerical data. For one, the students narrative is missing. A lickert scale analysis is not as rich in content as an interview. Another issue is the assumption of similar circumstances. The diversity in student ability and even in teaching ability makes it difficult to assume that students are facing similar challenges and circumstances.

Humanistic Approach

In a more post-modern worldview the Humanistic approach looks at the individual rather than the numbers. Data is much more qualitative in nature. The rationale behind this is that life has multiple perspectives to it and quantitative data only provides one perspective.

Humanistic evaluators want to understand the complexities of the environment they are assessing. This involves capturing narratives through interviews and focus groups. Observation is used not to count frequencies but to take notes of what is happening in the classroom.

The major issues with this approach is the smaller sample size that is required. It is not feasible to interview 400 students but perhaps 20 is doable. In contrast, conducting a survey with 400 students should not be a challenge for a scientific evaluator. Furthermore, there are questions as to the objectivity of the results.

Since qualitative data is processed by the researcher their own perspective can filter what they report when they share the perspective of the respondents, In contrast, scientific approaches are more objective in that computer processes and reports the results.

Conclusion

Instead of having a bias towards scientific or humanistic approaches to curriculum evaluation. It is better to look at the context of what needs to be evaluated and determine the most appropriate approach. It should be the context and not the preference of the evaluators that should decide which direction to take. In many situations, a mixture of both approaches may be appropriate but this involves much more work and complexity.

The Purpose of Curriculum Evaluation

Curriculum evaluation is a critical aspect of the educational process. Most schools have to evaluate their curriculum at one point or another. This post will share insights into defining evaluation as well as the distinction between measurement and evaluation.

Defining Evaluation

Evaluation of a curriculum happens in order to decide whether to accept, change, or eliminate various aspects of a curriculum. The overall goal is to understand if the curriculum is producing the desired results. This implies that the evaluators know what to expect prior to the evaluation and are looking for these predetermined results.

Evaluation is about gathering data. This data can be collected in many different ways. The various data collection approaches are the same as any used in research. They include observation, interviews, surveys, and more. The data is often aggregated and used to determine if the goals of a program are being met.

Measurement and Evaluation

Evaluation is not only about measuring a phenomenon. Instead, evaluation assigns value and meaning to the results of a study. It assesses quality through quantitative or qualitative means. This is in contrast to measurement which describes a phenomenon but does not interpret the quality of it.

It is common for there to be confusion over measurement and evaluation. To remember the difference, measurement describes something numerical. Evaluation, on the other hand, judges something qualitatively.

Conclusion 

It is important to remember that evaluation can happen at many different levels. The teacher can assess their unit plans. A department can assess their reading program. A school can assist its entire curriculum. Regardless of the level. Curriculum evaluation is often focused on determining how the curriculum is doing in terms of achieving the goals set for it.