Program evaluation is a necessity among educators. Without evaluation, it is unclear whether teaching and learning objectives are being met. Nursing education programs are mandated to evaluate by using objectives, but each program is free to devise its own method of evaluation. At our university, the College of Nursing Program Evaluation Committee (PEC) devised a systematic plan for evaluation that has been in place for 12 years.
This systematic plan has undergone revisions as the College of Nursing (CON) has changed. Recently, the university began using Blackboard as a platform for Web-based learning and online program evaluation. Online program evaluation is a reality-based method that is particularly helpful when the faculty engaged in teaching perform the evaluation, rather than when an individual evaluator is assigned to the task.
This article begins by reviewing the major elements of the CON program evaluation plan. We’ll describe how the program evaluation plan is actualized using the e-Enterprise platform, Blackboard. We’ll offer a step-by-step description of creating a site, folders, and sub-folders, and end by presenting the benefits of the site and online surveys.
The Evaluation Plan elements
In their discussion of nursing education, Carolyn Waltz, Susan Chambers, and Nan Hechenberger (see References at end of article) define evaluation as the systematic, continuous process of collecting and analyzing data about program elements in order to make decisions that will improve the quality and effectiveness of the program. The evaluation process includes identifying, collecting, analyzing and providing information to be used in decision making. The Program Evaluation Committee oversees the implementation of the evaluation plan and reviews the plan for effectiveness. The program evaluation model is dynamic and cyclic. It is a continual process with systematic feedback among the program elements of context, input, process, and outcome, as Daniel Stufflebeam observed (see References). The elements are defined as follows:
- The context determines whether the environment of the program supports the program’s mission, purposes, and objectives and is conducive to its success. The context in which the nursing program takes place includes clinical practice sites, conditions that influence the organization (such as the economy or the profession’s standards and direction), and societal needs and trends. The context of the program is examined for unmet needs as well as available opportunities. The focus is on intended outcomes.
- The input delineates resources available to the program and assesses policies, procedures, strategies, and plans for resource utilization to reach intended goals and objectives. This element considers students, faculty, curriculum plans, organizational structure and governance plans, policies and procedures, physical resources, facilities and services, and financial resources. The feasibility of achieving goals and objectives given available resources is evaluated. The focus is on intended means of the program.
- The process monitors the implementation of the plans and procedures of the program. The elements determine if the program is implemented as planned. Barriers and program flaws are identified. A record of actual events is a part of process evaluation.
- The outcome or product measures outcomes by assessing the product of the program including graduates, community services, and scholarly activities. This element is concerned with measuring the extent to which program purposes and objectives have been met on a regular basis. The focus is on both immediate program objectives and long-term program impact. Both intentional outcomes and side effects are included in this element of evaluation.
Thus our evaluation plan incorporates the four elements of context, input, process and outcome. Once the plan is devised, the instruments for gathering data are developed. The ongoing nature of data collection within the evaluation process is challenging, yet the use of e-Enterprise course sites make it easier.
The description following is our process. It will facilitate collection of evaluation data for any organization that uses Blackboard.
Creating Web sites for evaluation
A Webmaster creates the course site on the e-Enterprise platform, such as Blackboard. The site can be created as a course and named “
FIGURE 1 The initial Blackboard page lists the site administrators.
FIGURE 2 Announcements advise faculty about course features.
On the left of the screen, under the university’s abbreviation TWU, the Course Documents tab is the fourth one down (tabs appear as 3-D buttons in this area). The Documents site is used to create committee folders. Changing the tab name from “Course Documents” to “Committee Items” would make it clearer. However, tab changes are not currently an available feature on our university’s version of Blackboard.
Creating folders for evaluation
To create a committee folder, an individual must obtain administrator status from the Web manager to access the control panel. To access the control panel, scroll down to the bottom of the left column of icons and click on the control panel button. Figure 3 shows the control panel button circled. Clicking this button will produce a screen showing all of the course management options, arranged on control bars as seen in Figure 4. To create the committee file within Course Documents, begin by clicking on the “Course Documents” link on the left of the screen. In the next screen, shown in Figure 5, simply click on the “Add Folder” icon in the upper left of the Web page. Figure 6 shows the space used for typing the committee name and the box just under it contains an area for typing information about the folder contents. We titled this folder “Program Evaluation Committee” and described the contents as “This folder has sub-folders containing all surveys used by the CON and all PEC meeting minutes.” After entering the folder content description, scroll down on the same page and click “Submit.” When the newly named folder appears, click on the committee folder name to open the folder.
FIGURE 3 The control panel icon in the navigation area provides access to the Control Bars, allowing the user to create a committee folder.
FIGURE 4 Course administration options are shown as links on the various Control Bars.
FIGURE 5 Adding a folder is a one-click operation within the Course Documents area.
FIGURE 6 Naming and describing the new folder is done on the “Add Folder” screen.
Creating subfolders for evaluation
Figure 7 shows that the Program Evaluation Committee folder can have sub-folders for committee minutes, reports, survey forms, and the overall program evaluation plan. To create these subfolders, simply click on “add folder,” name it, and repeat the activity explained in Figures 5 and 6 until all desired folders are created.
FIGURE 7 Folder structure and contents appear on the “Course Documents” screen.
The subfolders within the main program evaluation folder organize information and provide all faculty with access to committee items. Additional folders were created for the overall evaluation plan, policies and procedures, surveys, minutes, and reports. Figure 8 shows the initial items and Figure 9 shows the reports available as faculty scroll down that page. Faculty view these items using the original tabs shown on the left of Figure 2 by clicking on the course documents icon and then clicking on the Program Evaluation folder. Thus they do not need access to the control panel which is a feature important only in setting up the folders. Their actual view is shown in Figure 10 with the tabs visible on the left side.
FIGURE 8 Initial items available to faculty.
FIGURE 9 Reports available to faculty.
FIGURE 10 The faculty’s view of folders and other items.
