Introduction

Institutional effectiveness (IE) at Jacksonville College involves the systematic review of all areas of campus activity in light of the mission of the College. The mission as published in the Jacksonville College Course Catalog is as follows:

Jacksonville College, a Christian, independent, co-educational, two-year liberal arts college exists to provide Christ-centered teaching and training that prepares students to lead meaningful lives that transform the world.

The Board of Trustees, administration, faculty, and staff are devoted to ongoing assessment with the intent to continuously improve the educational services rendered to the students of Jacksonville College. The assessment process revolves around the following questions:

1. Is the College operating within the scope of its mission?
2. Is the College accomplishing what it intended to accomplish?
3. What can the College do to improve the quality of education received by our students?
4. Have efforts to improve brought about successful changes within the institution?

Institutional Effectiveness and Strategic Planning

A distinction must be drawn between institutional effectiveness and strategic planning.

Strategic planning involves the direction of budgeted funds to accomplish a set of goals.

While institutional effectiveness may call for the expenditure of funds in order to

accomplish the quality improvements desired, the process of determining the effectiveness of educational programs is not a resource-based program. Institutional effectiveness is a means by which Jacksonville College demonstrates that its mission is not just an empty promise. It requires that administration, faculty, staff, and trustees show verifiable, measurable proof that the programs of the College are indeed fulfilling the mission of the College and are in a constant quest for quality.

The Establishment of an Institutional Effectiveness Committee

Because institutional effectiveness is a campus-wide effort, each facet of campus life is represented on the Institutional Effectiveness Committee. This committee is appointed by the President’s Cabinet and reports to the Senior Director of Institutional Research who serves as chair of the committee. Members of the committee include the Administrative Vice President, Registrar, Director of Systems and Communication, one SGA representative, and others as needed. The committee is charged with the task of overseeing the ongoing assessment process. It is the responsibility of this committee to review the goals that support the mission of the College, evaluate the assessment documentation, and utilize the results for improvement. It is also the responsibility of this committee to train College personnel in institutional effectiveness.

The Five-Column Model

The model used by the Institutional Effectiveness Committee to document effectiveness is the “five-column model” developed by Dr. Jim Nichols, a leading authority on effectiveness and assessment in higher education. The model divides the process into five sections, each of which is represented by one column of the model. The columns provide a structure for documenting the effectiveness process at specified times of the academic year.

Each college unit to be assessed in the institutional effectiveness process (hereafter referred to as an IE unit), whether instructional or non-instructional, completes the appropriate five-column form. Template of this form may be found in the Appendix to this document, identified as item A-1. An explanation of each of the four columns found on the form follows.

Column One: Expanded Statement of Institutional Purpose

Each IE unit must relate to the mission of the College and to at least one of the Institutional Goals identified by the Board of Trustees as goals to specifically detail the mission of the College. Each IE unit should be able to identify at least one goal statement that relates the unit to the mission of the College. These goals are published in the Jacksonville College Course Catalog  in Section II, entitled “Organization of the College.”  In column one of the model, each non-instructional unit must complete an expanded statement of purpose that relates the unit to the mission and to one of the goals. Section 7.1 of the 2017 Principles of Accreditation states that  “[t]he institution engages in ongoing, comprehensive, and integrated research-based planning and evaluation processes that (a) focus on institutional quality and effectiveness and (b) incorporate a systematic review of institutional goals and outcomes consistent with its mission” in prescribed areas. Accordingly, each IE unit must show through the expanded statement of purpose that it is guided by the mission statement. To summarize, column one requires the IE unit to complete three subdivisions: (1) the mission, (2) the goal statement to which the IE unit relates, and (3) an expanded statement of purpose. A statement of purpose is not necessary for instructional units since instruction is obviously linked to the mission of an educational institution.

Column Two:  Intended Educational Outcomes

Each IE unit utilizes column two to identify outcome statements that directly relate to at least one of the goal statements. Typically these outcomes focus on areas where improvement is needed. These statements must be clear and measurable. Dr. Nichols warns against writing process-oriented statements rather than results-oriented statements. Process-oriented statements focus on the “how-to” of institutional operations while results-oriented statements focus on what the institution’s constituents will know, think, or do. Consider the following outcome statements:

                (1) Student Services will hire another support staff person.

                (2) Student Services will enhance the retention of the resident student population.

The first statement is a process-oriented statement. It does not speak directly to the quality of service provided by Student Services. Staff additions do not always enhance the performance of a unit. Perhaps the statement could be altered to read that “Student Services will provide prompt response to student inquiries.”  Response times can be measured. Benchmarks may be set. The assessment of this outcome might indicate that an additional staff member is necessary to enhance the performance of Student Services. Statement 2 is a results-oriented statement already. The statement implies an action on the part of the students affected by Student Services, namely that they are retained at the institution. Retention rates are measurable.

While there is no single correct way to write a results-oriented outcome statement, it is of utmost importance that the statement be clear and easily understood. Compound statements should be avoided as a general rule. A compound statement such as, “Student Services will provide accurate and current information regarding any program in a friendly, prompt, and professional process” can seriously complicate the assessment process. It is best to break such statements into more manageable single statements that are far more easily assessed. Also, outcome statements must be measurable so that results can be documented, verified, and used as a basis for improvement. When writing an outcome statement, an IE unit should consider the following questions:

(1)   Does the outcome support one of the goals of the College?

(2)   Does the outcome address what the unit’s constituents will know, think, or do?

(3)   Is the outcome statement concise?

(4)   Is the outcome statement measurable?

An IE unit should restrict the number of outcome statements to three to five. It is more productive to address a few issues well rather than to generate mountains of meaningless, incoherent data. An IE unit might construct a long list of potential outcome statements. This list may include a variety of activities. From this long list, the unit develops a shorter, more manageable list of outcomes. To develop the short list, the unit should focus on three to five areas. The final outcome statements should be clear, concise, results-oriented statements that are easily assessed and clearly tied to the institutional mission.

Column Three: Means of Assessment and Criteria for Success

In column three, each IE unit will establish the anticipated criteria for success and benchmark performance standards for achieving the outcomes identified in column two. Assessment is not an end in itself but a means by which the IE unit strives for continuous improvement. It is the process of improving that is important. An effective assessment process utilizes multiple methods of assessment in an effort to provide an unbiased appraisal of the unit’s performance. Assessment tools should not rely heavily on the opinions of those employed within the IE unit. The assessment process should focus on those affected by the unit. For instance, the assessment statement, “Students will rate the quality of information provided by Student Services at 4.0 or higher on a 5.0 point Likert scale as indicated on question 17 of the Student Opinion Survey” is much preferred to the statement, “Staff members in Student Services will rate the quality of information provided by Student Services at 4.0 or higher on a 5.0 point Likert scale on the Departmental Performance Survey.”  Assessment statements should identify the method for assessment and the expected result. The declaration of criteria for success requires careful attention. Criteria set too low will be too easily attained and will not encourage the performance enhancement sought by the institutional effectiveness process. Criteria should be high enough to motivate improvement but should be ultimately attainable.

Assessment is often the most demanding part of the effectiveness process. A complicated method of assessment is not necessary. In fact, IE units should make every effort to take advantage of the assessments already available through the Office of Institutional Research.

Column Four: Assessment Results

At the end of the cycle, the IE unit’s performance in meeting the established criteria in column three is documented in column four. Assessment results should be stated clearly and concisely, without defensiveness or rationalization. This column requires only the statement of facts. Either the IE unit met the indicated benchmarks or it did not. Superior assessment is found in the establishment of meaningful and motivating benchmarks. A benchmark that is reached consistently is probably too low. Failure to meet an established benchmark is acceptable if the results of the assessment lead to the enhancement of the quality of service provided by the IE unit. At this point the IE unit should consider possible reasons for the less-than-expected result. The prevailing question should be,”What can Jacksonville College do to improve its services?”

Column Five: Use of Results

In column five, each IE unit explains how the assessment results are used to improve performance. The goal of assessment is the improvement of instructional and support programs within the institution. There are several concepts to consider when writing up the results. First, the IE unit must listen to the results. The assessment results often provide enlightening information as to how the unit can improve its services. Then the unit must focus on meaningful improvement. The purpose of the institutional effectiveness process is improvement of services, so the unit should take advantage of the opportunity to focus in on those changes that will truly enhance performance. Once again, the narrative within column five should provide facts.  The unit should avoid exaggerations of its efforts to improve performance based on the results of assessment. Improvements in column five should be genuine and verifiable with evidence that may be documented. If an IE unit chooses to declare an intended improvement, its efforts must be tracked the following year to demonstrate its implementation.

Closing the Loop

The completion of column five ends the current institutional effectiveness cycle. A new cycle begins immediately. Frequently, at least some of the outcome statements will remain the same from one cycle to the next until the desired criteria for success are realized consistently. New outcome statements can now be added as the criteria from prior year statements are successfully achieved. In any case, the process is ongoing from one year to the next.

A generic example of the five-column model is presented on page 9 of this handbook. In the interest of brevity, the example considers only one educational outcome for this IE unit. The reader should note that the outcome statement is results-oriented and easily measured. Assessment involves the results of two surveys. Column four simply states the results of the assessment tools with no apology or clarification of any kind. Column five delineates a plan for meaningful improvement in the services provided by this generic Student Services unit. 

Example of Five-Column Model

                 1                                       2                                3                                  4                                5

The Institutional Effectiveness Cycle: Use of Results

The Institutional Effectiveness Cycle

The institutional effectiveness process is cyclic in nature. The preceding diagram illustrates the complete cycle. At Jacksonville College, each assessment cycle is staggered over a twelve-month period of time as follows:

August - September

All units, instructional and non-instructional, begin the IE cycle.

Column 5 “Use of Results” from each unit’s previous year report will guide the goals/objectives for the new IE cycle.

Columns 1-3 of the IE unit report must be submitted for review by September 15.

December - January

Analysis of fall data for Column 4 begins and must be completed by January 15.

June - August 

Analysis of both fall and spring data will provide information for Column 5 of IE report, thereby closing the loop. The completed report must be ready for submission by June 15. (Admissions Department: Submit report from previous year by September 30.)

Appendix

Institutional Effectiveness - A-1

Institutional Effectiveness Report from xxxx for (year)

College Mission Statement: Jacksonville College exists to provide a quality education from a biblical worldview that challenges minds, transforms lives, and equips students for servant leadership and lifelong learning.

Strategic Plan Goals:

Strategic Plan Objectives:

The xxxx will support the College’s Mission and Strategic Plan by:

                      1                                         2                                              3                                           4