|
A review of the substantial amount of information received from the
states regarding their various assessment policies and practices led
to the development of the conceptual framework to facilitate an analysis
for each state. The framework’s broadest function is to provide
a picture of the inputs, processes, outcomes, and impacts of each state
or system’s assessment policy. Within several of these overarching
categories, however, the framework also requires answers to several
specific questions.
This framework and its categories are presented here to offer one means
by which state policymakers could examine their own state assessment
policy, or that of others. The framework also permits a comparative
analysis of policies across states so commonalties, differences, trends,
and patterns can be discerned.
Original Initiative and Authority
For the purposes of this analysis,
any assessment activity initiated by a state legislature, state (higher)
education executive agency, or state college/university system governing
or coordinating board is considered public policy. No policy is, however,
in fact, a policy; it is a policy decision not to act.
We also noted how each state’s
assessment policy was initiated. Often, a state’s assessment policy
can have combinations of these means of initiation, or will have a history
in which these forms of authority/initiation are related.
Legislative Authority
Statutory action requiring compliance, typically the collection and
reporting of data
Intervention designed to place pressure on institutions, such as establishing
a Task Force or Commission to study higher education
SHEEO Authority
State agency adopts policy encompassing one or more of the six policy
types described below
Statewide planning and the type of state higher education agency
Gubernatorial Authority
Requiring State Higher Education Executive Agency to take action,
such as collect data, improve quality, study institutions, etc.
Placing public pressure on higher education to adopt reforms
State System Authority
Policy is self-initiated by state public higher education system
to respond to governmental pressure or to avoid further intervention.
Top
Policy Context
The policy context consists of four elements: historical,
political, economic, and social inputs.
Historical Inputs
The perceived need(s) for assessment, if any, and prior policies,
if any, which address that need
Political Inputs
The political circumstances in the state that led to the adoption
of the policy
The entities that influenced the policy content
Quality of relationships among prominent policy actors
Public opinion regarding higher education
Economic Inputs
Financial or other budgetary concerns that led to the development
of a policy
Economic needs/concerns of the state
Expenditures/Appropriations for higher education
Social Inputs
Demographic concerns, such as the need for college educated citizens,
or the shift or loss in population across the state
Top
Policy Type
Regulatory
The policy is designed to encourage/ensure compliance with regulations;
resources may be distributed, in part, based on successful compliance.
Reforming
The policy is designed to encourage/ensure reform of some type.
Quality Assurance
A policy focus that typically emphasizes assessment practices which
seek to improve quality or at least provide the public some assurance
that quality is a priority for state policymakers.
Accountability
Designed to make institutions accountable to some higher authority,
typically the governor and state legislature. each college and university
is held responsible by an external authority for reporting achievement
of particular standards of performance, such as the governor or the
legislature. Typically, the authority defines minimal levels of performance,
or the institution and the authority agree upon targets/goals. The
targets are typically defined at the level of the individual institution.
Institutional Improvement
Refers to improving educational programs, institutional management,
and teaching and learning. Assessment becomes a means of achieving
the goal of improvement.
Student Learning
In this policy schema colleges and universities are required to demonstrate
student-learning gains. The institution needs to have students exhibit
levels of performance on either measures of general skills and competencies
or on tests of specific knowledge related to general education and/or
major field curricula.
Top
Policy Stage
Our policy process model for higher education identifies
five stages in the development process for state assessment policies:
Problem Formation
The period when the need for a state-level assessment policy was
first recognized
Policy Formulation
Development of pertinent and acceptable proposed courses of action
for dealing with public problems
Policy Adoption
Development of support for a specific proposal such that the policy
is legitimized or authorized
Policy Implementation
Application of the policy to the problem
Policy Evaluation
Attempt by the state to determine whether the policy has been effective
Top
State Guidelines
What actions, assessment activities, reporting tasks,
etc. are required/mandated by a state entity?
Programs/Positions
What programs will be/have been created to carry out
the policy?
Who is responsible for data collection, analysis, and
reporting?
What entities continually monitor assessment activity?
What entities periodically review and evaluate the policy and its elements?
Top
Indicators and Outcomes
What common indicators for assessment and/or outcomes
are mandated across all institutions?
To what extent do indicators/outcomes vary across institutional types
and sectors?
How were these indicators established?
What criteria are used for evaluating achievement of
success?
Instruments/Methods
What common instruments to assess learning outcomes
are mandated across all institutions?
To what extent do instruments/assessment methods vary across institutional
types and sectors?
What instruments are nationally-normed and which are locally developed?
What instruments and assessment methods are currently in use?
Top
Teaching-Learning Elements
What specific elements of the assessment policy
are focused on faculty instruction and/or student learning?
Public Reporting
How are results/outcomes from assessment activity
shared amongst the prominent policy actors in the state?
What is the state’s pattern for sharing assessment information
with the public?
How effective has the policy been at enabling
policymakers to gather, organize, and analyze information from institutions?
Top
Database
What form of state-level computerized database exists?
1. Comprehensive statewide database at the SHEEO
level
2. Non-comprehensive statewide database at the SHEEO level
3. Multi-institutional database, not at SHEEO level
4. Limited multi-institutional database
5. No multi-institutional database
How is the database structured?
What elements are collected and used in the database?
Budget
What financial inducements are incorporated in
the policy design, or accompanied by budgetary provisions, to foster
institutional compliance with the policy?
How much, if any, state funds are provided to institutions
to offset the costs of adopting assessment practices?
Our research has revealed a trend toward four approaches
that states use to finance and otherwise provide incentives to colleges
and universities to implement assessment activities. They are:
1. Performance-based funding
2. Budget lines for assessment activities
3. Student fees for assessment purposes
4. Reimbursements to colleges and universities for assessment activities
Top
Regional Accreditation Association Relationship
What are the criteria and approaches set forth for
institutional accreditation regarding assessment and the requirements?
What are the methods and processes for assessment that the association
prescribes for institutions, including reporting, testing, and data
collection?
What types of institutional support are offered to campuses as they
implement these requirements, including training, resources, and services?
What is the association’s relationship with state agencies, including
the influence of the association on institutions, and the coordination,
cooperation, communication between the association and state higher
education agencies?
Has there been a formal evaluation of the association's criteria and
standards, including a review and reflection upon its effectiveness
in fostering assessment activity at the institutional level?
Top
Disciplinary Accreditation
To what extent is disciplinary accreditation used as
a means of assessment?
To what extent can the questions for the regional associations be used
to examine relationships at the disciplinary/program level?
Technology Focus
In what ways are institutions evaluated on the
extent to which they incorporate new methods for delivery of instruction
into their assessment programs?
Top
|
