SCALE

What?

Transforming learning by transforming assessment.

Who?

Raymond Pecheone, Executive Director, Stanford Center for Assessment, Learning and Equity (SCALE).

Where?

U.S.A.

The Stanford Center for Assessment, Learning and Equity (SCALE), launched in 2009, focuses on the development of:

Pre-service and teacher evaluation performance assessments for teachers and administrators at the school, district and state levels and a performance-based system for student assessment to support the development of the next generation of formative and summative assessments at the district, state and federal level.

Performance assessments are tests in which the test taker does real-world tasks to that require them to demonstrate the knowledge and skills the test is intended to measure, rather than by answering questions asking how to do them.

“All of our work is laser-focused on building high quality performance assessments that are educative – meaning students or teachers are taking the assessment, they’re learning from that process, and it provides actionable information to supervisors and coaches about how to improve their practice to support continuous learning.”

The big vision for SCALE is: to improve instruction and learning through the design and development of innovative, educative, state-of-the-art performance assessments.

They achieve this by building the capacity of schools to use these assessments in thoughtful ways, to promote student, teacher, and organisational learning.

Linda Darling-Hammond – a Stanford Scholar – had been publishing about the importance of a fair and equitable assessment of teacher quality.

Her work converged with public policy in a number of states, where governments were looking to ensure anyone going into the profession, regardless of the university they graduated from or the region in which they lived, would meet a standard for teaching.

Starting in California, Linda and Raymond began work with a consortium of 18 California Universities to develop a reliable and valid measure of teacher quality through a performance-based assessment system. Later, Linda, Raymond and the now established Stanford Center for Assessment, Learning and Equity (SCALE) designed and developed a performance assessment for students, embedded in curriculum, in response to the growing commitment to Deeper Learning across the country.

1: The start point

A proof of concept – reliable and valid

Having established a successful proof point in California, and then in Washington State, Raymond and Linda received a grant from the Flora Family Foundation and others, to bring their performance assessment to scale, nationally. They designed and developed the EdTPA – a support and assessment programme for teacher candidates, which is being used for different purposes in different states.

“We were first to the market with a reliable and valid performance assessment of teaching, and the states were looking for establishing standards for teaching in order to license new teachers. So it was a confluence between the emergence of state policy amid the wider tone of the accountability system.”

Inoculate yourself for innovation

Raymond and colleagues were able to create space to do innovative work and create proof points without being constrained by the pressures that come with working at scale and worrying about sustainability.

“The luxury that we have at Stanford is that we care about the quality. We care about making it efficient, but we’re not going to strip it down to it’s bare essentials simply for the purposes of greater efficiency and profit, while losing educability of the work.”

2: Taking off

Don’t compromise, be bold!

Raymond and colleagues at SCALE were clear on the purpose of their work and were steadfast in their resolve to stick with it, no matter what challenges they faced.

“If you go in looking at all of the possible constraints to innovation and change and start to chip away at your work or fundamental ideas because you think they’re ‘too advanced’, you’re already marginalising and trivialising your work.”

Resisting a compliance approach

States who more successfully adopted SCALE’s teacher performance assessment were more careful in building buy-in from stakeholders, rather than relying on compliance.

“We found the best way to get people on board – on both teacher and student assessment – is for them to actually score teacher work, or give them a framework for assessment to look at student work – if it’s for students. Teachers find that tremendously empowering.”

Multiple iterations, revisions and refinements

Even though the demand for their teacher performance assessments spanned 40 states, Raymond and colleagues never rested on their laurels. They tinkered and tweaked their approaches to respond to local contexts and need. Sticking to your vision and model matters, but so does differentiation because context matters – it’s not one-size-fits-all. The challenge is being smart about how you implement and how fast or slow you go forward in a particular context, while not compromising on the goal or the vision.

“You don’t have to be perfect, but you have to learn as you’re going. In doing multiple iterations over time – any time we thought we could make an improvement in the assessment we did. We didn’t wait years to change it.”

3: Keeping going

Making the case to people

Raymond recognises the need to balance boldness with clarity around the rationale, and what evidence and proof points exist. Anchoring this in the reality of people’s lived experiences is critical.

“You owe people an explanation of what you’re doing and why; you’ve got to be sensitive of local context; and you’ve got to communicate in the terms that they understand.”

Investment in implementation support

SCALE have invested heavily in providing the resources and support for the successful implementation of their performance-based assessment systems. They provide direct Professional Development with universities and networks of universities around implementation, as well as issues of equity. The emphasis is on teaching the core constructs that drive the assessment design, rather than purely focusing on the practicalities.

“I think with 20-20 hindsight, we might have done more around support – building awareness and understanding, building the collective wisdom and support for the work – we could have always done more, particularly at the front end in preparing people for the assessment.”

Collaboration at the core of your theory of action

In developing both their student and teacher performance assessments, SCALE recognise the collaborative contributions of a broad range of stakeholders and beneficiaries of education. They tend to convene Design Teams of administrators, captains of business and industry, as well as faculty. Raymond and colleagues have the expertise, but they emphasise a collaborative process of developing the assessments aligned to the local context.

“No matter how smart you might be in building an assessment, if we didn’t have a collaborative element to it, it would not work. For example, we’re working with the state of Virginia, Colorado, around building performance assessments that are aligned to their state standards and their policy, but also deliberately and purposefully bringing in experts in service-learning based on interactions with business and industry.”

There was a pre-existing appetite across the U.S.A for trying these new assessment systems out.

There was a coalition of the willing in both cases – state legislatures who were eager to implement the teacher performance assessment, and forward-looking administrators or school networks interested in deeper learning and therefore portfolio-based student assessment. SCALE found that the task was to moderate and support each partner, based on the local context.

“There are some schools really ready to do this and they can dive right in – those are usually the pioneers. They do all of the original work. Then there’s other schools who want to dip their toes.”

System impact

SCALE’s EdTPA is currently being used in 40 states in the U.S.A, either for licensure or for programme accreditation, with well over 1000 Higher Education institutions, as well as a number of alternate route programmes in the country adopting it in some form.

Learner impact

• Longitudinal data suggests there is some predictive validity to performance assessments that can predict both student learning in the teacher case, and college success and graduation in the student case.
• Longitudinal data has shown that Charter schools which SCALE has worked with, which serve low income students who would be the first in their family to attend college courses if they got accepted, were achieving a higher acceptance rate. For one school in particular, 95% of their students got into college. Of that 95%, 95% actually finish a four year degree.