Real-time collaboration and continuous action
create ideal conditions for future success.
boilerplate image

Criteria for High-Quality Assessment Systems

Webinar:


Panelists
Linda Darling-HammondEdD, Charles E. Ducommun Professor of Education,
Stanford University; Director, Stanford Center for Opportunity Policy in Education
James W. PellegrinoPhD, Co-Director, Learning Sciences Research Institute,
University of Illinois at Chicago
Bob Wise, President, Alliance for Excellent Education

What does a high-quality assessment system look like? States will have to make important decisions about assessments in the coming years, as the products developed by the two state consortia come on line. To help guide those decisions, nearly two dozen of the nation’s leading assessment experts developed a set of criteria. These criteria address what the assessments should measure, how they should measure these abilities, and how the assessments should be used.

In this webinar, panelists explored the statement of criteria and its implications for states. Linda Darling-Hammondand James Pellegrino described the criteria and what they mean, and Bob Wise moderated the discussion. Panelists also addressed questions submitted by webinar viewers from across the country.

Additional Resources:

Linda Darling-Hammond’s PowerPoint presentation.

Support for this webinar is provided in part by
the William and Flora Hewlett Foundation.

The Alliance for Excellent Education is a Washington, DC–based national policy and advocacy organization that works to improve national and federal policy so that all students can achieve at high academic levels and graduate from high school ready for success in college, work, and citizenship in the twenty-first century. www.all4ed.com
Follow the Alliance on Twitter at www.twitter.com/all4ed; Facebook at www.facebook.com/all4ed; and the Alliance’s “High School Soup” blog at www.all4ed.com/high-school-soup.

                                                                     

>>> GOOD AFTERNOON, MY NAME IS 
BOB WISE AND I'M PRESIDENT FOR 
THE ALLIANCE OF EXCELLENT 
EDUCATION A NONPROFIT POLICY 
ORGANIZATION IN WASHINGTON, D.C.
I'LL BE YOUR HOST TODAY.
WE'RE VERY GLAD THAT YOU HAVE 
JOINED US FOR THIS NEXT HOUR AS 
WE EXAMINE IMPORTANT NEW 
RESEARCH ON ASSESSMENTS WITH 
IMPLICATIONS FOR ALL STATES.
WHILE WE HAVE THIS CONVERSATION, 
PLEASE JOIN IN ON TWITTER USING 
THE HASH TALK AT ALL FOR ED.
AS YOU KNOW, ADOPTED BY 46 
STATES AND THE DISTRICT OF 
COLUMBIA CALL FOR HIGHER 
EXPECTATIONS FOR STUDENT 
ACHIEVEMENTS THAN MOST PREVIOUS 
STATE STANDARDS.
SPECIFICALLY, THESE COMMON 
COURSE STANDARDS ASK STUDENTS 
NOT ONLY TO DEMONSTRATE THAT 
THEY UNDERSTAND CORE CONTENT BUT 
TO ALSO BE ABLE TO USE THEIR 
KNOWLEDGE TO THINK CRITICALLY 
AND CREATIVELY.
TO SOLVE COMPLEX PROBLEMS AND 
COMMUNICATE EFFECTIVELY.
THESE PEOPLE LEARNING COMPETENCY 
ARE VITAL TO ENSURE THAT 
STUDENTS CAN SUCCEED IN A GLOBAL 
ECONOMY AND BECOME ACTIVE 
CITIZENS IN A COMPLEX WORLD.
THESE STANDARDS ARE WORTHY 
ASPIRATIONS.
WE'RE MOVING RAPIDLY FROM 
ASPIRATION TO IMPLEMENTATION.
THE KEY PART OF THE 
IMPLEMENTATION PROCESS IS THE 
DEVELOPMENT OF NEW ASSESSMENT 
THAT TRULY MEASURE THESE DEEPER 
LEARNING COMPETENCY.
AS WE KNOW, TEACHERS AND 
STUDENTS PAY A LOT OF ATTENTION 
TO WHAT'S TESTED AND IF THE 
TESTS DON'T MEASURE WHAT THE 
STANDARDS EXPECT THEN THE 
ASPIRATIONS STANDARD WILL NOT 
FULLY BE REALIZED.
WHAT DO HIGH QUALITY ASSESSMENTS 
LOOK LIKE?
A GROUP AS A NATION LEADING 
ASSESSMENT EXPERT HAVE NOW COME 
UP WITH A CRITERIA.
STATES CAN USE THIS CRITERIA AS 
THEY IMPLEMENT NEW ASSESSMENTS, 
ALIGNED FOR COLLEGE AND CAREER 
READINESS.
WE'RE FORTUNATE TODAY TO HAVE 
WITH US TWO OF THE AUTHORS OF 
STATEMENT OF CRITERIA WHO CAN 
EXPLAIN WHAT THEY MEAN AND WHY 
THEY'RE IMPORTANT.
APPEARING BY TELEPHONE IS DR. 
LINDA DARLING-HAMMOND, CHARLES 
E. DUCOMMUN PROFESSOR OF 
EDUCATION STANFORD UNIVERSITY 
AND DIRECTOR AT THE STANFORD 
CENTER ON OPPORTUNITY FOR POLICY 
EDUCATION.
OR S.C.O.P.E.
LINDA HAS STUDIED AROUND THE 
WORLD AND IS LEAD AUTHOR.
ALSO JOINING US BY TELEPHONE 
HAVE ANOTHER AUTHOR OF THE 
STATEMENT AND EMINENT EXPERT ON 
ASSESSMENT DR. JAMES PELLEGRINO.
JIM IS THE CO-DIRECTOR OF 
LEARNING SCIENCES RESEARCH 
INSTITUTE AT THE UNIVERSITY OF 
ILLINOIS, CHICAGO, AND HE WAS 
THE CHAIR OF THE NATIONAL 
RESEARCH DOWN, COMMITTEE ON 
LEARNING AND 21st CENTURY 
SKILLS.
WE ARE ALSO VERY FORTUNATE TO 
HAVE WITH US HERE IN THE STUDIO 
KRISTIN AMUNDSEN, THE NEW 
EXECUTIVE DIRECTOR OF THE 
NATIONAL ASSOCIATION OF STATE 
BOARDS OF EDUCATION.
KRIS WAS MOST RECENTLY THE VICE 
PRESIDENT OF EDUCATION SECTOR 
AND HAS BEEN A SCHOOL BOARD 
MENTOR AND STATE LEGISLATOR IN 
VIRGINIA.
WE ARE THRILLED SHE'S AGREED TO 
PARTICIPATE IN WHAT I BELIEVE IS 
HER FIRST WEEK ON THE JOB.
WE LOOK FORWARD TO YOUR 
PARTICIPATION ON THIS WEBCAST 
ESPECIALLY ON THE VIDEO WHERE 
YOU CAN ASK QUESTION.
WE'LL GO TO THE WINDOW ROOM TIME 
TO TIME TO SEE IF WE CAN RESPOND 
TO IT.
PLEASE KNOW WE CAN'T GET TO 
EVERY QUESTION BUT WE'LL TRY.
WITH THESE THINGS OUT OF THE 
WAY.
LET'S TURN TO LINDA 
DARLING-HAMMOND WHO HAS A 
PRESENTATION. 
>> THANKS VERY MUCH, BOB.
THE REASON FOR THIS PUBLICATION, 
CRITERIA FOR HIGH QUALITY 
ASSESSMENT IS THAT WE'RE AT THIS 
CRITICALLY IMPORTANT MOMENT IN 
REALLY OUR NATION'S HISTORY AND 
CERTAINLY IN THE TRAJECTORY OF 
EDUCATION REFORM IN STATES.
WHICH IS THAT AS THE COMMON 
COURTS, SOME STATES HAVE 
DEVELOPED THEIR OWN STANDARDS.
THERE'S THIS STRONG PRESS TO 
MOVE THE SYSTEM OF EDUCATION TO 
BE MORE COGNIZANT OF THE HIGHER 
ORDER THINKING AND PERFORMANCE 
SKILLS NEEDED IN THE 21st 
CREPTRY BUT WE HAVE AN 
ASSESSMENT SYSTEM CURRENTLY THAT 
FOR THE MOST PART DOESN'T 
MEASURE THESE SKILLS.
SO THE QUESTION ON THE TABLE FOR 
STATES, FOR THE FEDERAL 
GOVERNMENT, OR LOCAL DISTRICTS 
AND OTHERS ABOUT WHAT SHOULD WE 
BE STRIVING FOR AS WE DESIGN, 
APPROVE, AUTHORIZE AND FUND THE 
NEW ASSESSMENT SYSTEMS.
ON THE NEXT SLIDE, WHY SEE THE 
LIST OF EDUCATION AND ASSESSMENT 
EXPERTS WHO JOINED TOGETHER TO 
CREATE THIS STATEMENT FROM ALL 
ACROSS THE COUNTRY.
PEOPLE WHO HAVE BEEN INVOLVED 
FOR MANY YEARS IN DEVELOPING THE 
LEADING EDGE, THINKING IN THE 
FIELD.
THE DOCUMENT WAS PUBLISHED BY 
THREE CENTER, STANFORD CENTER 
FOR OPPORTUNITY POLICY AND 
EDUCATION.
UCLA'S CREST CENTER AND 
UNIVERSITY OF ILLINOIS AND 
CHICAGO CIRCLED LEARNING 
SCIENCES RESEARCH INSTITUTE.
SO IT REPRESENTS A VERY BROAD 
SET OF PERSPECTIVES ABOUT WHAT'S 
NECESSARY TO HAVE ASSESSMENTS 
THAT REALLY DO JUSTICE TO THE 
STANDARD.
AS PRESIDENT BARACK OBAMA SAID 
IN MARCH 2009, WHEN THE 
ASSESSMENT CONSORTIA WERE ABOUT 
TO BE AUTHORIZED.
I'M ALLING ON DEVELOPING 
ASSESSMENTS THAT DON'T MEASURE 
STUDENTS CAN FILL IN A LOT OF 
TESTS BUT WHETHER THEY POSSESS 
21st CENTURY SKILLS LIKE PROBLEM 
SOFTING, ENTREPRENEURSHIP AND 
CREATIVITY.
THAT'S A TALL ORDER FOR 
ASSESSMENTS.
IT'S ONE THAT ACROSS THE WORLD 
NATIONS HAVE BEEN REDESIGNING 
THEIR ASSESSMENT SYSTEMS TO TRY 
TO ACCOMPLISH.
AT THE NEXT SLIDE WE JUST REMIND 
OURSELVES THAT OF COURSE THE 
GOAL OF THE COMMON CORE STATE 
STANDARDS IS THAT THEY BE FEWER, 
HIGHER AND DEEPER, THAT THEY BE 
MORE FOCUSED, THAT THEY REACHED 
TO THE HIGHER ORDER OF THINKING 
SKILLS AND PROMOTE THIS KIND OF 
DEEPER LEARNING THAT ALLOWS 
STUDENTS TO TRANSFER WHAT THEY 
THOUGH TO NEW PROBLEMS AND 
SITUATIONS.
THE GORDON COMMISSION WAS 
ANOTHER RECENT LARGE-SCALE 
EFFORT TO IDEA WAYS OF MOVING 
OUR ASSESSMENT SYSTEMS FORWARD 
TO MEET THE NEEDS OF STUDENTS 
FOR FEEDBACK ON THEIR LEARNING, 
TEACHERS AND PARENTS FOR DEEP 
UNDERSTANDING OF WHAT HAS BEEN 
LEARNED OR HOW PEOPLE LEARNED 
AND SYSTEMS TO SEE HOW WE CAN 
TRACK THE LEARNING PROGRESS.
THE GORDON COMMISSION PUT THIS T 
THIS WAY.
TO BE HELPFUL IN ACHIEVING THE 
LEARNING GOALS.
THE ASSESSMENT MUST REFLECT THE 
COMPETENCIES THAT THE CHANGING 
WORLD DEMANDS.
TO DO SO THE TASKS AND 
ACTIVITIES MUST BE MODELED.
WORTHY OF ATTENTION AND ENERGY 
OF TEACHERS OR STUDENTS.
TO PUT IN T IN ANOTHER WAY.
THEY MUST BE TESTS WORTH TEACHES 
TO.
THAT DON'T TRUNCATE THE 
LETTERING PROCESS.
BUT IN FACT EXPAND IT. 
>> I MENTIONED OTHER NATIONS ARE 
TRANSFORMING ASSESSMENTS.
WHEN YOU LOOK AT PLACES LIKE 
SINGAPORE LIKE THE UK.
LIKE AUSTRALIA.
PROVINCES IN CANADA AND SO ON WE 
FIND MOST COUNTRIES USE 
OPEN-ENDED ESSAYS AND PROBLEMS 
THAT STUDENTS NEED TO SOLVE AND 
EXPLAIN THEIR ANSWERS TO AND 
ALSO INCREASINGLY PERFORMANCE 
TASKS THAT ARE SOMETIMES CALLED 
CLASSROOM-BASED ASSESSMENTS THAT 
ACTUALLY REQUIRES STUDENTS TO 
DESIGN AND CONDUCT 
INVESTIGATIONS AND WRITE THOSE 
UP IN THE WRITTEN PRODUCTS OF 
THEIR INVESTIGATIONS ARE SCORED 
IN THE SYSTEM.
TO COLLECT DATA TO ANALYZE THEIR 
FINDINGS OFTEN EARLY AS WELL AS 
IN WRITING SO WE ARE LATE COMERS 
IN THE PROCESS, WE HAVE A LOT 
THAT WE CAN LEARN FROM AND MODEL 
ON THE WORK THAT SOME OTHER 
COUNTRIES HAVE DONE.
WE SEE THE MULTI STATE, 
PARTNERSHIP FOR ASSESSING 
READINESS THEY'RE DEVELOPING 
PERFORMANCE TASKS AS PART OF 
TESTS THAT WILL CHALLENGE 
STUDENTS IN WAYS THAT WILL BE 
NEW IN STUDENTS FOR MANY STATES.
FOR EXAMPLE, THIS TEST FROM THE 
SMARTER BALANCE ASSESSMENT 
CONSORTIUM ASKS STUDENTS TO 
FIGURE OUT WEATHER NUCLEAR POWER 
WILL BE AN ASSET OR PROBLEM IN 
THEIR STATES.
THEY HAVE TO GO ONLINE AND DO 
RESEARCH ABOUT THIS.
THE STUDENTS ARE PUT IN THE 
POSITION OF BEING A CHIEF OF 
STAFF TO A CONGRESS PERSON.
THEY HAVE TO THEN ILLUSTRATE 
FROM THE SOURCES WHAT THE PROS 
AND CONS ARE.
THEY HAVE TO EVALUATE THE 
CREDIBILITY OF DIFFERENT SOURCES 
AND THEN THEY HAVE TO ACTUALLY 
COMPOSE A FULL LENGTH REPORT FOR 
THE CONGRESS PERSON.
IN WHICH THEY USE THAT EVIDENCE 
TO JUSTIFY A POSITION.
PRO OR CON OF WHETHER THE POWER 
PLAN SHOULD BE BUILT IN THEIR 
STATE.
OBVIOUSLY THIS REACHES FOR THE 
HIGHER ORDER THINKING SKILLS 
KIDS HAVE TO USE IN CAREERS AND 
HIGHER EDUCATION.
SIMILARLY, THE PARK ASSESSMENT 
TEST ASKS STUDENTS TO WEIGH AND 
BALANCE CLAIMS IN THIS CASE.
THEY HAVE TO EVALUATE WHICH OF 
SEVERAL CLAIMS ABOUT AMELIA 
EARHART AND HER NAVIGATOR ARE 
ACTUALLY MOST BELIEVABLE AND 
CREDIBLE AND WELL DEFENDED AND 
THEN TO WRITE AN ESSAY, 
ANALYZING THE STRENGTH OF THE 
ARGUMENTS IN MULTIPLE TEXTS.
THERE ARE FIVE CRITERIA THAT THE 
ASSESSMENT EXPERTS HAVE 
IDENTIFIED AS CHARACTERIZING 
HIGH QUALITY SYSTEMS OF 
ASSESSMENT.
WE USE THE TERM SYSTEMS, BECAUSE 
NO SINGLE TEST OR ASSESSMENT CAN 
MEET ALL OF THE PURPOSES AND 
STATES ARE GOING TO HAVE TO 
THINK ABOUT WHAT SET OF 
ASSESSMENTS, SOME OF THEM 
PERHAPS ADMINISTERED OR DESIGNED 
AT THE STATE LEVEL.
SOME ADMINISTERED AND DESIGNED 
AT THE LOCAL LEVEL BUT ALL OF 
THEM TOGETHER PROVIDE AN 
OPPORTUNITY TO INFORM THE 
TEACHING OF THESE HIGHER ORDER 
SKILLS AND EVALUATE THEIR 
LEARNING.
FOR INDIVIDUAL STUDENTS.
THE FIRST OF THE CRITERIA IS 
THAT THERE OUGHT TO BE A 
DEMONSTRATION THAT THE SYSTEM 
ASSESSES HIGHER ORDER SKILLS.
ONE WAY TO THINK ABOUT THOSE 
SKILLS IS THIS DEPTH OF 
KNOWLEDGE TAXONOMY REPRESENTED 
BY A WHEEL.
IF YOU LOOK ON THE LEFT SIDE 
WHERE THE TWO ARROWS ARE, YOU'LL 
SEE THAT LEVELS THREE AND FOUR 
IN THIS TAXONOMY REPRESENT WHAT 
WE CALL HIGHER ORDER THINKING 
SKILLS.
THE ABILITY TO CRITIQUE AND 
FORMULIZE, ASSESS, COMPARE, 
DESIGN, AND SYNTHESIZE 
INFORMATION, TO ANALYZE AND SO 
ON.
LEVELS ONE AND TWO, TO SORT OF 
MEMORIZE, IDEA, RECOGNIZE, TELL, 
ARE IN FACT NOW THE VAST 
MAJORITY OF THE ITEMS ON CURRENT 
TESTS.
IN FACT, A RECENT REPORT FOUND 
THAT FEWER THAN 2% OF MATH ITEMS 
ON STATE TESTS AND 20% OF TESTS 
CURRENTLY MEASURE THE HIGHER 
ORDER FOR STRATEGIC THINKING 
SKILLS.
THE CRITERIA SUGGESTS THAT MOST 
ITEMS SHOULD EVALUATE SKILLS 
THAT ALLOW STUDENTS TO TRANSFER 
THEIR LEARNING TO NEW SITUATIONS 
AND THAT AT LEAST A THIRD OF THE 
ITEMS IN THE NEW MATHEMATICS 
ASSESSMENTS AND HALF IN THE 
ENGLISH ARTS ASSESSMENTS, SHOULD 
MEASURE THOSE HIGHEST LEVELS IN 
THREE AND FOUR IN THIS TAXOTOMY 
OF THINKING SKILLS.
IN FACT THE CURRENT MULTISTATE 
CONSORTIUM EFFORTS LOOK LIKE 
THEY SHALL MEET THE STANDARDS 
FOR THEIR TEST.
THE SECOND CRITERIA THESE 
CRITICAL ABILITIES OUGHT TO BE 
ASSESSED WITH HIGH FIDELITY.
THAT IS THEY OUGHT TO BE 
ASSESSED IN THE WAY THAT 
STUDENTS WILL HAVE TO USE THEM 
AND THOSE ABILITIES WHICH ARE IN 
THE COMMON CORE STATE STANDARDS 
IN THE NEXT GENERATION SCIENCE 
STANDARDS AND OTHER STATE 
STANDARDS INCLUDE THE ABILITY TO 
RESEARCH, TO EXPERIMENT AND 
EVALUATE THE RESULTS OF SUCH 
INVESTIGATIONS, TO COMMUNICATE 
ORALLY AS WELL AS IN WRITING, TO 
USE TECHNOLOGY, COLLABORATE, TO 
ENGAGE IN MODELING AND DESIGN.
THESE SHOULD REPRESENT THE KINDS 
OF TASKS THAT ARE USED.
OR SINCE THEY'RE ACTUALLY ASKED 
TO ENGAGE IN RESEARCH TO 
DEMONSTRATE THEIR COMMUNICATION 
SKILLS, RATHER THAN THROUGH VERY 
REMOTE PROXIES.
THE THIRD CRITERION IS THE 
ASSESSMENTS BE BENCHMARKED THAT 
IS THAT THE RIGGER OF THEIR 
CONTENT BE COMPARABLE TO THOSE 
IN -- OF THE ASSESSMENTS IN HIGH 
ACHIEVING NATIONS THAT THEY 
INCLUDE PRODUCTIVE TASKS 
ASSESSING 21st CENTURY SKILLS 
LIKE THOSE IN SUCH JURISDICTIONS 
AND THAT THEIR EXPECTATION OF 
PERFORMANCE, LEVEL OF 
PERFORMANCE STANDARD ALSO BE 
COMPARABLE AND THAT WILL DEPEND 
ON STUD CHASE COMPARE THE 
ASSESSMENTS ACROSS THESE 
JURISDICTIONS.
FOURTH THE CRITERIA SUGGESTS THE 
ASSESSMENT BE STRUCTURALLY 
SENSITIVE AS WELL AS 
EDUCATIONALLY VALUABLE.
SOME TESTS IT TURNS OUT ARE NOT 
VERY INSTRUCTIONALLY SENSITIVE.
THEY DON'T ACTUALLY MEASURE 
SKILLS IN A WAY THAT IS 
REPRESENTED IN THE CURRICULUM 
TEACHABLE AND LEARNABLE.
THEY MAY LEE RELIE MORE ON 
TEST-TAKING SKILLS.
THEY MAY RELY ON STUDENTS 
OUTSIDE OF SCHOOL, CULTURAL 
CAPITAL.
THERE ARE WAYS TO DESIGN TESTS 
THAT THEY ARE MORE STRUCTURALLY 
SENSITIVE.
WE THINK THIS IS VERY IMPORTANT 
AND WE WANT TO BE SURE THE 
ASSESSMENTS ENGAGE STUDENTS IN 
STRUCTURALLY VALUABLE ACTIVITIES 
THAT PROVIDE USEFUL INFORMATION 
ABOUT THINGS STUDENTS USE AND 
BEYOND.
FINALLY WE WANT STUDENTS TO BE 
VALID.
THEY SHOULD MEASURE WHAT THEY 
SAY THEY'RE MEASURING.
THEY SHOULD BE ACCURATE AND 
RELIABLE THEY NEED TO BE 
DESIGNED TO BE ACCESSIBLE AND 
UNBIASED AND KEY FEATURE OF 
VALIDITY.
THEY SHALL BE USED APPROPRIATELY 
FOR THE PURPOSES WHICH THEY'RE 
DESIGNED AND FOR WHICH THEY ARE 
VALIDATED.
THAT TYPICALLY NEEDS THEY NEED 
TO BE IN THE CONTEXT OF MULTIPLE 
MEASURES AS EVIDENCE ABOUT 
STUDENT LEARNING, THE NEW 
SYSTEMS OF ASSESSMENT SHOULD 
ULTIMATELY INCLUDE PERFORMANCE 
TESTS THAT ARE IMPLEMENTED IN 
CLASSROOMS AS WELL AS MORE 
SOPHISTICATED ON DEMAND TESTS IN 
ORDER TO MEET THIS CRITERIA, 
THEY SHOULD INVOLVE TEACHERS.
THE ASSESSMENT SO THEY'RE IN 
FACT STRUCTURALLY USEFUL AND 
VALUABLE.
THEY SHOULD PROVIDE INFORMATION 
THAT CAN HELP US DEVELOP 
STRONGER CURRICULUM AND TEACHING 
AND WE THINK THEY SHOULD BE USED 
FOR IMPROVEMENT.
AND CONTINUAL INFORMATION RATHER 
THAN FOR PUNISHMENT.
I'LL STOP THERE AND LET OTHERS 
COMMENT. 
>> THAT'S VERY COMPLETE.
JIM, LET ME TURN TO YOU AND I'LL 
TURN TO CHRIS FOR REACTION OF 
LINDA'S PRESENTATION AND OPEN IT 
UP FOR QUESTION.
JIM, OF COURSE, YOU WERE THE 
CO-AUTHOR OF THIS DOCUMENT. 
>> THANK YOU, GOVERNOR WISE.
I THINK LINDA DID AN EXCELLENT 
JOB GIVING THE SENSE OF FIVE 
CRITERIA AND WHAT'S ASSOCIATED 
WITH EACH OF THEM AND THE 
IMPORTANCE OF EACH.
I SIMPLY SORT OF WANT TO 
EMPHASIZE I GUESS TWO THINGS.
ONE IS SOMETHING THAT LINDA 
MENTIONED.
AND IT'S THAT WE NEED TO THINK 
IN TERMS OF SYSTEMS OF 
ASSESSMENTS SINCE THERE'S NO 
SINGLE ASSESSMENT THAT WE'RE 
GOING CONSTRUCT THAT IS GOING TO 
MEET THE NEEDS WE HAVE IN THE 
EDUCATIONAL SYSTEM FOR FOR 
SUPPORTING TEACHERS IN THE 
CLASSROOM.
AT THE SAME TIME WE'RE TRYING TO 
DO SYSTEM MONITORING AND COLLECT 
RESULTS FOR PURPOSES OF 
ACCOUNTABILITY.
WE NEED TO THINK IN TERMS OF 
SYSTEM OF ASSESSMENTS THAT WILL 
INCLUDE LARGE-SCALE ASSESSMENTS 
OF THE TYPE THAT ARE BEING 
DEVELOPED BY THE TWO CONSORTIA 
AND OTHER GROUPS BUT ALSO 
INCLUDES ASSESSMENTS INTENDED TO 
FUNCTION MUCH CLOSER INSTRUCTION 
THAT MIGHT BE USED ON A PERIODIC 
BASIS BY TEACHERS TO INFORM 
THEIR PRACTICE AND GUIDE 
STUDENTS LEARNING.
I THINK THE IMPORTANT THING IS 
WHETHER WE'RE TALKING ABOUT 
ASSESSMENTS THAT ARE DEVELOPED 
BY THE CONSORTIA PART OF LARGE 
SCALE STATE TESTING FOR 
ACCOUNTABILITY AND OTHER 
PURPOSES OR WHETHER WE'RE 
TALKING ABOUT ASSESSMENTS THAT 
WE DEVELOP TO SUPPORT TEACHING 
AND LEARNING IN LINE WITH THE 
COMMON CORE THESE CRITERIA NEED 
TO BE THOUGHT ABOUT AS 
APPLICABLE TO ALL OF THE KINDS 
OF ASSESSMENTS THAT WE DESIGN 
AND IMPLEMENT AND USE FOR 
VARIOUS INTERPERTIVE PURPOSES.
WE CAN'T SORT OF SAY WHAT WE USE 
AT THE CLASSROOM LEVEL DOESN'T 
HAVE TO MEET MANY OF THESE SAME 
CRITERIA, BECAUSE IT'S MORE 
INFORMAL. 
>> I THINK THINKING IN TERMS OF 
FIVE CRITERIA AND ALL FIVE FOR 
ANY ASSESSMENTS THAT WE'RE 
DEVELOPING OR THAT WE PLAN TO 
USE TO SUPPORT THE PROCESS OF 
IMPLEMENTING THE COMMON CORE 
STANDARDS AND HELPING TO MAKE 
PROGRESS TOWARDS ACHIEVING THE 
GOALS OF THOSE STANDARDS IN THE 
NEXT GENERATION SCIENCE 
STANDARDS WILL REQUIRE HIGH 
QUALITY ASSESSMENTS THAT MEET 
THESE FIVE CRITERIA AND DO THAT 
IN A FAIRLY RIGOROUS WAY.
WHICH MEANS WE HAVE A LOT OF 
WORK AHEAD OF US IN TERMS OF 
DEVELOPING AND VALIDATING. 
>> INDIVIDUAL COMPONENTS OF THIS 
SYSTEM AS WELL AS THIS SYSTEM AS 
A HOLE.
I'LL JUST QUIT THERE FOR A 
MOMENT AND COME BACK WHEN WE 
HAVE SOME QUESTION. 
>> CRIS, YOU'RE NOT THE DIRECTOR 
OF NATIONAL BOARDS OF EDUCATION.
ISN'T THIS WHAT IS FACING 
OFFICIALS WHAT KIND OF 
ASSESSMENTS TO ADOPT AND WHAT 
IMPORTANCE DO YOU ADOPT TO?
>> IT'S CLEAR, THE HEAVY LIFTING 
WILL COME OVER THE NEXT 18 
MONTHS TO YEARS AS THESE 
ASSESSMENTS GET ROLLED OUT INTO 
INDIVIDUAL STATES. 
>> BECAUSE THE THINGS THAT WORK 
IN ONE STATE MAY ONLY PERSONALLY 
WORK IN ANOTHER.
THEY WILL HAVE TO BE INDIVIDUAL 
DECISIONS BUT AS I WAS LISTENING 
TO THE EXCELLENT PRESENTATION 
THAT LINDA MADE.
THE FIVE CRITERIA, I THOUGHT OF 
FOUR KEY QUESTIONS THAT I 
THOUGHT A STATE POLICY MAKER 
NEEDS TO ASK THEMSELVES IN LIGHT 
OF THOSE CRITERIA.
FIRST, I THINK STATE 
POLICYMAKERS NEED TO ASK 
REALISTICALLY WHAT ARE THE 
SKILLS AND KNOWLEDGE THAT WE 
NEED TO ASSESS ON THE STATE 
LEVEL.
WHAT ARE THE THINGS THAT ARE KEY 
TO US.
SECOND, WHAT IS OUR STATE'S 
CAPACITY TO DEVELOP THOSE 
ASSESSMENTS OR EVEN TO 
ADMINISTER THE TEST IN SOME 
CASES.
AND I WOULD POINT OUT THAT 
ISSUES LIKE BAND WIDTH MAY 
BECOME A REAL CONCERN FROM SOME 
PLACES LICKICALLY IN SOME SMALL 
RURAL AREAS. 
>> CONNECTIVITY AND ABILITY TO 
HAVE TOOLS TO BE ONLINE AND TAKE 
THE TEST. 
>> THIRD I THINK WE NEED TO 
THINK CLEARLY ABOUT GOING INTO 
ASSESSMENTS LIKE THE ONES WE ARE 
TALKING ABOUT.
WHAT RESOURCES ARE WE GOING TO 
NEED?
I WILL STAY THAT I AM CERTAINLY 
GUILTY THAT DURING THE LAST 
ROUND OF EDUCATIONAL CHANGE. 
>> WE WORKED VERY HARD ON THE 
STANDARDS AND DID A LOT OF -- I 
THINK SOLID WORK. 
>> AND THEN WE AWARDED THE 
ASSESSMENTS TO THE LOWEST 
BIDDER.
IT'S QUITE CLEAR IF WE TALK 
ABOUT THE KINDS OF TESTS THAT 
ARE BEING ENVISIONED IN THE 
SCOPE -- AND BY BOTH CONSORTIA.
THOSE WILL BE MUCH MORE 
EXCESSIVE.
BUT I THINK WE'LL HAVE TO HAVE 
STATES THINK ABOUT IT AS NOT 
ONLY INVESTMENT AND ASSESSMENT 
BUT FRANKLY TEACHER ASSESSMENT 
AND DEVELOPMENT, IF THEY ARE 
ALIGNED IN THE WAY THEY ARE 
TALKING ABOUT, THEY'LL ACTUALLY 
BE ALSO INSTRUCTIONAL.
I THINK RESOURCES IS THE THIRD 
KEY QUESTION AND FINALLY.
STATES NEEDS TO THINK BEFORE 
THEY GO TOO FAR DOWN IN THE 
ROAD.
BUT WHAT IT IS THEY PLAN TO DO 
WITH THE INFORMATION.
THOSE ARE GOING TO BE FOUR KEY 
POLICY AREAS THAT STATES ARE 
GOING TO HAVE TO ADDRESS AROUND 
THE ADOPTION OF ANY OF THOSE NEW 
ASSESSMENTS. 
>> AS A STATE POLICYMAKER, MAKES 
THESE DECISIONS, WHAT ROLE DOES 
THIS DOCUMENT AND THE FIVE 
ELEMENTS THAT LINDA OUTLINED.
WHAT ROLE CAN IT PLAY AND SHOULD 
IT PLAY?
>> CAN THE STATE AGREE, CAN THE 
STATE POLICYMAKERS AGREE THIS IS 
OUR GOAL THAT WE WANT TO MOVE 
AWAY FROM THE TEST OF?
I DON'T KNOW THAT THERE ARE MANY 
PEOPLE STILL OUT THERE SAYING 
THAT'S THE BEST WAY TO ASSESS.
IF WE WANT TO LOOK WHAT THE 
BOARD OR COMMISSION CALLS TESTS 
OR TEACHING, THEN IT'S A STATE 
POLICY MAKING ORGANIZATION.
FIRST ADOPT THESE AS OUR FIVE 
GOALS.
IT THEN CAN'T BEGIN TO ANSWER 
THESE OTHER KEY QUESTION ABOUT 
CAPACITY, ABOUT RESOURCE, ABOUT 
WHAT TO DO WITH THE INFORMATION 
IN THE CONTEXT OF THOSE 
IMPORTANT CRITERIA. 
>> LINDA, LET ME TURN FOR A 
MOMENT TO -- PLEASE, IF ANY ONE 
OF YOU JUMP IN AT ANY TIME.
LINDA, LET ME ASK YOU ON THE 
INTERNATIONAL FRONT, YOU 
MENTIONED HOW ONE OF THE 
ELEMENTS IS IMPORTANT THAT THE 
ASSESSMENTS BE INTERNATIONALLY 
BENCHMARKED.
I MIGHT ASK EVERYONE TO OLD 
DECEMBER 3rd.
THIS IS THE TRY ANNUAL RELEASE 
OF THE RESULTS COME DECEMBER 
3rd.
MORE OFTEN THAN CONGRESSIONAL 
ELECTION EVERY THREE YEARS.
HOW DOES THE COMMON CORE 
ASSESSMENT. 
>> SMARTER BALANCE MEASURE UP 
AGAINST THE CRITERIA THAT YOU'VE 
LAID OUT.
YOU AND JIM LAID OUT AND OTHERS 
IN THE REPORT AND PARTICULARLY 
THIS SEASON, THE PROGRAMS FOR 
INTERNATIONAL STUDENT ASSESSMENT 
MEET THESE CRITERIA. 
>> WELL, THE EARLY ANALYSES OF 
THE SPECIFICATIONS OF THE NEW 
ASSESSMENTS OF PARK AND SMARTER 
BALANCE SHOW THEY ARE LIKELY TO 
BE QUITE A BIT MORE AMBITIOUS IN 
MEASURING HIGH ORDER SKILLS AND 
COULD QUITE CONCEIVABLY MEET 
THAT FIRST CRITERION.
THEY ARE GOING TO GO FURTHER IN 
TERMS OF HIGH FIDELITY OF SOME 
TASKS WHERE WE TALK ABOUT ASKING 
KIDS TO DO THE SEARCH.
THERE'S TESTS ON A SMALL SKILL 
OVER THE COUPLE DAYS GET KIDS TO 
RESEARCH THEIR SKILLS AND SO ON.
THEY WON'T MEASURE ALL OF THE 
STANDARDS IN THE HIGH FIDELITY 
WAY AND NEED TO BE COMPLEMENTED 
WITH OTHER ASSESSMENTS AND THERE 
ARE A GROUP OF STATES THAT ARE 
JOINED TOGETHER IN SOMETHING 
CALLED INNOVATION LAB NETWORK.
AND BEYOND THOSE WHO ARE LOOKING 
AT OTHER KINDS OF ASSESSMENTS TO 
USE AS COMPLEMENTS TO THE 
CONSORTIA ASSESSMENTS BUT THEY 
WILL CERTAINLY, WITH THE 
CONSORTIA ASSESSMENT AS CORE.
STATES ARE A LONG WAY MEETING 
THE HIGH QUALITY ASSESSMENT.
I POINT YOU TO STATES LIKE NEW 
HAMPSHIRE, KENTUCKY, SOME OTHER, 
OREGON, THAT ARE ARTICULATING 
THE WAYS IN WHICH THEY WANT 
STUDENTS TO COMPLEMENT OR 
AUGMENT THE BASIC ASSESSMENTS 
WITH ADDITIONAL PRODUCTS AND 
PROJECTS THAT MEET THE REST OF 
THE STANDARDS. 
>> PISA IS CERTAINLY ADVANCED IN 
TERMS OF THE DEGREE OF THINKING 
SKILLS OVER THE CURRENT STATE 
DESKS.
IT DOES ASK STUDENTS TO EXPLAIN 
AND DEFEND THEIR THINKING TO 
APPLY WHAT THEY KNOW TO KNOW 
SITUATIONS.
PRETTY UNIFORMLY, THEY ARE ALSO 
INTRODUCING IN 2015 A 
COLLABORATIVE PROBLEM SOLVING 
TEST AND THEY ARE GETTING READY 
TO INTRODUCE AND ICT COMPONENT, 
INTERACTIVE COMPUTER AND 
LITERACY COMPONENT TO THE 
ASSESSMENT.
SO WE SEE PISA, LIKE OTHER 
COUNTRY, MOVING FAIRLY QUICKLY 
TO INCORPORATE. 
>> MORE AND MORE OF THESE HIGHER 
ORDER SKILLS. 
>> JIM, ANYTHING YOU WANTED TO 
ADD TO THAT?
>> WELL, I THINK THAT LINDA HAS 
CAPTURED THE FACT AT FIRST 
GLANCE, THE INITIAL ANALYSES, 
THE ASSESSMENTS FROM THE TWO 
CONSORTIA DO SEEM TO ALIGN WITH, 
AT LEAST THE FIRST FEW CRITERIA 
AS WE CAN FIGURE IT OUT NOW, 
BECAUSE WE HAVE TO REMEMBER THAT 
THERE'S A LOT OF DEVELOPMENT.
AND WE'RE STILL UNDER WAY.
AND WE WILL ULTIMATELY KNOW FROM 
RESEARCH WHETHER OR NOT SOME 
OTHER CRITERIA, SUCH AS WHETHER 
THEY'RE STRUCTURALLY SENSITIVE 
AND EDUCATIONALLY VALUABLE.
WHETHER THEY ARE VALID, RELIABLE 
AND FAIR.
THAT WILL HAVE TO COME FROM A 
VARIETY OF SOURCES OF EVIDENCE 
THAT WILL INCLUDE EVALUATIONS BY 
EXPERTS BUT THEN IRPEER CAL 
STUDIES AND EMPIRICAL DATA, SO 
IT WILL TAKE A WHILE FOR ALL OF 
THIS TO ROLL OUT.
I THINK THE THING THAT WE HAVE 
TO ALSO REMEMBER ABOUT PISA IS 
THAT IT HAS A NUMBER OF 
STRENGTHS BUT IT'S ALSO DESIGNED 
IN A WAY TO BE IN SOME WAYS 
INSTRUCTIONALLY AND CURRICULUM 
INDEPENDENT.
SO I DON'T KNOW THAT WHEN WE 
LOOK AT AN ASSESSMENT LIKE PISA, 
WE WOULD LOOK AT IT AND SORT OF 
SAY WE COULD SAY RIGHT NOW, 
WETTER IT MEETS ALL OF THE 
DIFFERENT CRITERIA THAT WE HAD 
PROPOSED FOR THE ASSESSMENTS 
THAT WE'RE TALKING ABOUT USING 
IN THE UNITED STATES AS PART OF 
OUR EDUCATIONAL SYSTEM.
BUT THEN, AGAIN, YOU NEED TO 
UNDERSTAND, PISA IN THE CONTEXT 
WHICH IT WAS DEVELOPED AND IT'S 
USED AND IT IS NOT INTENDED TO 
FUNCTION IN THE SAME WAY AS THE 
KIND OF ASSESSMENTS THAT WE'RE 
TALKING ABOUT, WHETHER THE PARK 
OR SMARTER BALANCE OR A SYSTEM 
OF ASSESSMENTS THAT IS SUPPOSED 
TO BE LINKED TO THE COMMON CORE 
STATE STANDARDS OR IN THE FUTURE 
NEXT GENERATION SCIENCE 
STANDARDS.
PISA HAS NICE DESIRABLE ELEMENTS 
ABOUT THE KIND OF KNOWLEDGE AND 
SKILLS THAT THEY ARE TAPPING 
INTO.
THEY HAVE SOME VERY NICE MODELS 
FOR THE WAY IN WHICH THEY HAVE 
IMPLEMENTED TASKS AND ALSO MADE 
USE OF TECHNOLOGY.
SO THERE ARE SOME VERY NICE 
THINGS THERE THAT LINE UP WITH 
SOME OF THE CRITERIA THAT WE'VE 
OUTLINED AND NICE THINGS TO LOOK 
AT FOR ALL THE ASSESSMENTS THAT 
WE USE IN THE UNITED STATES 
MIGHT BE IMPROVED IN TERMS OF 
THEIR KNOWLEDGE AND SKILLS AND 
HOW THEY GO ABOUT COLLECTS THAT 
EVIDENCE. 
>> THANKS TO BOTH OF YOU FOR 
POINTING OUT WHERE PISA DIFFERED 
FROM THE CONSORTIA.
 KRIS, ALL OF YOU SPARKED A 
NUMBER OF QUESTIONS.
A LOT OF PEOPLE ARE ASKING WHAT 
MECHANISMS CAN BE IN PLACE TO 
ENSURE ASSESSMENT MEETS THE 
CRITERIA.
AS YOU SAID STATE OFFICIALS MOVE 
TO -- 
>> I THINK THE KEY IS REALLY TO 
MAKE SURE THIS IS BUILT INTO A 
STATE DEPARTMENT OR STATE BOARD 
OF EDUCATION REGULAR SORT OF 
POLICY PROCESS, BECAUSE SOME 
PEOPLE ARE TO GO BACK AND 
REVISIT AND THEY CONTINUOUSLY 
ASSESS WHERE THEY ARE ON THE 
GOALS.
SO FIRST THEY ADOPT, AND THEN 
YOU HAVE TO CONTINUALLY GO BACK 
AND LOOK TO SEE HOW WE'RE DOING.
BY MEASURING THOSE, I THINK IT 
WILL BE KEY FOR BOARDS TO TAKE 
THAT STEP.
FIRST SETTING OUT THE GOALS AND 
THEN CONTINUOUSLY TRYING TO 
MEASURE THEMSELVES AGAINST IT.
I WOULD ALSO PUT IN A PITCH 
HERE, I THINK THAT IS THE TASK 
THAT BELONGS PROBABLY MORE 
APPROPRIATELY TO A STATE TO A 
STATE BOARD AND GOD LOVE THEM.
STATE LEGISLATURES WHICH I WAS 
UPON, DON'T DO NECESSARILY WELL. 
>> I SHARE THAT SENTIMENT.
PAUL IN NEW HAMPSHIRE ASK JIM 
AND LINDA DO THE AUTHORS 
ANTICIPATE BEING DEVELOPED FOR 
THE EVALUATION OR ASSESSMENT 
SYSTEM?
ANYONE WANT TO TAKE THAT?
>> I COULDN'T HEAR THE LAST PART 
OF WHAT YOU SAID.
SO IF YOU COULD REPEAT THE 
QUESTION, SO, AS THIS DOCUMENT 
LACE OUT THE NECESSARY CRITERIA, 
WILL THERE BE A -- YOU 
ANTICIPATE AN ADDITIONAL 
DOCUMENT THAT IS BEING DEVELOPED 
THAT ACTUALLY PERMITS THAT HELPS 
WITH THE EVALUATION OR 
VALIDATION OF ASSESSMENT 
SYSTEMS?
>> YES, YES.
JIM AND I ARE INVOLVED WITH 
OTHERS TRYING TO DEVELOP SOME 
MORE FINER GRAIN GUIDANCE NOW 
ABOUT WHAT WOULD ALLOW YOU TO 
SORT OF EVALUATE A PROPOSED 
ASSESSMENT OR SYSTEM OF 
ASSESSMENTS TO SEE IF THEY MEET 
THE STANDARDS WHAT KIND OF 
INDICATORS AND STUDIES WILL KNEE 
NECESSARY WE ARE OPENING TO 
CLEAT THAT RELATIVELY SOON. 
>>. 
>> SOG TO LOOK FORWARD TO.
WE'LL SCHEDULE THE WEBINAR 
APPROPRIATELY. 
>> QUESTION FROM PEGGY.
WHERE IT K TEACHERS GET 
SUPPORTER RESEARCHERS TO PREPARE 
STUDENTS FOR THE HIGH QUALITY 
AASSESSMENTS. 
>> I'LL SAY THAT THAT IS 
CERTAINLY GOING TO BE WHEN I WAS 
TALKING ABOUT WHAT RESOURCES 
THIS STATE WOULD NEED.
I THINK THE PROFESSIONAL 
LEARNING WILL HAVE TO BE A BIG 
KEY TO THE WAY STATES THINK 
ABOUT ROLLING OUT NEW ASSESSMENT 
SYSTEMS PARTICULARLY THE HIGHER 
LEVEL-THINKING SKILLS.
THAT WILL BE A STATE 
RESPONSIBILITY.
IT WILL ALSO BE A BIG 
RESPONSIBILITY. 
>> AND THEN I AM DELIGHTED THAT 
THE TWO TEACHER ORGANIZATIONS 
HAVE BOTH STEPPED UP TO THE 
PLATE IN TERMS OF DOING STUFF 
ONLINE SHARING AND I THINK 
THAT'S ALSO GOING TO BE VERY 
IMPORTANT.
BUT I THINK AS STATES MOVE INTO 
THIS ARENA, AND AS LOCAL SCHOOL 
SYSTEMS MOVE INTO THIS ARENA 
THEY NEED TO DEVOTE -- 
>> I WOULD JUST PIGGY BACK TO 
ADD THAT AS STATES UNDERSTAKE 
THIS WORK THERE ARE A LOT OF 
RESOURCES KIND OF FLOWING INTO 
THE SYSTEM AND APPEARING ON THE 
WEB FOR BOTH COMMON CORE 
CURRICULUM AND ASSESSMENT TOOLS.
THAT STATES CAN HELP DISTRICTS 
AND SCHOOLS SORT THROUGH AND 
FIGURE OUT HOW TO IDENTIFY THE 
MOST HELPFUL AND HIGH QUALITY 
RESOURCES.
SOME STATES HAVE SUGGESTED THAT 
THERE OUGHT TO BE SORT OF A YELP 
FOR COMMON CORE TOOLS WHICH 
TEACHERS GET TO RATE THEM OR 
PANELS OR EDUCATORS IN STATES 
MIGHT DO SO.
I THINK THOSE ARE BEGINNING TO 
POP UP AS WELL.
BUT WITHIN THE MULTI STATE 
CONSORTIA THE SMARTER BALANCE 
SYSTEM, FOR EXAMPLE, THERE WILL 
ALSO BE OFFERED A SET OF 
FORMATIVE TOOLS FOR INSTRUCTION 
AND ASSESSMENT THAT ARE MAPPED 
TO THE COMMON CORE AND 
INTEGRATED INTO THE SYSTEM WITH 
THE INTERIM TESTS AND SO ON.
I THINK THAT'S ANOTHER RESOURCE 
THAT WILL BE AVAILABLE FOR 
TEACHERS IN THOSE STATES.
>> AS A CONCERN, I DID SEE A 
GALLUP POLL TODAY WHERE 30 
PERCENT OF STUDENTS TONIGHTth 
THERE'S ANY CHANGE AT ALL IN THE 
WAY THEY TEACH TO IMPLEMENT THE 
COMMON CORE AND ASSESSMENTS 
WHICH I FIND MILDLY ALARMING. 
>> I'M SORRY, LINDA, GO AHEAD. 
>> AS I SAID I HOPE THE 
SUPERINTENDENTS THAT THINK THEY 
ARE THE ONES THAT HAVE ALREADY 
BEEN MOVING IN THIS DIRECTION 
FOR SOME TIME AND HAVE MADE 
STRONG GAINS. 
>> LET'S HOPE. 
>> SO, JIM, EASTON CONNECTICUT 
ASKS ANY SITE VIEW AS A MATTER 
OF FACT FROM A 2009 ARTICLE IN 
WHICH YOU STATED, QUOTE, THE 
POLICY COMMUNITY WANTS ANSWERS 
TO DESIGN AND IMITATION QUESTION 
SO MEASUREMENT SCIENCES CAN 
ANSWER.
THIS IS A SPECIALLY COMPLEX 
TOPIC, THE MOST VALID TWICE 
MEASURE THAT KNOWLEDGE OF SKILL 
AS CROSS TIME.
CLOSE, QUOTE. 
>> I ALWAYS LIKE WHEN SEVERAL 
YEARS AGO THEY COME BACK.
SO IT CONTINUES, AS MUCH AS IT 
OCCURRED IN 2009, HOW WOULD DR. 
PELLEGRINO CHARACTERIZE THAT 
PERFORMANCE TO DATE?
CAN LOCAL EDUCATORS BE AINSURED 
OR CONCERNED?
>> IT'S ALWAYS WONDERFUL WHEN 
SOMEBODY QUOTES YOUR WORDS BUT 
THEN YOU HAVE TO SORT OF NOW 
MAKE A STATEMENT WITH REGARD TO 
WHAT DO YOU THINK ABOUT IT NOW?
I DO THINK THAT SINCE THAT WAS 
WRITTEN AND WHAT HAS HAPPENED IN 
THE ENSUING YEARS, THAT IT'S NOT 
JUST THE MEASUREMENT COMMUNITY 
THAT I THINK IS -- WE NEED TO 
THINK ABOUT, BUT WE NEED TO 
THINK ABOUT WHEN I USED THAT 
QUOTE, I TALKED ABOUT LEARNING 
SCIENCES AND MEASUREMENT 
SCIENCES COMMUNITIES AND I THINK 
WHAT HAS TRANSPIRED OVER THE 
LAST, OH, HALF A DOZEN YEARS IS 
THAT IN BOTH COMMUNITIES THERE'S 
BEEN A FAR BETTER DIALOGUE 
DIALOGUE, ABOUT, FOR EXAMPLE, 
WHAT DOES IT MEAN TO KNOW 
SOMETHING, TO UNDERSTAND 
CONCEPTS AND TO BE ABLE TO 
ENGAGE IN THE KINDS OF KNOWLEDGE 
AND SKILLS WE ARE NOW TALKING 
ABOUT IN THE COMMON CORE AND 
WHAT ARE THE KINDS OF EVIDENCE 
THAT WE WOULD TAKE TO SUPPORT 
THAT STUDENTS HAVE IN FACT 
DEVELOPED THOSE FORMS OF 
COMPETENCY.
SO THE LEARNING SCIENCES 
COMMUNITY HAS PERFORMED HARDER 
TO WHAT KIND OF PERFORMANCES ARE 
WE TALKING ABOUT.
WHAT IS MORE SOPHISTICATED WHAT 
IS LESS SOPHISTICATED.
AND WHAT HAS CHANGED OVER TIME.
THE IT'S BEEN PART OF A DIALOGUE 
IN TERMS OF FIGURING OUT WAYS TO 
MOVE BEYOND THE SIMPLE KIND OF 
MEASUREMENT APPROACHES AND 
SCORING APPROACHES THAT HAVE 
DOMINATED LARGE SCALE TESTING IN 
THE UNITED STATES WHERE WE 
SIMPLY AGGRAVATE PERFORMANCE AND 
GET A TOTAL SCORE TO MORE 
SOPHISTICATED KIND OF EVIDENCE 
MODELS BOTH TO PROVIDE 
INFORMATION THAT'S POTENTIALLY 
USEFUL ON LARGE SCALE AND. 
>> THERE'S STILL A LOT OF WORK 
TO BE DONE BUT IT'S WORK THAT 
REALLY REQUIRES THE COOPERATION 
AND CLAP RACIAL OF MULTIPLE 
COMMUNITIES.
IT REQUIRE DISCIPLINARY EXPERTS, 
PEOPLE EXPERT IN MATHEMATICS, 
LEARNING SCIENCES BECAUSE IT'S 
ONLY WHEN THEY WORK TOGETHER TO 
CLARIFY WHAT THEY'RE TALKING 
ABOUT TALKING ABOUT STUDENTS 
ENGAGING IN ARGUE MENTATION 
USING EVIDENCE FROM TEXTS AND 
HOW WE WOULD GET EVIDENCE FROM 
THAT AND TRANSLATE THAT INTO 
SOME USEFUL SCORING SYSTEM 
WHETHER FOR FEEDBACK FOR 
TEACHERS IN THE CLASSROOM OR 
PURPOSES OF CREATING SOME 
OVERALL SCORE ON AN ASSESSMENT.
WE HAVE MADE PROGRESS.
A LOT MORE NEEDS TO BE DONE BUT 
I THINK WE HAVE BECOME A LOT 
MORE SOPHISTICATED IN ALL OF 
THOSE COMMUNITIES IN TERMS OF 
TALKING TO EACH OTHER, USING 
THAT KNOWLEDGE IN TEN HELPING TO 
DESIGN BETTER ASSESSMENTS FROM 
THE OUTSETS.
I THINK SOME OF THAT IS GOING ON 
AS A KEY PART OF WHAT IS 
HAPPENING.
BOTH SMARTER BALANCE IN TERMS OF 
HOW TO THINK THROUGH TRANSLATION 
OF STANDARDS INTO ASSESSMENT 
FRAMEWORKS AND THEN TO THE 
DESIGN OF TASKS THAT ARE GOING 
TO GIVE THE EVIDENCE THAT IS 
DEEMED USEFUL AND VALID WITH 
RESPECT TO THE MEANING AND 
STANDARDS AND INTENT OF 
STANDARDS AND HOPEFULLY 
TRANSLATE THAT INTO SCORING 
SYSTEMS AND REPORTING SYSTEMS 
THAT WILL BE HELPFUL TO THE 
EDUCATIONAL COMMUNITY. 
>> LINDA, FROM AUSTRALIA, THEY 
ASK HOW IMPORTANT IS THE ROLE OF 
TEACHERS ASSESSMENTS AND HIGH 
QUALITY ASSESSMENT SYSTEM.
CAN YOU SEE A TIME WHERE THEY 
TAKE A MORE SIGNIFICANT ROLE IN 
HIGH-STAKES ASSESSMENT AS THEY 
ALREADY DO IN THE STATE OF 
QUEENSLAND?
QUEENSLAND DOES REALLY WONDERFUL 
WORK WITH A SYSTEM OF ASSESSMENT 
WHICH I'LL EXPLAIN FROM THE 
NONAUSTRALIAN LISTENERS.
INCLUDES PORTFOLIOS OF STUDENT 
WORK LOTS OF PAPERS AND PROJECTS 
THAT ARE DESIGNED TO THE STATE 
CURRICULUM AND SOON THE NATIONAL 
CURRICULUM WHICH IS COMING 
ONLINE AND WHICH ARE SCORED IN A 
MODERATED PROFESSION BY TEACHERS 
WHICH ARE SORT OF CALIBRATED 
THROUGH A STATE PANEL OF 
EDUCATORS AND OTHER EXPERTS TO 
COMMON STANDARD.
SO IT'S A VERY RICH, EDUCATIVE 
PROCESS WHICH ALLOWS KIDS TO 
ENGAGE IN VERY, VERY CHALLENGING 
KINDS OF WORK AND CERTAINLY 
INCLUDES A LOT OF THE TEACHER 
DESIGN AND EVALUATION ELEMENTS 
IN IT.
WHICH OF COURSE DRIVES DEEPER 
UNDERSTANDING AND STRONGER 
INSTRUCTION.
AND I DO THINK THAT IN SOME 
STATES NOW THAT ARE LOOKING AT 
HOW TO AUGMENT, WHAT THE 
CONSORTIA ASSESSMENTS ALLOW THEM 
TO MEASURE IN TERMS OF 
MATHEMATICS AND ENGLISH LANGUAGE 
ARTS WITH DEEPER LEARNING 
ASSESSMENTS THAT ARE MADE MORE 
CONSISTENT THROUGH VARIOUS TOOLS 
AND TECHNOLOGY THAT HAVE COME 
AVAILABLE IN RECENT YEARS THAT 
WE WILL SEE MORE ENGAGEMENT OF 
LOCALLY DEVELOPED RICH 
ASSESSMENTS IN THE SYSTEM THAT 
IS USED, FLEX, IN DISTRICTS THAT 
ARE GRADUATING STUDENTS ON THE 
BASIS OF PORTFOLIOS OF THEIR 
WORK AND SIMILAR SYSTEMS TO 
THAT, BOTH KENTUCKY AND NEW YORK 
HAVE GIVEN WAIVERS TO SCHOOL 
DISTRICTS THAT HAVE DEVELOPED 
THESE KIND OF VERY DEEP, 
RELIABLY SCORED KINDS OF 
ASSESSMENTS AND WE MAY SEE THAT 
BEGINNING TO BECOME MORE COMMON 
IN SOME OTHER PARTS OF THE 
COUNTRY AS YOU, KNOW THE 
COMMITMENT TO THE COMMON CORE 
STANDARDS AND SKILLS THEY REALLY 
REQUIRE US TO ATTEND TO BECOMES 
DEEPER AND STRONGER.
>> WILL ASSESSMENTS BE SCORED 
LOCALLY?
HOW DO YOU TRAIN PEOPLE SCORING 
EXAMS TO ENSURE CONTINUITY, 
PARTICULARLY WE'RE TALKING ABOUT 
THE MUCH MORE ROBUST ASSESSMENT?
>> THAT BIGGIE BACKS ON WHAT I 
WAS SAYING.
MAYBE I'LL SAY ONE MORE WORD IF 
THAT'S OKAY AND OTHERS MAY CHIME 
IN.
WHEN YOU HAVE VERY EXTENDED 
PIECES OF WORK, THEY MAY BE 
SCORED -- CERTAINLY, IN MOST 
CASES, THEY NEED TO BE SCORE AT 
LEAST IN PART BY TEACHERS 
ALTHOUGH THERE IS MORE CAPACITY 
FOR WHAT WE CALL AI SCORING 
THROUGH COMPUTERS.
THERE ARE CLASSES OF WORK WHERE 
THAT KIND OF SCORING CAN BE 
WIDELY DONE AND OTHER CLASSES OF 
WORK WHERE WE'RE STILL LEARNING 
HOW TO USE TOOLS LIKE THAT.
MUCH OF IT IF NOT MOST OF IT 
WILL BE DOWN BY TEACHERS WHO ARE 
MANAGED EITHER ONLINE.
CALIBRATION OF TEACHERS TO SCORE 
CONSISTENTLY WITH BENCHMARKS AND 
RUE BRICKS TO A STANDARD AND/OR 
TEACHERS COLLECTED IN GROUPS 
WHERE A SYSTEM OF MODERATION IS 
USED TO HELP THEM BE ABLE TO 
SCORE CONSISTENTLY TO A 
BENCHMARK, SOME PEOPLE WILL BE 
FAMILIAR WITH ADVANCED PLACEMENT 
SCORING PROCESSES WHERE ATP 
TEACHERS COME TOGETHER ACROSS 
THE COUNTRY AND SCORE ESSAYS 
TOGETHER.
THEY USE A SIMILAR SYSTEM AS DO 
A LOT OF OTHER FOLKS.
IF YOU WANT TO HAVE TEACH EARPS 
SCORING THE LIVELY CONSISTENCY.
THERE HAS TO BE A TRAINING 
PROCESS.
CALIBRATION PROCESS AND OFTEN AN 
AUDITING PROCESS TO COME FROM 
BEHIND AND READ WORK THAT HAS 
BEEN SCORED IN A WAY THAT READS 
EXPECTATIONS, BENCHMARK, 
ET CETERA.
WE DO ACTUALLY HAVE A LOT OF 
EXPERIENCE DOING THIS WELL, 
RELIABLY AND TO A VERY HIGH 
STANDARD.
I THINK WE'VE LEARNED A LOT 
SINCE THE DAYS OF THE EARLY 90s.
IN FACT IN THE LAST ROUND OF 
KENTUCKY'S WRITING PORTFOLIO 
WHICH THEY'VE USED FOR MORE THAN 
15 CAREERS, THE AUDIT SCORING 
WHICH WAS DONE CENTRALLY OF THE 
LOCALLY SCORED WORK FOUND A 
RELIABILITY OR CONSISTENCY 
STATISTIC OF 99% THE LIABILITY 
IN THE SCORES THAT HAD BEEN 
PRODUCED LOCALLY AND THOSE THAT 
WERE SECURED THROUGH THE AUDIT 
PROCESS.
SO I THINK WE NEED TO DEVELOP 
THOSE SYSTEMS WHERE WE DON'T 
HAVE THEM BUT WE DO KNOW HOW TO 
DO IT. 
>> AND THIS IS -- I JUST ADD TO 
THAT, ONE OF THE KEY THINGS 
ABOUT WHAT LINDA JUST DESCRIBED 
IS THE FACT THAT TO PUT IN PLACE 
A SYSTEM SUCH AS THAT REQUIRES 
THERE BE MULTIPLE COME POINTS 
AND THAT THERE BE ADEQUATE TIME 
AND TRAINING FOR EXAMPLE THE 
SYSTEMS USED IN QUEENSLAND OR 
ANY OTHER CASES, USUALLY IT 
TAKES A WHILE FOR THESE SYSTEMS 
TO STABILIZE FOR THE 
PARTICIPANTS TO COME TO REALLY 
UNDERSTAND AND UTILIZE THE 
CRITERIA AND BE ABLE TO PRODUCE 
CONSISTENT JUDGMENTS ON THEIR 
OWN AS WELL AS CONSISTENT WITH 
WHAT THE TARGETS ARE.
THIS GOES BACK TO ONE OF THE 
OTHER ISSUES WHICH IS THAT THESE 
INVESTMENTS IN TEACHER SCORING 
ARE ALSO INVEST MANIES TO 
PROFESSIONAL DEVELOPMENT.
BECAUSE ONE OF THE KEY THINGS 
AND I KNOW THIS IS TRUE IN THE 
AP PROGRAM IS THAT THROUGH THE 
PROCESS OF SCORING TEACHERS COME 
TO MORE DEEPLY UNDERSTAND WHAT 
THE CRITERIA ARE THAT CONSTITUTE 
HIGH LEVELS OF PERFORMANCE AND 
THAT IS REFLECTED BACK IN HOW 
THEY MODIFY THEIR TEACHING 
PRACTICE AS WELL AS THEIR OWN 
ASSESSMENT PRACTICES IN THE 
CLASSROOM BECAUSE THEY HAVE A 
CLEARER SENSE OF WHAT THE 
TARGETS ARE AND THE FORMS OF 
EVIDENCE THAT STUDENTS NEED TO 
PRODUCE IN THEIR OWN CLASSROOMS 
THAT ARE ALIGNED WITH THE 
SO-CALLED EXTERNAL TARGETS THAT 
MIGHT BE INVOLVED IN A 
STANDARDIZED ASSESSMENT PROGRAM, 
WHETHER IT'S SOMETHING LIKE PARK 
OR SMARTER BALANCE OR THINGS 
LIKE INTERNATIONAL BACHELORATE 
WHICH OFFER INVOLVES MODERATION 
PROCESSES OR TEACHERS SCORING 
THE AP EXAMS. 
>> SO A LOT OF QUESTIONS COMING 
IN ON IMPACT OF THESE 
ASSESSMENTS ON POLICYMAKERS.
SO, KRIS, I'M GOING TO START 
WITH YOU OBVIOUSLY.
0 HOW DO YOU THINK POLICYMAKERS 
DEAL WITH LOWER PERFORMANCE ON 
COMPLEX ASSESSMENTS?
WILL THEY HAVE THE PATIENCE?
>> I THINK THAT'S GOING TO BE 
ABSOLUTELY KEY TO THE KEY 
MESSAGE.
AND WE DO HAVE EXAMPLES OF 
PLACES WHERE THAT'S BEEN DONE.
I WOULD HOLD OUT TENNESSEE, FOR 
EXAMPLE, WHEN THEY CHANGED THEIR 
SCORES AND WENT OUT AND DID A 
VERY COMPREHENSIVE JOB OF 
COMMUNICATING WITH, YOU KNOW, 
CROSS AUDIENCES AND SAYING, YOU 
KNOW WHAT, OUR SCORES ARE GOING 
DROP.
THAT'S GOING TO BE A GOOD THING.
THAT'S A SIGN THAT WE ACTUALLY 
DID RAISE OUR EXPECTATIONS FOR 
STUDENTS.
BECAUSE THEY DID THAT.
WHEN SCORES DROPPED THERE WAS 
RELATIVELY LITTLE KERFUFFLE.
I THINK IT'S GOING TO BE 
ABSOLUTELY ESSENTIAL, NOT JUST 
POLICY MAKERS BUT ALSO THE 
GENERAL PUBLIC AND MEMBERS OF 
THE NEWS MEDIA COME AWARE OF THE 
FACT THAT BECAUSE WE ARE RAISING 
EXPECTATIONS WE WILL SEE 
PROBABLY A TEMPORARY DROP IN 
PERFORMANCE.
THAT'S A GOOD SIGN.
THAT'S A SIGN THAT WE ARE 
RAISING EXPECTATIONS. 
>> LET ME FOLLOW UP REAL QUICKLY 
AND ASK OTHERS TO JUMP IN, 
BECAUSE FOLLOWING RIGHT UP IS 
LOU IN NORTH CAROLINA WITH A 
STATEMENT THERE'S AN 
OVERWHELMING SENTIMENT THERE'S 
TOO MUCH TESTING.
FROM LEGISLATORS WHO CONTROL THE 
MONEY AND ASSESSMENTS THAT ARE 
MORE DEMANDING AND FEWER 
MULTIPLE CHOICE ITEMS THAT COST 
MORE TO SCORE.
.
THAT'S ALL RIGHT.
THAT WAS THE FIRST QUESTION THAT 
SAID WHAT IS IT WE NEED TO BE 
ASSESSING?
I THINK AS NEW HIRE LEVEL 
ASSESSMENTS COME IN, SOME OF THE 
OTHER STUFF THAT SCHOOLS ARE 
DOING IS, OUT OF NECESSITY NEED 
TO DROP AWAY.
WE CERTAINLY CAN'T HAVE KIDS 
INVOLVED IN TESTING FOR THE 
ENTIRE 180 DAYS OF A TYPICAL 
SCHOOL YEAR.
ON THE OTHER HAND, IF WE HAVE 
TESTS THAT AT WE SAID AT THE 
BEGINNING OF THIS WEBINAR THAT 
WE'RE TEACHING TO.
THAT'S THE MESSAGE THAT WE'RE 
NOT JUST TESTING.
WHAT WE'RE DOING IS HELPING KIDS 
LEARN.
WE'RE EVALUATING WHAT KIDS 
LEARN.
THAT'S THE ONLY WAY, I THINK, 
THAT YOU CAN CONVINCE 
CASH-STRAPPED STATE LEGISLATORS 
THAT THEY HAVE TO DO MORE THAN 
LOOK FOR A LOWEST COMMON 
DENOMINATOR TEST.
THIS IS MORE THAN JUST A TEST.
THIS IS REALLY A LEARNING 
EXPERIENCE AS WELL. 
>> LINDA AND JIM I KNOW I HEARD 
YOU BOTH SPEAKING ON THIS A 
GREAT DEAL.
ANY THOUGHTS?
>> WELL, I WOULD ADD TO WHAT BOB 
JUST SAID IS THAT WHEN WE HAVE 
THE KIND OF TASKS AND SITUATIONS 
THAT REFLECT THE HIGHER LEVEL 
SKILLS AND THE KINDS OF CRITICAL 
ABILITIES THAT ARE REFLECTED IN 
THE COMMON CORE, WHAT WE'LL HAVE 
IS TASKS, WE SHOULDN'T JUST 
THINK OF THEM AS TESTS BUT THEY 
ARE TASKS THAT ARE INHERENTLY 
LEARNING OPPORTUNITIES SO ONE OF 
THE THINGS TO THINK ABOUT IS THE 
TASKS THAT WE'RE TALKING ABOUT 
THAT WILL CONSTITUTE EVIDENCE OF 
STUDENT LEARNING A, ACHIEVEMENT 
AND STUDENT THINKING ARE 
ACTUALLY VERY GOOD INSTRUCTIONAL 
TASKS AND WHAT THEY DO IS 
PROVIDE INFORMATION THAT 
FUNCTIONS IN AN ONGOING 
ASSESSMENT PROCESS THAT INFORMS 
THE TEACHING PRACTICE BUT IS 
ALSO WORTH HAVING STUDENTS 
ENGAGED IN THE ACTIVITIES.
THOSE WILL BE COMPLEMENTED BY 
PERHAPS STANDARDIZED TASKS THAT 
ARE ADMINISTERED EITHER WITHIN 
THE CLASSROOM OR SUCH AS AT THE 
END OF THE YEAR LIKE THE PARK 
ASSESSMENTS FOR PURPOSES OF 
OBTAINING ADDITIONAL EVIDENCE 
FOR PURPOSES OF SYSTEM 
MONITORING AND ACCOUNTABILITY.
I THINK THE OTHER THING TO 
REALIZE IS THAT RIGHT NOW 
SCHOOLS HAVE BEEN OVERBURDENED 
WITH MANY, MANY ELEMENTS OF 
TESTING THAT PARADE AROUND UNDER 
VARIOUS LABELS SUCH AS BENCHMARK 
TESTING, INTERIM ASSESSMENT, 
FORMATIVE ASSESSMENT AND MANY OF 
THOSE COMPONENTS WHICH DISTRICTS 
OFTENTIMES PAY A LOT OF MONEY 
FOR WILL ACTUALLY BE UNNECESSARY 
AND WILL HOPEFULLY DROP OUT OF 
THE SYSTEM AND THE FOCUS WILL BE 
ON THE KINDS OF HIGH QUALITY 
ASSESSMENTS THAT CAN INFORM 
TEACHERS AND THAT CAN INFORM 
POLICYMAKERS AND THEY WILL BE 
WITH THE KINDS OF TASKS AND 
PERFORMANCES THAT ARE REALLY 
INSTRUCTIONALLY RELEVANT AND 
INSTRUCTIONALLY SUPPORTIVE. 
>> I HAVE TO PIGGY BACK ON THAT, 
BOB, TO REMIND LISTENERS AND 
VIEWERS THAT SOME COUPLE OF 
MONTHS AGO, AEE SPONSORED A 
WEBINAR ABOUT ANOTHER ANALYSIS 
WHICH LOOKED AT THE COST OF 
ASSESSMENT AND WE FOUND THIS WAS 
A STANFORD REPORT THAT THE 
AMOUNT OF MONEY SPENT ON ALL OF 
THOSE BENCHMARK INTERIM TEST 
PREP MATERIALS DOUBLES THE 
AMOUNT OF MONEY THAT'S CURRENTLY 
BEING SPENT BY STATES ON 
TESTING.
IT'S OVER $50 A PUPIL ON 
AVERAGE.
BUT IT DOESN'T ACTUALLY RAISE 
THE QUALITY OF TEACHING, 
LEARNING AND ASSESSMENT BECAUSE 
IT'S ALL GEARED TO THESE VERY 
LOW-LEVEL TESTS.
SO THAT SAME AMOUNT OF MONEY 
SPENDS TO SUPPORT HIGHER QUALITY 
ASSESSMENTS COULD TRANSPORT 
CURRICULUM AND TEACHING IN VERY 
PRODUCTIVE WAYS AND POINT US AT 
A MORE USEFUL LETTERING IN THE 
LONG RUN. 
>> THANK YOU FOR POINTING, 
CALLING THAT PREVIOUS ALLIANCE 
WEBINARS.
I'M HEARING TWO IMPORTANT POINTS 
THAT THE THREE OF YOU ARE 
MAKING.
FIRST OF ALL IS THAT THERE ARE 
OTHER -- THAT IN TERMS OF COST 
OF ASSESSMENTS THEMSELVES, THAT 
THERE ARE OTHER ASSESSMENTS THAT 
WILL BE REPLACED. 
>> THERE ARE OTHER WAYS THAT YOU 
OFFSET THE COSTS AND THAT'S THE 
ISSUE OF COSTS.
ALSO, WHAT I HEARD COME THROUGH 
LOUD AND CLEAR, WITH THE 
CRITERIA THAT HAVE BEEN 
ENUNCIATED.
THIS IS A LEARNING EXPERIENCE AS 
MUCH AS IT IS A TESTING 
EXPERIENCE, SO THERE'S A MUCH 
HIGHER LEVEL OF QUALITY DATA 
BEING PROVIDED TO TEACHERS THAT 
STUDENTS ARE BENEFITTING FROM.
IT'S NOT JUST ABOUT TESTING FOR 
TESTING'S SAKE.
IT'S A LEARNING EXPERIENCE AS I 
RECALL.
THAT WAS ONE OF THE KEY 
DETERMINATIONS OF THE GORDON 
COMMISSION AND OTHERS AS WELL.
LET ME JUST ADD MY PERSONAL 
OBSERVATION AS POLICYMAKERS MOVE 
INTO THIS.
IF YOU'RE GOING TO SPEND $10,000 
TO $12,000 PER STUDENT AND THEN 
BE SATISFIED WITH A $10 TEST, 
WHAT KIND OF CRAZINESS IS THIS?
I MEAN, I GOT A CAR THAT I'M 
HOPING IS STILL 10 THOUSAND 
WORTH AND I SPEND ROUTINELY $200 
A YEAR TO ASSESS IT TO MAKE SURE 
IT'S WHERE IT IS TO BE.
I LIKE TO THINK OUR CHILDREN AND 
CARS COULD AT LEAST GET THE SAME 
ATTENTION.
IN PARTICULARLY, LINDA, YOU MADE 
THIS POINT.
YOUR OPENING REMARKS AS WE MOVE 
INTO AN AREA OF MUCH HIGHER 
STANDARDS, THAT MAKING SURE THAT 
THE NEEDS BY WHICH WE DETERMINE 
WHETHER OR NOT OUR STUDENTS AND 
CHILDREN ARE ACHIEVING THOSE 
STANDARDS AND LEARNING OUTCOMES 
ED, NEED TO KEEP UP WITH THEM.
THE WORST THING I CAN THINK OF, 
YOU MOVE TO A SET OF HIGHER 
STANDARDS BUT KEEP AN OLD SET OF 
ASSESSMENTS THAT WEREN'T 
ADEQUATE TOWARDS HE THEY WERE 
ROLLED OUT AND EACH YEAR BECOME 
INCREASINGLY LESS SO.
THANK YOU VERY MUCH FOR LETTING 
ME DO MY ASSESSMENT THEN ON 
THIS.
BUT YOU ALL HAD SET IT UP.
WE ONLY HAVE A FEW MINUTES.
LET ME TURN TO LINDA, JIM AND 
KRIS TO SEE IF THERE ARE CLOSING 
OBSERVATIONS YOU LIKE TO MAKE.
THANK YOU FOR YOUR 
PARTICIPATION?
>> I'LL JUST ECHO WHAT YOU SAID, 
BOB.
THIS IS A MOMENT IN OUR HISTORY 
WHERE WE'VE MADE AN EXPLICIT, 
PRETTY MUCH NATIONWIDE DECISION 
TO LOOK AMBITIOUSLY FORWARD AT 
RAISING OUR EXPECTATIONS FOR 
LEARNING TO MEET THE DEMANDS OF 
THE SOCIETY AND FIGURING OUT HOW 
ASSESSMENT SYSTEMS CAN FOLLOW 
SUIT AND CAN SUPPORT THAT MOVE, 
IS GOING TO BE REALLY CRITICAL.
SO WE'RE PLEASED THAT THE 
FEDERAL GOVERNMENT HAS ACTUALLY 
BEEN ASKING FOR THIS KIND OF 
GUIDANCE AND THAT THERE'S A 
POSSIBILITY TO REALLY RETHINK IN 
A MAJOR WAY WHAT'S POSSIBLE 
ACROSS THE COUNTRY. 
>> JIM, ANY CLOSING THOUGHTS?
>> I JUST ADD THAT ASSESSMENTS 
CAN BE A CRITICAL FACTOR IN 
IMPROVING THE LEARNING OUTCOMES 
THAT WE SEEK.
WHEN THEY'RE WELL DESIGNED AND 
WELL FOUGHT OUT THEY CAN BE VERY 
SUPPORTIVE OF THE DOOLES WE HAVE 
IN MIND BUT IN WE UNDERMINE THAT 
AND TRY TO DO THINGS CHEAPLY AND 
QUICKLY, WE'LL ACTUALLY WORK 
ACROSS THE PURPOSES THAT WE HAD 
IN MIND.
SO IT'S WORTH THE INVESTMENT.
IT'S WORTH THE TIME TO DO IT 
RIGHT AND TO CAREFULLY MONITOR 
THE QUALITY OF WHAT'S BEING 
DELIVERED SO THAT WE ENGAGE IN A 
CONTINUOUS IMPROVEMENT PROCESS 
WITH THE ASSESSMENT SYSTEMS WE 
DESIGN AND IMPLEMENT. 
>> YOU KNOW, SIX YEARS AGO WE 
PASSED -- SO THIS IS A PART OF A 
LONG EVOLUTION IN THIS COUNTRY'S 
COMMITMENT I THINK THE EXCITING 
THING FOR ME IS A PERSON WHO 
WORKS WITH STATE BOARDS OF 
EDUCATION IS THAT IT'S VERY 
CLEAR THAT THE FIELD OF PLAY IS 
MOVING FROM WASHINGTON BACK TO 
THE STATES AND I THINK THAT'S 
GOING TO MAKE IT A REALLY 
EXCITING TIME TO WATCH WHAT 
HAPPENS IN ALL 50 OF THE STITTS. 
>> AND TO THAT POINT, AS LINDA 
POINTED OUT A NICEWIDE DECISION 
HAS BEEN MADE ABOUT COLLEGE AND 
CAREER READY BUT IT'S A 
NATIONWIDE DECISION THAT'S BEEN 
MADE STATE BY STATE.
IT WAS NOT A FEDERAL DECISION IT 
WAS NOT A DECISION.
IT WAS A DECISION OF 46 STATES 
IN THE DISTRICT OF COLUMBIA.
TO RAISE THEIR STUDENTS TO 
COLLEGE AND CAREER READY.
NOW COMES THE SAME ISSUE OF THE 
STATE ADOPTING ASSESSMENTS THAT 
THEY ALREADY ADOPTED AND ARE 
BEGINNING TO IMPLEMENT.
I WANT TO THANK EVERYONE WHO HAS 
PARTICIPATED TODAY.
IF YOU MISSED ANY OF TODAY'S 
WEBINAR, ARCHIVE, VIDEO AND 
POWER POINT SLIDES WILL BE 
AVAILABLE AT 
WWW.ALLFORED/WEBINAR IN THE NEXT 
DAY OR TWO.
I WANT TO THANK YOU THE 
FOUNDATION FOR MAKING THIS 
WEBINAR POSSIBLE AND OF COURSE 
OUR PARTICIPANTS LINDA DARLING 
HAMMOND, JIM PELLEGRINO AND KRIS 
AMENDSEN.
FOR THE ALLIANCE FOR EXCELLENT 
EDUCATION.
THANK YOU VERY MUCH FOR JOINING 
US.
Categories: Uncategorized
thumbnail

Action Academy

Welcome to the Alliance for Excellent Education’s Action Academy, an online learning community of education advocates. We invite you to create an account, expand your knowledge on the most pressing issues in education, and communicate with others who share your interests in education reform.

Register Now

or register for Action Academy below:


Join the Conversation

Your email is never published nor shared.

What is this?
Multiply 10 by 1 =
The simple math problem you are being asked to solve is necessary to help block spam submissions.

Close

 

Every Child a Graduate. Every Child Prepared for Life.