Prairie State Wire

Illinois State Board of Education Technical Advisory Committee for Assessment and Accountability met June 4 & 5.

Here is the agenda provided by the committee:

Day 1: Tuesday, June 4

8:15 Coffee and light breakfast available

8:30 - 8:45 1. Welcome, March Meeting Recap, and Agenda for Current Meeting

Following welcome and introductions, we will review the March meeting summary and agenda for the current meeting.

8:45 – 9:15 2. ISBE Update

ISBE will provide relevant updates that affect TAC work.

9:15 - 10:45 3. Illinois Assessment of Readiness (IAR) and Illinois Science Assessment (ISA)

3A. Form Construction

Pearson will review the process of form construction for the IAR and ISA, including test and calibration specifications as well as relevant operational constraints, and discuss psychometric targets for Test Information Functions (TIFs) and Test Characteristic Curves (TCCs).

3B. Procedures for Calibration and Equating

Pearson will review the procedures and specifications for calibration and equating.

3C. Differential Item Functioning (DIF)

Pearson will review when and how DIF analyses are produced, a summary of DIF findings by program, what happens when items are flagged for C-DIF, and how results are documented.

Focus Questions for the TAC

• Are procedures for form construction, calibration and equating, and DIF, consistent with professionally acceptable practices?

• Given the potential for new performance levels to be established for the IAR and ISA, are there any areas in the TIF that need to be bolstered?

• What refinements, improvements, or additional studies should ISBE consider and why?

• What suggestions does the TAC have regarding ongoing monitoring and documentation?

10:45 - 11:00 BREAK

11:00 - 12:00 4. Exploring Student Choice on the IAR

ISBE is interested in learning more about student choice on the IAR. For example, what are the potential benefits and risks of allowing students to choose from among performance task prompts? Pearson will share what they’ve learned from other states and identify the key constraints that should be considered.

Ultimately, ISBE wants to ensure that any path forward preserves core technical properties of the IAR such as score reliability, inferential validity, systemic fairness, and overall comparability of different possible versions of the IAR. 

Focus Questions for the TAC

• Which options may be most promising for supporting different kinds of student choice on the IAR?

• What aspects of the underlying rationales for supporting student choice require clarification to support a meaningful deliberation of the issue?

• What evidence should be collected to support these options?

12:00 - 1:00 LUNCH

1:00 - 2:00 5. Unified Assessment Standard-setting

The Center will provide a general update on the broader plans to implement the Unified Standard Setting process as discussed in March. Next, ISBE has decided to hold a workshop in June with an advisory group to develop policy definitions and guidance. This workshop will be facilitated by the Center, which will present the expected outcomes, agenda, and methodology for this workshop.

Focus Questions for the TAC:

• Does the proposed methodology for the workshop seem reasonable and defensible given ISBE’s goals for this work?

• What next steps should ISBE plan to ensure success and technical quality of the unified standard setting process?

2:15 – 2:30 BREAK

2:30 – 4:00 6. Representing Academic Achievement in the School Accountability System

In previous TAC meetings, the Center led a discussion on how the academic indicator might be changed to better represent the foundational programmatic goals of ISBE. In this meeting, the Center will provide some options for calculating academic achievement based on these discussions, demonstrating the impact of alternative computational approaches using a sample of legacy data.

Focus Questions for the TAC:

• Which, if any, of these computational changes are advisable based on their design and impact?

• What additional follow-up studies and/or methodological adjustments does the TAC recommend to the Center (or ISBE) for further analyses to support an empirical deliberation of computational options?

• What other options does the TAC recommend for adjusting the achievement (or related) indicators in the system to accomplish ISBE’s goals?

4:00 - 4:30 Public Comment, Debrief, and Review Action Items

Following public comment, we will identify action items and issues that need further discussion on day 2 or at future meetings.

8:15 Coffee and light breakfast available

8:30 – 10:00 7. Graduation Rate

ISBE has identified some challenges with how the graduation rate is currently included in school accountability. For example, graduation is a very narrow band, so it is difficult to differentiate performance. Additionally, students with disabilities may remain enrolled until age 22, which impacts the computation of the 4-year graduation rate.

ISBE will describe these and other core issues of concern while the Center will provide a brief review of how other states handle such issues in statewide school accountability settings.

Focus Questions for the TAC:

• Should the graduation rate computations for federal accountability purposes be altered given ISBE’s identified challenges? If so, which model(s) from other states or complementary computational approaches are most promising for ISBE to explore?

• For each of the possible alternate approaches, what are some possible risks and unintended consequences that may arise from the computational adjustments, either for individual schools or the system overall?

10:00 – 10:15 Break

10:15 – 11:15 TAC Planning Session

ISBE and the TAC will identify action items from the meeting and priorities for follow-up.

11:15 - 11:30 Public Comment

11:30 Adjourn Day 2 / END OF MEETING

https://www.isbe.net/Documents_TAC/060424-Agenda.pdf