Rakesh Mohan, Director of Idaho’s Office of Performance Evaluations (OPE), delivered a scathing report to the Joint Legislative Oversight Committee today regarding Idaho’s longitudinal data system, (Idaho System of Educational Excellence).

In short, OPE’s conclusions highlighted a series of misteps, starting from the beginning of development, to the current status-quo of dysfunction millions of dollars and years after the program began.

I’ve reported in the past on both ISEE and its classroom companion, Schoolnet previously.  Those posts are worth taking a look at for some history and context of how we arrived at the current problems we have today here and here.  Note:  The report covered today was exclusive to ISEE, not Schoolnet (although a report is coming soon on Schoolnet as well).

A sampling:

Districts felt that the department had not provided adequate financial support or training to prepare for the changes in data collection. Some districts also felt the department failed to consider business practices and data collection processes at a district level when designing the ISEE data elements, format, and system.

OPE reported that districts spend on average two weeks in a single month in preparing the monthly data upload to ISEE.   This becomes a particular burden for smaller districts who cannot afford to hire an additional individual simply to fulfill the ISEE requirement, especially when the district is not entirely sold that the data being reported is helping improve student instruction and achievement.

However, a district’s funding is ties to the monthly upload; no data means no Benjamins.  The report states:

ISEE workload is generally not time consuming enough to justify adding a full time position, but the workload is too time consuming to add ISEE tasks to existing job responsibilities.

In addition, if the district’s staff member who loads and submits data to ISEE makes an error, thousands of dollars in district’s funding is at stake.  No pressure, right?

Regardless of the cause, data errors can result in a substantial loss of funding for districts. For example, a district reported losing thousands of dollars in pay for performance funding by accidentally entering a “1” rather than a “2” for teacher contract type.

Report Highlights

Also consider that districts use a hodgepodge of district level student information systems.  Each one of these systems collects and reports student data in different ways, many of which are not compatible from one system to another without manual data manipulation by staff.  A single statewide uniform system may help alleviate this problem, but districts fear adopting a single system dictated by the state due to their negative experiences with the state mandate to utilize ISEE itself.

…districts are not in a position to implement and manage a statewide student information system. Additionally, stakeholders generally agree that implementing a statewide student information system would meet resistance from districts satisfied with their system vendor.

OPE noted that many stakeholders were excluded from the development process.  In other words, the IT staff at the state level weren’t necessarily talking with the boots on the ground at the district level in asking for certain data elements, how they would like those data elements reported, and why that data would be valuable to the district and/or the state.

Ultimately, the lack of a data governance strategy with program areas at the center perpetuated organizational silos despite the department’s move to centralized data collection. Organizational silos undermine the collaboration, data quality, and data use the department intended ISEE to promote.

An emphasis was given to the committee on the importance of the development of a governance structure that includes all stakeholders in the future in any development or changes of the longitudinal data system in order to help alleviate such problems in the future; particularly by being more inclusive of the different challenges posed by districts of different sizes.

Pete Kohler Interim Chief Deputy responded to the report.  However, he stated a working group meant to persue the reports recommendations has not yet been organized; however, he advised Supt. Ybarra will actively be engaging in making changes to the systems based on the report.

He was asked by the committee what the current IDE timeline is in tackling the issues outlined in the report.  He was noncommittal on a timeline for elements being solved, although he stated a team should be assembled to look into the issue by this summer.

He also noted that he expects some resistance by districts around the state to any changes made at the state level based on their experience in dealing with both the state and developers of ISEE regarding problems the encountered in trying to fulfill the data report mandate.

The committee recommended scheduling a follow-up meeting updating the policy makers on headway made in tackling the continued problems as early as late fall of this year.

Committee members displayed some heartburn of how ISEE was allowed to fall into its current state, but sounded cautiously optimistic the new admonistration under Ybarra would provide an opportunity to recessitate ISEE into a tool that is truly valuable to both districts and their students, as well as providing data the state itself needs.

I will have a more detailed writeup and analysis coming soon.  In the meantime, take a read of the report’s conclusions yourself.