A very interesting report which highlights the difference data science is playing in education.
Finding 1: Having enough information is seldom the decisive factor in making most education decisions; instead, decision-makers desire to have sufficient government capacity.
Enacting education policies, changing programs, and allocating resources are complex decisions that demand weighing multiple factors, such as having sufficient capacity and financial resources, having enough information, and having the support of the public. So where do data and information fall within a decision-maker’s cost-benefit analysis?
We found that information is not as important as technical capacity, financing, and political support. Some decisions, however, depend more on having sufficient data and information, such as creating or abolishing schools or grades, and testing students. One possible explanation could be that leaders feel they need strong justification (via an evidence base) for these decisions which could become easily politicized.
Finding 2: Education decision-makers use evidence to support the policymaking process, for both retrospective assessment and forward-looking activities
But while information may not be the most decisive factor in education decisions, its role is significant. We found that decision-makers in the education sector are more likely to use data and analysis as compared to other sectors (such as health and governance), including for forward-looking purposes, such as design and implementation of policies or programs, as well as retrospective assessments of past performance. As shown in Figure 2, most education sector decision-makers (over 70 percent) report using data or analysis fairly consistently throughout the policymaking process.
Finding 3: Education decision-makers most often use national statistics from domestic sources and program evaluation data from international sources.
Decision-makers overwhelmingly rely on national statistics from domestic sources and program evaluation data from international organizations. The high use of national statistics points to the salience of such data for each country, including, for example, dropout rates for primary school students by district or municipality, the number of schools providing secondary education in each village, or pupil-teacher ratios in urban vs. rural areas.
Finding 4: Education decision-makers consider administrative data and program evaluations most essential, and want more of the latter, signaling a gap between need and supply.
We asked leaders about their wish list—what types of information would they want more of? We found that those who allocate and manage resources place a premium on administrative data (e.g., number of schools, teachers, students) and government budget and expenditure data (e.g., school-level budgets, expenditure per student). Meanwhile, those working on personnel management need teacher performance data, whereas leaders tasked with overseeing instructional matters need program evaluation data and student-level assessment data. Given respondents’ wish lists, we identified four opportunities for data producers to respond to unmet demand: (1) program performance and evaluation data; (2) budget and expenditure data; (3) student-level assessment data; and (4) teacher performance data.
Finding 5: Education decision-makers value domestic data that reflect local context and point to policy actions, and improving the timeliness and accessibility of information will make it more helpful.
Having identified some of the gaps that exist in meeting the needs of education decision-makers, we asked what producers and funders of data should do better or differently to meet the data demands. Leaders said that data from both domestic and international sources were most helpful when they provide information that reflects the local context. They also viewed information from international sources as most helpful because it provides policy recommendations (43 percent) and is often accompanied by critical financial, material, or technical support (36 percent). Leaders viewed domestic data as helpful when it was available at the right level of aggregation, as well as timely, trustworthy, and insightful.
When asked what improvements producers could undertake to make data more valuable, respondents suggest improving the timeliness and accessibility, as well as improving data disaggregation, accuracy, and trustworthiness. The respondents requested data from the national government, in particular, to be more accessible and disaggregated.
Finding 6: Decision-makers strongly support strengthening their countries’ education management information system (EMIS) to bolster their education data ecosystem.
Beyond finding general areas of improvement for education data, we also asked respondents to rank a list of specific solutions. Respondents largely agreed on the seven solutions proposed, rating all of them as “extremely important”, on average. But of the seven solutions, the recommendation to strengthen the EMIS within the education ministry resonated with the highest number of respondents.