Understanding the Limitations of Descriptive Research and Data
By Jenny Nagaoka, Shelby Mahaffie, Alexandra Usher, and Alex Seeskin
At the To&Through Project and the UChicago Consortium, we have done a lot of critical reflection over the past year about the ways we show up as individuals and as an organization, the limitations of descriptive research and data, and the importance of framing our findings around equity. The introduction to our annual report on the Educational Attainment of Chicago Public School Students is a step in this direction, and we have excerpted it below. Our hope is that by sharing a slightly edited section of the report more broadly, we can contribute to the much-needed conversation about the role of research and data in creating a more equitable education system.
In a city with both a long history and present reality of segregation and systemic racism, access to high quality education in Chicago has never been equitable.¹ Over the past several years, Chicago’s educators and community leaders have elevated an all-too-delayed dialogue about the systematic barriers facing communities of color, and we hope our annual look at the attainment of Chicago Public Schools (CPS) students on five key milestones can contribute to that discourse.² Though it is an imperfect and incomplete metric (especially when used to assess individual students), the differences in educational attainment by race/ethnicity, gender, and disability status are reflective of the different learning opportunities and career pathways to which different groups of students have access. Our goal is for readers to come away with a clearer understanding of the current state of educational attainment in CPS and be motivated to take part in conversations and action to dismantle the oppression inherent in our current systems.
Like many reports from the To&Through Project and the University of Chicago Consortium on School Research (UChicago Consortium), the annual attainment report is descriptive, meaning that it seeks to answer the how, where, when, and who questions of the problem, rather than the why. This is important: sometimes in education, describing what is happening in careful detail is critical in order to help practitioners and other stakeholders see the effect of their practice. But leaving the why up to the reader also runs the risk that people will intentionally or unintentionally ascribe educational outcomes solely to the choices and capacity of CPS students, families, and communities, disregarding the broader and longstanding impact of racism.³
Therefore, before moving forward with the analysis, it is critical to state plainly that the differences in attainment that we see across our findings are due to a long history of racist and oppressive policies and structures and to the historical and ongoing oppression of people of color in Chicago and in the United States. As such, these data are meant to be consumed as part of a collaborative dialogue about the inequitable policies, systems, and practices that prevent CPS students, and particularly Black and Brown students, from reaching their academic potential.
First, there is a long history in Chicago of systemic racism that has included intentional economic and educational disinvestment in communities of color, and it is vital that we see these data in the historical and sociological history of Chicago and its education system.
Moreover, as we look at the outcomes for CPS students, there are several things that are important to keep in mind. First, there is a long history in Chicago of systemic racism that has included intentional economic and educational disinvestment in communities of color, and it is vital that we see these data in the historical and sociological history of Chicago and its education system.⁴ Unless Chicago makes bold, intentional changes, our current system will sustain these inequities and continue to actively exclude communities of color. The responsibility for making changes to the system and creating equitable pathways to these attainment milestones belongs squarely with adults and institutions in Chicago. Although students and families must be at the table for these decisions, the responsibility for change cannot lie with them.
Looking at data that is disaggregated by race/ethnicity, gender, and disability status can arouse feelings of vulnerability, shame, and fear that can prevent honest dialogue across differences.
Second, it is critical for readers to understand that the data we present in the report can be difficult to discuss. Looking at data that is disaggregated by race/ethnicity, gender, and disability status can arouse feelings of vulnerability, shame, and fear that can prevent honest dialogue across differences. Readers, particularly White readers, should embrace this vulnerability, while surfacing and rejecting any intentional or unintentional biases that connect outcome data to racist narratives of ability.
The data in our annual report on educational attainment represent individual students, who every day face — and overcome — systemic barriers, and whose voices should be elevated as experts of their own lived experiences.
Finally, while important, the data are inherently insufficient to understand students’ experiences at CPS and after graduation. Due to data limitations, we are unable to report on outcomes for students who pursue pathways other than a college degree, such as opportunities in the military or workforce. We are also limited in our disaggregation by the data CPS has collected each year on race/ethnicity, gender, disability status, and other student identifiers.⁵ Most importantly, student and family perspectives are necessary to fully understand the barriers that face students of color and students with disability status in Chicago. Ideally, readers are putting the limited, but vital quantitative data in the report in conversation with other research and their own experiences in the field. There is a considerable body of research on the role that systemic racism plays in Chicago’s schools and history that we recommend as a starting place for readers seeking to learn more.⁶ The data in our annual report on educational attainment represent individual students, who every day face — and overcome — systemic barriers, and whose voices should be elevated as experts of their own lived experiences.
To read the full report, please visit: https://toandthrough.uchicago.edu/postsecondary-attainment-indices
¹ Ewing (2018); Payne (2008); Todd-Breland (2018).
² Chicago Public Schools has launched several efforts — centered in the district’s new Office of Equity — at examining how the city’s education systems disproportionately and persistently create added barriers for black and brown students and communities.
³ Chicago Beyond Equity Series (2019).
⁴ Drake & Clayton (1945); Henricks, Lewis, Arenas, & Lewis (2017); Moore (2016); Rothstein (2017); Sampson (2011); Wilson (1987).
⁵ Historically, CPS has collected data that groups students into one of two gender categories: male and female. Additionally, the racial categories available in our data do not accurately reflect the full spectrum of races and ethnicities embodied by CPS students. Many students do not fit into one of these categories, but we believe that there are still insights to be gained from analysis of this data. We hope in the future to be able to report data that more fully describes the identities of CPS students.
⁶ See footnotes 1, 3, and 4.
