More Students are Not Meeting NAEP’s Minimum Achievement Expectations: NCES’s Efforts Toward a Better Portrayal of These Students’ Performance
- The two primary measures that the National Center for Education Statistics (NCES) uses to report student performance results for the National Assessment of Educational Progress (NAEP) are average scores on the NAEP reporting scale and the percentages of students performing at each of the three NAEP achievement levels.
- Below
NAEP Basic, since it is not a formal achievement level, does not have a formal name, policy definition, or knowledge and skills description. The absence of a formal description makes it difficult to describe:
- What these students know and what they can and cannot do
- How these students differ from students performing at other NAEP achievement levels.
- As shown in the
first post in this series, the percentage of students performing below the
NAEP Basic level is relatively large and has been trending higher over recent assessment cycles.
- When examined further, the results are more alarming, with higher percentages of students performing below the
NAEP Basic achievement level among some groups and in some jurisdictions.
This is the second of a three-part series highlighting the large and increasing percentage of students performing below the
NAEP Basic level. The
first post in the series reported that between 19 and 40 percent of students, depending on grade and subject, could not even partially master the knowledge and skills that are considered fundamental for solid academic performance in reading and mathematics. In addition, the current pandemic-induced education crisis is heightening the concern about the future of these students. This blog post describes NCES’s efforts to address this concerning trend.
In December 2020, NCES hosted a meeting of nine nationally recognized experts to seek recommendations about how best to inform the public of its concerns about these students and accurately portray their performance. NCES posed the following questions to the panel:
-
Data: What do we need to know about students who score below
NAEP Basic—who are they, what do they know, and what can they do in the various subjects assessed by NAEP?
-
Measurement: Are changes in NAEP procedures (e.g., design, administration, and analysis) necessary to improve the measurement and description of students who score below
NAEP Basic?
-
Reporting: How can NAEP better describe the performance of students who score below
NAEP Basic when reporting results?
The recommendations of the expert panel are grouped into four categories and are described in the following figure.
The panel recommended that performance of students who do not reach the
NAEP Basic level should have its own label instead of being a residual category and the description should clearly state what students know and can do at this new level.
The panel highlighted the need for more information in two key areas: (1) the types of instruction provided to students performing below
NAEP Basic compared with students performing at or above
NAEP Basic, and (2) how students performing below
NAEP Basic approach items on the NAEP assessment compared with students performing at or above
NAEP Basic.
The panel observed that NAEP items skew toward higher difficulty, and some students may therefore become discouraged by what they see as inaccessible items. The panel recommended that the distribution of item difficulty be made to mirror the distribution of student performance.
The panel recommended having an increased focus on reporting descriptive statistics over time for students performing below
NAEP Basic to aid policy-makers, researchers, and others in understanding which students are falling behind and the learning gaps that most need to be addressed.
NCES has made several efforts in response to the panel’s recommendations. For example, the NAEP item development team is exploring how to improve the NAEP assessment instruments. Currently, there are not enough assessment items in the NAEP item inventory to accurately assess the knowledge and skills of students performing at the lower end of the NAEP scale. More items that fully cover the NAEP score scale could provide more detailed information on the knowledge and skills of students whose performance does not reach the
NAEP Basic cut-point score. The NCES reporting team is also identifying ways to provide a more complete portrayal of students performing at all points on the NAEP scales. In addition, NCES has informed the National Assessment Governing Board, the board that makes policy for NAEP, about this alarming trend and the recommendations of the expert panel. Informing the Governing Board helped initiate various discussions on the NAEP achievement levels.
Closing Remarks
NCES hopes to shine a bright light on the problem—a light that could spur action across the entire educational community, including teachers, parents, and policymakers. NCES plans to provide stakeholders with more details on the performance of this group of students on a regular basis. A more accurate portrayal of what students performing below
NAEP Basic know and can do will hopefully contribute to a useful dialogue among stakeholders about the actions that are necessary for all students in this nation to meet the expectations outlined by our policymakers.
The
next post in this three-part series provides a list of helpful resources available from NCES and its parent organization, the Institute of Education Sciences, that could benefit states, districts, teachers, parents, and researchers in their effort to support students performing below
NAEP Basic.