NAEP Technical DocumentationDevelopment of the Mathematics Indices

For the 2015 mathematics assessment, several indices of policy interest were developed that satisfied both theoretical criteria based on content analysis, and empirical criteria based on multivariate statistical techniques. This resulted in the creation of several new reporting elements. The development of the 2015 mathematics indices can be summarized in three main steps:

Question selection. A content analysis of the existing subject-specific 2015 mathematics survey questionnaires was conducted to select sets of the questions that were theoretically interpretable and meaningful as a conceptual unit, as potential indices to measure specific constructs of interest.
Examination of empirical relationships. Factor analysis was used to explore and verify the empirical properties of the data. Construct validity of the potential indices was evaluated through factorial validity with respect to the survey question responses, and the convergent and discriminant validity of the factor with respect to other factors. If the factor had the expected pattern of relationships and non-relationships, the construct validity of the factor as representing the intended index was supported.
Index scoring. The partial credit item response theory (IRT) model was used to scale the indices. Scaling of the index items was first conducted to get the item parameters and was based on the marginal maximum likelihood methodologies. After the parameters were estimated, expected a priori (EAP) scores were calculated as the estimate of the index score. Then, the EAP scores were transformed to have a mean of 10 and a standard deviation of 2, and were reported on a scale from 0–20.

Indices were formed for grades 8 and 12 mathematics, but the grade 4 survey questionnaire did not include a sufficient number of questions that would allow for the creation of indices.

Index of Students' Views on Mathematics

The tables below present the items forming the index of students' views on mathematics at grades 8 and 12, respectively. This index was designed to measure students' interest in and enjoyment of mathematics. At grade 8, students were asked to indicate how often they agreed with two statements about their mathematics class (never or hardly ever, sometimes, often, or always or almost always). They were also asked to indicate the extent to which they agreed with two statements about mathematics (strongly disagree, disagree,agree, or strongly agree). At grade 12, students were asked to indicate the extent to which they agreed with two statements about mathematics (strongly disagree, disagree, agree, or strongly agree). They were also asked to indicate how often they agreed with three statements about their mathematics class (never or hardly ever, sometimes, often, or always or almost always).

Items in the index of eighth-grade students' views on mathematics, grade 8 mathematics combined national and state assessment: 2015
How often do you feel the following way in your math class? Fill in one oval on each line.
Response categories
Item			Never or hardly ever	Sometimes	Often	Always or almost always
M824904	a.	The math work is engaging and interesting.	A	B	C	D
M824905	b.	I am learning.	A	B	C	D
Please indicate how much you DISAGREE or AGREE with the following statements. Fill in one oval on each line.
Response categories
Item			Strongly disagree	Disagree	Agree	Strongly agree
M820901	a.	Because math is fun, I wouldn't want to give it up.	A	B	C	D
M820904	b.	I like math.	A	B	C	D
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.

Items in the index of twelfth-grade students' views on mathematics, grade 12 mathematics national assessment: 2015
Please indicate how much you DISAGREE or AGREE with the following statements. Fill in one oval on each line.
Response categories
Item			Strongly disagree	Disagree	Agree	Strongly agree
M820905	a.	Mathematics is one of my favorite subjects.	A	B	C	D
M820907	b.	I take mathematics because it will help me in the future.	A	B	C	D
How often do you feel the following way in your mathematics class? Fill in one oval on each line.
Response categories
Item			Never or hardly ever	Sometimes	Often	Always or almost always
M824901	a.	I have a clear understanding of what my mathematics teacher is asking me to do.	A	B	C	D
M824904	b.	The mathematics work is engaging and interesting.	A	B	C	D
M824906	c.	I am learning mathematics.	A	B	C	D
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.

Index of Students' Computer Use in Mathematics

The table below presents the items forming the index of students' computer use in mathematics at grade 8. This index was designed to measure students' use of computers to complete mathematics-related activities. Ten items asked students how often they used a computer for math at school, how often they used different types of computer programs when doing math for school or homework, and how often they used computers to get help with mathematics (never or hardly ever, once every few weeks, about once a week, two or three times a week, or every day or almost every day).

Items in the index of eighth-grade students' computer use in mathematics, grade 8 mathematics combined national and state assessment: 2015
Response categories
Item			Never or hardly ever	Once every few weeks	About once a week	Two or three times a week	Every day or almost every day
M814301	a.	How often do you use a computer for math at school?	A	B	C	D	E
When you are doing math for school or homework, how often do you use these different types of computer programs? Fill in one oval on each line.
Response categories
Item			Never or hardly ever	Once every few weeks	About once a week	Two or three times a week	Every day or almost every day
M816001	a.	A spreadsheet program for math class assignments	A	B	C	D	E
M816201	b.	A program that presents new math lessons with problems to solve	A	B	C	D	E
M816301	c.	The Internet to learn things for math class	A	B	C	D	E
M816401	d.	A calculator program on the computer to solve or check problems for math class	A	B	C	D	E
M816501	e.	A graphing program on the computer to make charts or graphs for math class	A	B	C	D	E
M816601	f.	A statistical program to calculate patterns such as correlations or cross tabulations	A	B	C	D	E
M816701	g.	A word processing program to write papers for math class	A	B	C	D	E
M825001	h.	A program to work with geometric shapes for math class	A	B	C	D	E
How often do you use e-mail, instant messages, blogs, or text messages to do any of the following? Fill in one oval on each line.
Response categories
Item			Never or hardly ever	Once every few weeks	About once a week	Two or three times a week	Every day or almost every day
M820603	a.	Get help with math from someone other than your teacher, family, classmates, or friends	A	B	C	D	E
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.

As noted above, partial credit IRT models were estimated for each index, in order to estimate the EAP scores. The following tables show the IRT parameters for the 2015 mathematics indices.

IRT parameters for items in the index of eighth-grade students' views on mathematics, grade 8 mathematics combined national and state assessment, by item: 2015
Item	b_j	d_j1	d_j2	d_j3	d_j4	d_j5
M824904	0.12	1.10	-0.04	-1.06	†	†
M824905	-1.55	0.99	-0.15	-0.84	†	†
M820901	-0.08	1.37	0.18	-1.55	†	†
M820904	-0.26	1.06	0.35	-1.40	†	†
† Not applicable. NOTE: The number of d_ji parameters is one less than the number of categories for the item. For items scaled with the partial credit model, parameters a and c are not estimated. The a parameter value is exactly one and the c parameter is exactly zero. For item j, b_j represents a location parameter related to item difficulty, and d_ji represents the category threshold parameter for category i of item j; d_ji may not sum to zero because of rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.

IRT parameters for items in the index of twelfth-grade students' views on mathematics, grade 12 mathematics national assessment, by item: 2015
Item	b_j	d_j1	d_j2	d_j3	d_j4	d_j5
M820905	0.18	0.78	0.12	-0.90	†	†
M820907	-0.58	0.83	0.48	-1.30	†	†
M824901	-0.61	1.26	-0.23	-1.04	†	†
M824904	0.31	0.99	0.01	-0.99	†	†
M824906	-1.03	0.96	-0.06	-0.91	†	†
† Not applicable. NOTE: The number of d_ji parameters is one less than the number of categories for the item. For items scaled with the partial credit model, parameters a and c are not estimated. The a parameter value is exactly one and the c parameter is exactly zero. For item j, b_j represents a location parameter related to item difficulty, and d_ji represents the category threshold parameter for category i of item j; d_ji may not sum to zero because of rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.

IRT parameters for items in the index of eighth-grade students' computer use in mathematics, grade 8 mathematics combined national and state assessment, by item: 2015
Item	b_j	d_j1	d_j2	d_j3	d_j4	d_j5
M814301	0.82	0.04	-0.09	0.03	0.03	†
M816001	1.00	-0.23	0.16	0.08	-0.01	†
M816201	0.72	0.12	0.12	-0.05	-0.19	†
M816301	0.76	0.33	0.00	-0.06	-0.28	†
M816401	0.80	0.10	0.11	0.03	-0.24	†
M816501	1.08	-0.03	0.13	0.12	-0.22	†
M816601	1.24	-0.22	0.32	0.01	-0.11	†
M816701	1.15	-0.23	0.15	0.11	-0.04	†
M825001	1.10	-0.04	0.11	0.01	-0.09	†
M820603	0.73	0.14	0.10	0.08	-0.32	†
† Not applicable. NOTE: The number of d_ji parameters is one less than the number of categories for the item. For items scaled with the partial credit model, parameters a and c are not estimated. The a parameter value is exactly one and the c parameter is exactly zero. For item j, b_j represents a location parameter related to item difficulty, and d_ji represents the category threshold parameter for category i of item j; d_ji may not sum to zero because of rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.

The EAP scores generated from the partial credit IRT models were transformed to reporting index scores ranging from 0 to 20, with a mean of 10 and a standard deviation of 2, as noted above.

As a reporting aid, cut points were used to divide the grade 8 and 12 students' views on mathematics index into low, moderate, and high classifications, and the grade 8 students' computer use in mathematics index into never, once every few weeks, about once a week, and twice a week or more classifications.

To form these classifications, response categories for each item were first coded as numerical values (e.g., the first response category strongly disagree was coded as 1; the second response category disagree was coded as 2). Note that for all of the indices, each index value corresponds to a response average (e.g., for the grade 8 students' views on mathematics index, a transformed index score of 5.3 corresponded to an average response of 1, or an average response of strongly disagree). In general, higher response averages correspond to smaller index values, and lower response averages correspond to larger index values. The index scores were divided into a range of classifications based on the response averages. The cut points for each index were determined based on the average response to the relevant set of items. For instance, for the index of students' views on mathematics, grade 8 students were classified as follows:

Students with index scores associated with an average response less than the second response category (a transformed index score of 8.0) were classified as low on the index. That is, students who on average responded less than disagree on a question with response options strongly disagree, disagree, agree, and strongly agree generally reported that mathematics was not fun for them and that they did not like mathematics, and were classified as low on the index.
Students with index scores associated with an average response greater than or equal to the second response category but less than the third response category (a transformed index score of 10.3) were classified as moderate on the index.
Finally, students with index scores associated with an average response greater than or equal to the third response category were classified as high on the index. That is, students who on average responded agree or higher generally reported that mathematics was fun for them and that they liked mathematics, and were classified as high on the index.

The tables below show the index values, transformed scale scores, and score classifications for the mathematics indices.

Index values and transformed scores for the index of eighth-grade students' views on mathematics, grade 8 mathematics combined national and state assessment: 2015
Classification	Index value	Transformed score	Percentage of students
Low	-2.33828	5.3	1
	-1.90698	6.2	2
	-1.55811	6.9	3
	-1.25650	7.5	5
Moderate	-0.97852	8.0	6
	-0.70785	8.6	9
	-0.43219	9.1	11
	-0.14178	9.7	13
High	0.17072	10.3	15
	0.51069	11.0	15
	0.88444	11.8	9
	1.30786	12.6	6
	1.82304	13.7	6
NOTE: Detail may not sum to totals because of rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.

Index values and transformed scores for the index of twelfth-grade students' views on mathematics, grade 12 mathematics national assessment: 2015
Classification	Index value	Transformed score	Percentage of students
Low	-2.29060	5.4	1
	-1.88720	6.2	1
	-1.56845	6.9	1
	-1.29810	7.4	3
	-1.05629	7.9	4
Moderate	-0.83170	8.3	6
	-0.61734	8.8	8
	-0.40831	9.2	10
	-0.20056	9.6	11
	0.00998	10.0	11
High	0.22796	10.5	11
	0.45936	10.9	9
	0.71266	11.4	8
	1.00157	12.0	7
	1.35129	12.7	5
	1.81310	13.6	4
NOTE: Detail may not sum to totals because of rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.

Index values and transformed scores for the index of eighth-grade students' computer use in mathematics, grade 8 mathematics combined national and state assessment: 2015
Classification	Index value	Transformed score	Percentage of students
Never	-1.10948	7.8	12
	-0.60072	8.8	9
	-0.29782	9.4	8
	-0.09662	9.8	8
	0.04954	10.1	8
	0.16291	10.3	6
	0.25520	10.5	6
	0.33309	10.7	5
	0.40072	10.8	5
	0.46075	10.9	4
Once every few weeks	0.51500	11.0	4
	0.56481	11.1	3
	0.61113	11.2	3
	0.65472	11.3	3
	0.69605	11.4	2
	0.73557	11.5	2
	0.77364	11.6	2
	0.81060	11.6	2
	0.84681	11.7	1
	0.88254	11.8	1
About once a week	0.91796	11.8	1
	0.95316	11.9	1
	0.98833	12.0	1
	1.02373	12.1	1
	1.05972	12.1	#
	1.09653	12.2	#
	1.13447	12.3	#
	1.17378	12.4	#
	1.21479	12.4	#
	1.25801	12.5	#
Twice a week or more	1.30403	12.6	#
	1.35368	12.7	#
	1.40794	12.8	#
	1.46816	12.9	#
	1.53622	13.1	#
	1.61481	13.2	#
	1.70814	13.4	#
	1.82280	13.7	#
	1.97002	13.9	#
	2.17019	14.3	#
	2.46150	14.9	#
# Rounds to zero. NOTE: Detail may not sum to totals because of rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.

Last updated 02 December 2021 (ML)

Printer-friendly Version