Skip to main content Skip to main navigation menu Skip to site footer
Articles
Published: 2022-12-31

Indonesian Madrasah Competency Assessment: Students’ numeracy based on age

Universitas Islam Negeri (UIN) Sunan Ampel Surabaya
Universitas Islam Negeri Sunan Ampel Surabaya
Universitas Islam Negeri Sunan Ampel Surabaya
UIN Maulana Malik Ibrahim Malang
AKMI Numeracy Age difference Madrasah students

Galleys

Abstract

[English]: This study aims to analyze primary madrasah students’ numeracy based on their age difference (10-12 years old) and to confirm whether the older the students, the better their academic ability when compared to younger students as prior studies suggest. This study used the results of the year 2022 Indonesian Madrasah Competence Assessment (AKMI) in East Java comprising 71244 students. Students’ numeracy was categorized into five proficiency levels: need support, basic, proficient, skilled, and need creative space. Data shows that students who participated in AKMI were dominated by 11-year-old students. In each age group, the spread of the proficiency level is similar, for example, the largest number of students in each group is at the proficient level. Moreover, students in the 11-year-old group have the highest average score, but the difference between the group is not significant. This indicates that they have slightly better numeracy skills than the other groups. Despite the slight difference, the distinct characteristics of each age group could be identified, for example, the number of students, proficiency levels, and average numeracy scores.

[Bahasa]: Penelitian ini bertujuan menganalisis numerasi siswa Madrasah Ibtidaiyah (MI) berdasarkan perbedaan umur (10-12 tahun) dan mengonfirmasi apakah siswa yang lebih tua memiliki kemampuan akademik yang lebih baik daripada siswa yang lebih muda sebagaimana ditunjukkan oleh penelitian terdahulu. Penelitian ini menggunakan data hasil Asesmen Kompetensi Madrasah Indonesia (AKMI) di Jawa Timur yang melibatkan 71244 siswa. Numerasi siswa dikelompokkan dalam lima kategori kemahiran, yaitu perlu pendampingan, dasar, cakap, terampil, dan perlu ruang kreasi. Data menunjukkan bahwa peserta AKMI terbanyak adalah siswa usia 11 tahun. Pada setiap kelompok umur, sebaran level kemahiran tidak berbeda, misalnya, sebagian besar siswa pada setiap kelompok umur berada pada level cakap. Selain itu, siswa pada kelompok umur 11 tahun memiliki rata-rata nilai numerasi tertinggi, namun perbedaan setiap kelompok tidak signifikan. Hal ini menunjukkan bahwa siswa usia 11 tahun memiliki numerasi yang sedikit lebih baik. Walaupun memiliki perbedaan yang tidak signifikan, perbedaan karakteristik setiap kelompok umur tetap dapat diidentifikasi, misalnya jumlah siswa pada tiap kategori usia, level tingkat kemahiran literasi numerasi, dan rata - rata skor literasi numerasi.

Downloads

Download data is not yet available.

Introduction

One of the educational issues in Indonesia, which is under public's radar in general and educators in specific is students' performances in numeracy and literacy (Perdana & Suswandar, 2021). The result of a national assessment shows that Indonesian students' numeracy is under-performed (Setyawati et al., 2021). Indonesian Madrasah Competency Assessment (AKMI) conducted by the Ministry of Religious Affairs (MoRA) (Kusaeri et al., 2022) also indicates the same result. Similarly, results of international assessments, such as PISA (OECD, 2019) and TIMMS (Mullis et al., 2016) have not shown improvement for Indonesian students.

Numeracy skills are essential for students (Liu, 2020), which help them understand, analyse, and solve problems (Patriana et al., 2021). Moreover, it makes the use of mathematics more effective in the context of daily life and work (Geiger et al., 2015). Numeracy is considered a term used to identify knowledge and abilities needed to accommodate mathematics in daily life in various types of relevant contexts (Askew, 2015).

The concern about students' numeracy in Indonesia has triggered the government to issue some related policies. Since 2002, the government has begun to reform mathematics education (including the assessment process) through the Indonesian Realistic Mathematics Education (PMRI, the Indonesian version of Realistic Mathematics Education or RME) (Kusaeri, 2020). The reform was carried out two years after participation in the PISA survey. This reform was inspired by an evident relationship between PISA and RME (Zulkardi et al., 2020). The implementation of PMRI is expected to be an innovation in mathematics instructional practices and assessment in Indonesia. Another policy is the introduction of HOTS (Higher Order Thinking Skills)-based problems in the National Examination since 2016 (MoEC, 2018). Problems used in the exam involve a stimulus with a model of presenting daily life problems (problem-solving). Thus, students are expected to become more familiar with PISA-like problems (Kusaeri, 2019). However, such policies have not managed to improve students' numeracy significantly.

Since 2020, the MoEC has initiated a reform on the National Examination. It is now called Minimum Competency Assessment (AKM). AKM is designed to measure reading literacy and numeracy as cognitive learning outcomes (Indahri, 2021). In a similar effort to improve students' numeracy, MoRA has implemented Indonesian Madrasah Competency Assessment (AKMI) in 2021. It is a national diagnostic test, involving literacy in mathematics, science, reading, and social culture. The test was conducted for madrasah students in grades 5 (10-12 years old), 8, and 11. In this case, students who participated in AKMI are diverse in terms of age.

Some studies indicated that the age of students influences their numeracy skills (Whitely et al., 2021). However, age differences in learning practices are often neglected (Abdurrahman, 2010). Bujuri (2018) found that there were differences in abilities between students based on age. Students aged 12 years have better cognitive abilities than students aged 10 years and 11 years. Likewise, Whitely et al. (2021) show that older students tend to be superior in terms of academic achievement and self-confidence than younger students. This study, using a large pool of data (71244 primary students) as its added value, aims to further identify students’ numeracy skills based on their age- whether the age consistently determines students’ numeracy skills as prior studies suggest or not.

Numeracy in AKMI

Numeracy is considered a term used to identify the knowledge and abilities needed to accommodate mathematics in daily life in various types of relevant contexts (Askew, 2015; Geiger et al., 2015). It is a person's ability to use symbols and numbers related to mathematics to solve problems, analyse information in the form of graphs, charts, tables, and others, and interpret the results of analysis for the decision-making (WEF & BCG, 2015). Similarly, Ekowati et al. (2019) define numeracy as the ability to understand, apply, analyse, and interpret problems involving symbols, models, or mathematical language delivered in spoken or written form and related to daily life. Furthermore, it is related to the ability to think critically in understanding and analysing reading content by using reasoning through the application of mathematical concepts, procedures, facts, and/or tools in calculations with various representations (graphics/tables/charts or other representations) to interpret, predict, and/or make decisions (Mahdiansyah & Rahmawati, 2014). Key terms related to numeracy are mathematics, contextual problems, decision-making, and reasoning. Numeracy includes some components including context, domain content, and cognitive level (Wijaya & Dewiyani, 2021). The context could be personal, work, social, or scientific.

In the context of AKMI (MoRA, 2022), numeracy is defined as students' ability to critically understand, analyse, and reason on the content of texts by applying the concept, procedure, fact, and mathematical tools in calculations or measurements involving figures or basic mathematical symbols contextually, either abstract or real, which is presented in various representations (graph/table/chart or others) to interpret, predict, and make a decision. The context used in numeracy problems involves personal, employment, social, and scientific. The domain is numbers, algebra, geometry and measurement, and statistics and probability. There are three levels of cognitive level, namely knowing, applying, and reasoning. The numeracy test items in AKMI are very diverse, including multiple choice, true/false, complex multiple choice, matching, and short entries. One of the differences between AKMI and other tests such as PISA or AKM is the given stimulus for test items. It serves as an introduction for students to think about. A sample of the items in the form of complex multiple-choice item can be seen in Figure 1.

Figure 1. A sample of test item involving complex multiple choice in AKMI

Numeracy Skills and Students’ Age

Several previous studies (Balala et al., 2021; Durda et al., 2020; Whitely et al., 2021) focus on the comparisons of students' numeracy skills based on age differences. Whitely et al. (2021) reveal that age highly influences the development of students' numeracy skills. This is in line with the theory of the development of student's cognitive abilities (Ibda, 2015). In a technical finding, Napsiyah et al. (2022) show that students with high cognitive levels can use numbers and symbols, interpret information in various forms, and interpret analysis results to predict and make decisions. Meanwhile, students with low cognitive levels can only analyse information in various forms. Moreover, a study by Balala et al. (2021) shows that apart from gender, demographic characteristics, and socioeconomic status, student age significantly influences students' numeracy skills. The relationship between age and numeracy does not only occur among school-aged people but also occurs in adults. The link between age and numeracy skills in adulthood can be seen in the result of PIAAC (Program for the International Assessment of Adult Competencies) (Durda et al., 2020). This test shows that someone at a productive or mature age has superior cognitive abilities compared to other age groups. In short, levels of students' cognitive abilities follow their development (Bujuri, 2018).

Methods

This is a qualitative study, where data was presented narratively (Sugiyono, 2007). Tables, charts, or graphs are used to support data visualization. This study fully referred to the year 2022 AKMI data in East Java province. There were 71244 5th-grade students who participated in a computerized multistage adaptive testing (MSAT) that come from 7356 primary madrasah. The MSAT was designed for three booklets, comprising 12 items in each booklet. Students were given 90 minutes to accomplish 36 test items.

The test was developed through seven comprehensive stages: establishing frameworks, item specifications, stimulus and items drafting, internal reviews, external reviews, items tryouts, and revisions. There were five kinds of test items developed: multiple choice, complex multiple choice, matching, true-false, and short entries. The tryout of the developed items resulted in a high reliability score (0,91) and 0,25- 0,72 interval for item discrimination power, both of which show that the items can provide accurate information and differ in students' ability in numeracy.

In the data analysis, students were firstly grouped based on their age: 10, 11, and 12 years old. In each group, students' scores were categorized into five levels of proficiency (Table 1) and calculated to obtain average scores used for comparison. The results of the analysis were presented in tables, graphs, or diagrams with narrations or interpretations.

Proficiency level Score interval Description
Need creative space 70.01–100 Students are able to represent mathematical objects or situations, use problem-solving strategies, and reason on related content.
Skilled 55.01–70 Students are able to represent mathematical objects or situations, use problem-solving strategies, and reason on problems with a high level of difficulty in related content.
Proficient 40.01–55 Students are able to represent mathematical objects or situations, use problem-solving strategies, and reason on problems with a medium level of difficulty in related content.
Basic 25.01–40 Students are able to represent mathematical objects or situations, use problem-solving strategies, and reason on problems with a low level of difficulty in related content.
Need support 0–25 Students are not able to represent mathematical objects or situations, use problem-solving strategies, and reason on related content.
Table 1. The level of students’ numeracy

Findings

Figure 2 shows the number of primary madrasah students who participated in the year 2022 AKMI based on the age category. AKMI participants were dominated by 11-year-old students (47329 participants) followed by 10-year-old students (9752 participants). Meanwhile, the least number of participants were students aged 12 years (4163 participants). Referring to the government regulation of school age, the age requirement for primary school entrance is 7 years. Accordingly, students aged 11 years were the majority. Table 2 presents students’ numeracy level referring to their age group. In addition to the table, Figure 3 shows the percentage of students’ numeracy levels within each group.

Figure 2. Year 2022 AKMI participants based on age groups

Proficiency level Age group
10 years old 11 years old 12 years old
Need support 1204 2784 268
Basic 5355 12945 1112
Proficient 11769 28198 2476
Skilled 1421 3390 307
Need creative space 3 12 0
Total 19752 47329 4163
Table 2. Students’ proficiency level in numeracy based on their age

Figure 3. The percentage of students’ numeracy level in each age group

Overall, the results of the 2022 AKMI in numeracy reveal several important findings. First, despite having a non-proportional curve, each group has a similar distribution of proficiency level. Second, the majority of students (around 60%) in each age group were on the proficient level or the middle level. This could be good news about madrasah students’ numeracy in Indonesia. Lastly, students at the top two levels (skilled and need creative space) were less than 10% of the whole students. This would be a challenging part of the educational reform to improve students’ numeracy.

The total number of 10-year-old students participating in the 2022 AKMI in East Java was 19752 students. After being grouped based on their level of proficiency, the majority of students aged 10 years were on the proficient level (Table 2). At this level, students were able to represent mathematical objects or situations, use problem-solving strategies, and reason on problems with a medium level of difficulty in related content. This represents almost 60% of the group (Figure 3). In contrast, only three students were at the highest level (need creative space). These students were able to represent mathematical objects or situations, use problem-solving strategies, and reason on related content.

There were 47239 students, which is around 66% of all participants from all age groups, in the 11-year-old group that took the numeracy test. Over one-third of them (12945 students) were on the basic level of numeracy. At this level, students were able to represent mathematical objects or situations, use problem-solving strategies, and reason on problems with a low level of difficulty in related content. The difference between the lowest level (need support) and the skilled level was approximately one percent, which is not significant. In the need support level, students were not able to represent mathematical objects or situations, use problem-solving strategies, and reason on related content.

Of all age groups, the 12-year-old group has the least number of students, at 4163 or nearly 6%. The spread of the students' numeracy level in this group is akin to other age groups. However, no one at this age reached the top level of numeracy.

Figure 4 presents the average score of students' numeracy score in all age groups. Overall, there is no significant difference between each age group’s average numeracy. Students in the 11-year-old group have the highest average (42.49). Meanwhile, students aged 10 and 12 years have slightly below-average scores, 42.46 and 42.42 respectively. Referring to the difference and the nature of this study, which relies on the year 2022 AKMI numeracy data, two possible conclusions could be made. First, the 11-year-old student has slightly better numeracy than other age groups. Second, there is no significant difference at all. Both of these deviate from prior studies (Whitely et al., 2021) that show older students have better numeracy.

Figure 4. The average score of students’ numeracy based on the age category

Conclusion and Implications

The year 2022 AKMI test in numeracy, which involves 71244 primary students (aged 10-12 years) from 7356 madrasah in East Java, shows that most of the students (nearly 60%) have a proficient level of numeracy. Less than 1% of students are at the top level of numeracy (need creative space) and not more than 10% of students top the two levels (skilled and need creative space) of numeracy. Moreover, the numeracy level within each age group is distributed in a similar pattern. In terms of average numeracy score, no significant difference is found within the age group. This suggests that age difference does not determine the numeracy level despite prior studies (e.g., Whitely et al., 2021) suggest otherwise. In this case, the current study even found that 11-year-old students have slightly better average numeracy scores on the test. This result could possibly be affected by the fact that students’ age difference is only one to two years. Greater differences in age might result in distinct findings.

Students' proficiency levels found in this study pose a challenge for (mathematics) teachers in specific. They need to reflect upon their instructional practices- whether it supports students' development of numeracy or not. If it is not, then they are expected to design a mathematics learning that facilitates and supports the development of the skill. As there are differences in numeracy skills among students of different ages, it is critical to consider age differences in each grade by reviewing rules governing the minimum and maximum age limits for school admission at school levels. Indeed, factors affecting wide gaps in students’ age in certain grades should be highly considered and properly managed. Thus, the distinction in the student's age does not significantly cause differences in students’ numeracy skills.

References

  1. Abdurrahman, M. (2010). Pendidikan bagi anak berkesulitan belajar. Rineka Cipta.
  2. Askew, M. (2015). Numeracy for the 21st century: A commentary. ZDM-International Journal on Mathematics Education, 47(4), 707–712. Doi: 10.1007/s11858-015-0709-0
  3. Balala, M. M. A., Areepattamannil, S., & Cairns, D. (2021). Investigating the associations of early numeracy activities and skills with mathematics dispositions, engagement, and achievement among fourth graders in the United Arab Emirates. Large-Scale Assessments in Education, 9(13), 1–21. Doi: 10.1186/s40536-021-00106-4
  4. Bujuri, D. A. (2018). Analisis perkembangan kognitif anak usia dasar dan implikasinya dalam kegiatan belajar mengajar. LITERASI (Jurnal Ilmu Pendidikan), 9(1), 37. Doi: 10.21927/literasi.2018.9(1).37-50
  5. Durda, T., Gauly, B., Buddeberg, K., Lechner, C. M., & Artelt, C. (2020). On the comparability of adults with low literacy across LEO, PIAAC, and NEPS. Methodological considerations and empirical evidence. Large-Scale Assessments in Education, 8(1), 1–34. Doi: 10.1186/s40536-020-00091-0
  6. Ekowati, D. W., Astuti, Y. P., Utami, I. W. P., Mukhlishina, I., & Suwandayani, B. I. (2019). Literasi numerasi di SD Muhamadiyah. ELSE (Elementary School Educatio Journal), 3(4), 93–103. http://journal.um-surabaya.ac.id/index.php/pgsd/article/view/2541/1798
  7. Geiger, V., Forgasz, H., & Goos, M. (2015). A critical orientation to numeracy across the curriculum. ZDM - International Journal on Mathematics Education, 47(4), 611–624. Doi: 10.1007/s11858-014-0648-1
  8. Ibda, F. (2015). Perkembangan kognitif: Teori Jean Piaget. Intelektualita, 3(1), 27-38.
  9. Indahri, Y. (2021). Asesmen nasional sebagai pilihan evaluasi sistem pendidikan nasional. Jurnal Masalah-Masalah Sosial, 12(2), 195–215. Doi: 10.46807/aspirasi.v12i2.2364
  10. Kusaeri, K. (2019). Penilaian sikap dalam pembelajaran matematika. Jurnal Pendidikan Matematika, 4(2), 111–121. https://ejournal.unsri.ac.id/index.php/jpm/article/view/816/229
  11. Kusaeri, K. (2020). Reorientasi penilaian pembelajaran Matematika: Dulu, kini, dan mendatang. Surabaya: Universitas Islam Negeri Sunan Ampel.
  12. Kusaeri, K., Yudha, Y. H., & Kadarisman, Y. P. Hidayatullah, A. (2022). Do instructional practices by madrasah teachers promote numeracy? In the Proceedings of the International Conference on Madrasah Reform (pp. 1-5). Doi: 10.2991/assehr.k.220104.001
  13. Liu, H. (2020). Low-numerate adults, motivational factors in learning, and their employment, education and training status in Germany, the US, and South Korea. ZDM-Mathematics Education, 52(3), 419–431. Doi: 10.1007/s11858-019-01108-x
  14. Mahdiansyah & Rahmawati. (2014). Literasi matematika siswa pendidikan menengah: analisis menggunakan desain tes internasional dengan konteks Indonesia. Jurnal Pendidikan dan Kebudayaan, 20(4), 452–469. Doi: 10.24832/jpnk.v20i4.158
  15. Ministry of Education and Culture (MoEC). (2018). Ringkasan eksekutif hasil Ujian Nasional 2018 jenjang SMA/MA. MoEC.
  16. Ministry of Religious Affairs (MoRA). (2022). Framework kerangka kerja soal dan kisi–kisi Asesmen Kompetensi Madrasah Indonesia. MoRA.
  17. Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2016). TIMSS 2015 International results in mathematics. http://timssandpirls.bc.edu/timss2015/international-results/
  18. Napsiyah, N., Nurmaningsih, N., & Haryadi, R. (2022). Analisis Kemampuan numerasi matematis siswa berdasarkan level kognitif pada materi kubus dan balok. JagoMIPA: Jurnal Pendidikan Matematika dan IPA, 2(2), 45–59. Doi: 10.53299/jagomipa.v2i2.183
  19. OECD. (2019). PISA 2018 results: Combined executive summaries (Vols. I, II & III). OECD. https://www.oecd.org/pisa/Combined_Executive_Summaries_PISA_2018.pdf.
  20. Patriana, W. D., Sutama, S., & Wulandari, M. D. (2021). Pembudayaan literasi numerasi untuk asesmen kompetensi minimum dalam kegiatan kurikuler pada sekolah dasar Muhammadiyah. Jurnal Basicedu, 5(5), 3413–3430. Doi: 10.31004/basicedu.v5i5.1302
  21. Perdana, R., & Suswandar, M. (2021). Literasi numerasi dalam pembelajaran tematik siswa kelas atas sekolah dasar. Absis: Mathematics Education Journal, 3(1), 9-15. http://journal.univetbantara.ac.id/index.php/absis/article/view/1385/pdf
  22. Setyawati, M., Aisyah, H., & Kusaeri, K. (2021). Developing the problem of numeration literacy with Islamic context. MaPan, 9(2), 347-363. Doi: 10.24252/mapan.2021v9n2a11
  23. Sugiyono. (2007). Statistika untuk penelitian. Alfabeta.
  24. WEF & BCG (World Economic Forum & The Boston Consulting Group). (2015). New vision for education: Unlocking the potential of technology. World Economic Forum.
  25. Whitely, M., Phillimore, J., & Moorin, R. (2021). The effect of a child’s relative age on numeracy and literacy test results: An analysis of NAPLAN in Western Australian government schools in 2017. Australian Educational Researcher, 48(2), 249–265. Doi: 10.1007/s13384-020-00399-4
  26. Wijaya, A., & Dewiyani, S. (2021). Framework asesmen kompetensi minimum (AKM). MoEC.
  27. Zulkardi. Putri, R. I. I., & Wijaya. A. (2020). Two decades of realistic mathematics education in Indonesia. In M. van den Hauzel-Panhuizen (Ed.), International reflections on the Netherlands didactics of mathematics (pp. 325-340). Springer.

How to Cite

Kusaeri, K., Dwisanti, C., Yanti, A., & Ridho, A. (2022). Indonesian Madrasah Competency Assessment: Students’ numeracy based on age . Beta: Jurnal Tadris Matematika, 15(2), 148–156. https://doi.org/10.20414/betajtm.v15i2.558