South African learners have performed poorly in national and international assessments. There have been improvements over the years, but much remains to be done. What we need to understand is why learners answer questions the way they do. Diagnostic item analysis is a tool that can help us understand learners’ thinking, enabling us to identify strategies which we can use to improve performance. The Trends in International Mathematics and Science Study (TIMSS) is an assessment of the Mathematics and Science knowledge of learners around the world. South Africa took part in this assessment in 2015 at the Grade 5 and 9 level. In consultation with curriculum and assessment specialists from the Department of Basic Education, the HSRC conducted a qualitative analysis of learners’ responses to the TIMSS released items for Grade 5 Numeracy, and Grade 9 Mathematics and Science to uncover common errors and investigate how learners are answering questions in these subjects.