By Yasin Afana
On 5–8 September 2016, the International Association for the Evaluation of Educational Achievement (IEA) provided a workshop on ‘Using TIMSS and PIRLS Data for Secondary Analyses’ to the National Authority of Evaluation at the Higher Council of Education, Training and Scientific Research in Rabat, Morocco. This training course was developed following consultation between the IEA and the Moroccan National Authority of Evaluation to establish the training criteria, and their desired outcomes and long-term objectives.
The workshop provided an overview of the aims of the IEA’s Trends in International Mathematics and Science Study (TIMSS) and Progress in International Reading Literacy Study (PIRLS) studies, their theoretical and measurement frameworks, and the most appropriate methods of analysis for the data. Participants were also given training in how to develop and implement self-designed analyses plans, using the IEA’s own well established software tool, the IDB Analyzer.
The training was conducted in three parts. The first part was a brief review of all the information released from previous TIMSS and PIRLS studies, namely the theoretical frameworks, international reports, encyclopedias, technical reports, user guides, and international databases. The second part was an in-depth introduction to the sampling and survey design of large-scale assessments, with a specific focus on TIMSS and PIRLS studies, and their implications for statistical data analysis. In the third part of the training, the participants were introduced to the practical analysis of TIMSS and PIRLS data, and guided step-by-step through the process. The IEA IDB Analyzer (provided free-of-charge by the course organizers) handles all issues related to the analysis of large-scale assessment data. All examples and practical assignments were tailored to the Moroccan national context. Participants were encouraged to develop their own example research questions, use statistical analysis to answer these questions, and present the results to the other course members, who in turn were encouraged to provide feedback and discuss ideas for further research.
The 10 participants from the National Authority of Evaluation at the Higher Council of Education, Training and Scientific Research in Rabat were inspired by the workshop, and worked individually when required, and as a team when reviewing and providing feedback. All participants ably demonstrated that they had achieved the objective of the hands-on training; after this workshop, they were all able to:
- Comprehend the conceptual and theoretical underpinnings of IEA studies;
- Understand the sampling design and data structure of IEA databases, and their implications for secondary analysis;
- Recognize the necessity of applying specific statistical tools to analyze data from IEA studies;
- Retrieve IEA databases, international reports, technical documentation and questionnaires from the IEA’s website;
- Prepare IEA data for analysis using the software provided (the IDB Analyzer);
- Develop a sound analysis plan, taking into consideration the specific requirements of data, as well as their hypotheses; and
- Implement basic cross-national analyses of IEA data using the IDB Analyzer, both interpreting and presenting the results.
At the end of the workshop, all participants were positive about the training course, and showed great enthusiasm for using what they had learned to extend their educational research work. A follow-up training course was proposed, to focus on other educational research topics and consolidate the skills developed during this first course.
In Morocco, the training was provided by Yasin Afana, a researcher at the IEA Hamburg, currently also working on his PhD thesis in association with the University of Leicester (UK). Mr Afana can be contacted at firstname.lastname@example.org.
Our team of consultants can help you build research skills capacity among the stakeholders in your organization or education system who want or need to participate in international comparative studies or develop national assessments focused on monitoring educational outcomes.
For further information about our training services, please consult our website.