Effectiveness Analysis 2021/22

Thank you for your participation in makingAchange. This information page is mainly for teachers of participating schools.

The aim of makingAchange is to increase young people’s perception and awareness of climate change and sustainable development, and to strengthen their capacity to act and adapt. To this end, makingAchange implements various teaching/learning formats and continuously reviews their effectiveness. After an initial impulse by makingAchange, the teaching/learning formats (educational interventions) are to be continued independently by the participating schools, universities and associations and will thereby established in the long term.

What can the analysis of effectiveness do and what can it not do?

The results of the effectiveness analysis will be passed on directly to the Federal Ministry of Education, Science and Research. The effectiveness analysis can only determine in which areas and how strongly a teaching/learning format has an overall effect. This requires very high active or passive participant numbers (> 200 students) in a particular teaching/learning format. Therefore, the effectiveness analysis cannot make comparative statements about an individual school, teacher, student, workshop leader, course leader, university, association, trainer, ….

Teachers’ page

At the beginning of the school year, you as a teacher get access to a personal page for your school – the teacher page. The teacher page contains school-specific information and links to the consent form, pre-test, post-test, and final survey. The teacher page is valid for the entire school year. If you are a teacher and have not received access to the teacher page by September 30, please contact the effectiveness analysis team.

Active and passive participants

Active participants are students who directly participate in one of the teaching/learning formats. For carbon accounting or climate peer education, this is direct participation in the workshops. In the case of the interdisciplinary lessons, the phenological observations and the climate week, this is participation as a class.

Passive participants are students who are not participating in a teaching/learning format themselves, but one of their classmates is. Passive participants are not required to provide a statement of agreement. The pre-test and post-test questionnaires can be completed without a consent form, as they do not collect personal data as defined by the GDPR.

Time schedule

Declaration of consent

The statement of agreement regulates the handling of personal data according to the General Data Protection Regulation (GDPR) that is collected during participation in makingAchange. The consent form must be obtained from all students who actively participate in one of the teaching/learning formats of makingAchange. For this purpose, the teacher gives the students the link to the consent form (teacher page) and the students agree online in the statement of agreement together with their parents to participate in the project and the scientific use of the data.

On the teachers’ page, you as the teacher can see which students have already submitted the statement of agreement. The statement of agreement of all active participants must be received at the latest before the start of the educational interventions.

Pre-test

An essential part of the effectiveness analysis in this school year is the complete quantitative survey of impact indicators in all teaching/learning formats. For this purpose, a pre-test online questionnaire with closed and open response options will always be administered to an entire school class at the beginning of the school year. The survey is ideally conducted by you as the teacher during regular class time. The Pretest Online Questionnaire can be completed using any computer, smartphone, or tablet with Internet access.

The link for the pre-test online questionnaire can be found on the teachers’ page. The pre-test online questionnaire cannot be given as homework, because here only individual particularly motivated students would participate and thus the significance about the entire students would be lost. Pretest online questionnaires that are completed class wide as homework are automatically excluded from further analysis by an algorithm. However, it is possible to give the Pretest Online questionnaire as homework to individual students who were not present during the survey (e.g. illness). If you as a teacher would like to review the questionnaire in advance, please indicate your real year of birth when filling it out so that your data can be automatically distinguished from the students’ data.

Which students should complete the pre-test online questionnaire?

For the teaching/learning format CO2-balancing, all students of the class with the most students participating in the CO2-balancing workshops should participate in the pre-test. It is important that here the whole class participates in the pre-test, because it should be examined how big the effect on the classmates is. Here a multiplicative effect through the active participants on the passive participants is expected. Only one class needs to participate in the pre-test.

In the teaching/learning format of the Climate Peer Education, all students of the two classes that have the most students participating in the Climate Peer Education workshops should participate in the pre-test. Here, too, it is important that the entire classes participate in the pre-test, as it is intended to investigate how great the effect on the classmates is. Two classes are needed here to obtain the necessary sample size, since a lot less schools participate in the Climate Peer Education than in the CO2-balancing.

In the teaching/learning format of interdisciplinary lessons, phenological observations, and climate week, the entire class participates in the pre-test.

Kick-off

To kick off the educational interventions, a kick-off event is held in most teaching/learning formats. Information about the kick-off is not found on the teacher’s site, but is usually provided by the relevant project leaders. If students also participate in the kick-off, the pre-test should be completed before the kick-off. Unfortunately, this is not always possible due to time constraints, so in some cases the pre-test may be completed at the latest shortly after the kick-off.

Teaching/learn­ing formats (educational interventions)

Once the consent forms of all active participants have been received, the pre-test has been completed and any kick-off has taken place, the actual educational interventions can begin. There are the following teaching/learning formats in makingAchange.

• CO2-balancing
• Climate peer education
• Interdisciplinary project-based teaching
• Phenological observations
• Climate week
• and own formats that take place outside of makingAchange but are included in the effectiveness analysis.

Post-test

At the end of the school year after all educational interventions have been completed, the post-test online questionnaire is administered to all those students who also participated in the pre-test. The post-test should also ideally be administered by the teacher during class time. The post-test online questionnaire can also be completed using a computer, smartphone, or tablet.

Only data from those students who participated in the pre-test and post-test can be used for the effectiveness analysis. Therefore, each completed post-test online questionnaire is two times as valuable because it then automatically includes the associated pretest online questionnaire in the analysis.

The post-test online questionnaire differs from the pre-test online questionnaire essentially in that it has an additional open-ended question that takes at least 10 minutes to answer. Therefore, the post-test takes a little longer to complete than the pre-test. The link to the post-test online questionnaire will be on the teachers’ page in due course.

Final survey of the teacher

At the end of the school year after the completion of the educational interventions, you as a teacher will be asked to give a short description on the teachers’ page about which classes did what exactly in the project and how you yourself experienced the participation in the educational intervention. This information is very important in order to properly classify the data of the students in your class(es) from the pre-test and post-test.

Other Surveys

There will also be individual qualitative surveys this school year in the form of focus groups and interviews with students and teachers. However, due to the large number of schools and limited resources, this will only be able to take place in a few institutions.

Thank you

Thank you for reading through this information. All teaching/learning formats in makingAchange will only take place so that, in connection with the effectiveness analysis for the Federal Ministry of Education, Science and Research, we can find out which educational formats for climate change and sustainable development are best suited for permanent anchoring in Austria’s schools. Thank you for supporting this process with your commitment.

For further questions and information, please do not hesitate to contact the team of the effectiveness analysis.