TFP evaluation for a baseline study

TFP evaluation for a baseline study

by Prof. Robert McCormick -
Number of replies: 7

Dear colleagues

This is my first substantive contribution to our discussions of the Teacher Futures Programme, and so I do it with some sense of trepidation!

My main work up until 15th September will be the draft RME framework. This will have an outline of all that has to be done to carry out studies and other data collection to provide the research, monitoring and evaluation of the projects in the various countries. For this part of my task, the focus is on Kiribati and Sierra Leone, so I am addressing particularly Tekaribwa, Anouk, Prince and Muhammad (Junaid). Of course the baseline studies in Kiribati and Sierra Leone are likely to be the basis for that undertaken in the other countries. Not only that, but you have started to discuss common needs across the projects, for example on defining problem-based learning (there was some discussion of a framework for this earlier in the year, and if it has now been done I'd love to see it!).

I attach a short paper on some of my initial thoughts about what may need to be measured to start some discussion of what it is you are trying to achieve. Feel free to pitch in at any level: the overall structure of what I have dealt with or one particular part (e.g. problem-based learning). What will be needed for evaluation are clear definitions of what is meant by particular things to be achieved. Some of this clarity may be in the existing documents for Kiribati and Sierra Leone, which I have yet to read, and so if you tell me some of the things I need to know are there, just point to the particular documents. At this stage I would like you to help me clarify what you want to achieve so that we can construct measures and hence instruments that will capture the achievements. So, for example, if you are trying to enable teachers to create problem-based learning in the classroom, what behaviours are needed to observe to see if teachers are able to create this kind of learning.

I have a list of other things I need to know about the two countries and the projects, but I will hold back on these until I have read more of the documents.

I look forward to the discussion.

Best wishes

Bob

In reply to Prof. Robert McCormick

Re: TFP evaluation for a baseline study

by Betty Ogange -

Dear Bob,

This is to give you a formal 'big welcome' to the programme space and to indicate that you are looking in the right direction by aiming to engage with the Sierra Leone and Kiribati teams as you support the development of the RM&E framework for COL's TE initiative. I am sure Dr. Umar (of Nigeria) is looking keenly as Nigeria is planning a roll-out of the Green Teacher (GT) Programme in September, 2018. The GT is an environmental education programme that emphasizes on Problem Based Learning as its core pedagogy. I hope that Dr. Umar will make time to contribute to this discussion even though the baseline studies will be conducted in Sierra Leone and Kiribati in the first instance. Nigeria is likely to be the next stop.

All other countries (Gambia, Kenya, South Africa and Sri Lanka) are at the very initial stages of engagement with COL and the in-country consultants and some project contact persons are in this forum. I am confident that they are going to contribute to the discussions whenever they find something of interest or related to their country contexts. We hope to have a dynamic and highly adaptive RM&E framework that will be used across all partner countries.

I wish to add that teacher educator capacity in the Teacher Futures programme is built through workshops that enable them to develop resources for the SBTD / INSET / IN-SERVICE programme as well as support and monitor in-school teacher learning.

The outcome statement that is being operationalised here is 'More people become lifelong learners, achieve educational mobility, employment, entrepreneurship, and enhanced opportunities for livelihoods'. We are trying to support the teachers and teacher educators into becoming 'lifelong learners'.

Where there is lack of clarity, as you have aptly pointed out, we hope that the in-country interpretation as presented by the consultants and in the programme documentation can enable us to refine our understanding and presentation of the indicators.

I wish you, and everyone else, a productive engagement!

Betty

 

 

In reply to Prof. Robert McCormick

Re: TFP evaluation for a baseline study

by Prof. Muhammad Junaid -

Dear Prof.

Many thanks for your post containing your initial thoughts on the way forward with the baseline study in Kiribati and Sierra Leone. The proposed three-part Log frame of outputs, outcomes and impacts will be quite ideal for evaluating the impact of a programme that has already been implemented. Presently, the INSET Project in Sierra Leone has not taken-off. So the idea of a baseline before the commencement of the Project in October this year is to determine the extent to which the teacher educators engaged in teaching the distance learning courses in the selected project schools and the student teachers teaching in the schools (i.e. the target groups for the INSET Project) use active learning methodologies, collaborative learning and problem-based learning in their lessons; and to establish what the student learning outcomes are in both public and in-school examinations to serve as the benchmarks for subsequent evaluation of the impact of the INSET Project following its successful implementation. The instruments we need at this stage therefore, are those that will help us establish what indicators will be set for measuring the INSET process and the results, and how these could be measured. I hope that this brief explanation of where we are right now in the INSET Project in Sierra Leone is of some help.

Thanks and best regards,

Junaid.

In reply to Prof. Muhammad Junaid

Re: TFP evaluation for a baseline study

by Prof. Robert McCormick -

Dear Prof. Junaid

Yes you are right on what the purpose of the baseline is. From what you say there are two things at least that I would like more details on:

  • active learning methodologies, collaborative learning and problem-based learning in their lessons
  • student learning outcomes are in both public and in-school examinations

Details of how active learning are depicted in the SBTD toolkit (for example the Teacher self-assessment checklist; pp. 25-6), problem-based learning in the module (don't think I have seen this module), or learner-centred approaches module (pp. 30-31 has a list and no doubt the microlearning will support these).

On the learning outcomes, it would be useful for you and Prince to say what are the appropriate public examinations that could be used (there is a primary final exam. and I assume a JSS grade 3 exam.). I am also looking at environmental awareness questionnaires for students as a possible learning outcome (though not conventional 'test'). Would that be a good idea? Also what are the prospects of using in-school examinations (worried about comparability across schools). We can't construct new tests, but must use existing means.

Look forward to more discussion.

Bob

In reply to Prof. Robert McCormick

Re: TFP evaluation for a baseline study

by Anouk Janssens-Bevernage -

Hi Bob,

You write "there is a collective need to clarify some of the central concepts in the programme (e.g. of problem-based learning) so that instruments can be created that will measure the occurrence (or not) of what is seen as central to the TFP". I'm happy to contribute to this as I have assisted the Kiribati lecturers with this during the learning design practice (during the learning design workshop but also before and after the workshop when I support the team with their resource development).  Should we create a separate thread for this?

Kind regards,

Anouk

 

In reply to Anouk Janssens-Bevernage

Re: TFP evaluation for a baseline study

by Prof. Robert McCormick -

Hi Anouk

Yes that is a good idea, perhaps you could do that under this Forum 4. I assumed you were referring to PBL, but if wider that's fine too. You choose the title!

Best wishes

Bob

In reply to Prof. Robert McCormick

Re: TFP evaluation for a baseline study

by Prof. Muhammad Junaid -

Dear Prof. McCormic,

Thanks for raising these questions. Active learning methodologies refer to both teacher and student related classroom behaviours that focus on active engagement of learners in the learning processes as against being passive recipients of knowledge doled out to them usually through teacher-fronted talk characteristic of the  traditional lecture method. They include group-based and paired work, use of a variety of instructional strategies and materials, including ICTs, activities that engage students in demonstrating understanding, practicing concepts, or dramatizing acts or scenes and others that involve critical thinking and application of knowledge to personal experiences and contexts.

You are right on your concern about the use of in-school learning outcomes which may vary from school to school making generalisation or comparability across schools a bit difficult. Their use can be limited to assessing the progress made in each given school comparing the pre- and post-intervention outcome levels/performance in given tests. Both the pre- and post- tests should be based on what the JSS students should know and be able to do as prescribed by the nationally approved curriculum (to be provided by Brainard). In this context, the impact of the INSET Project can be measured through monitoring the JSS students' learning outcomes using the in-school examinations, the end of JSS Exam and the post-INSET test that will be conducted at intervals in the life span of the Project. Your proposed use of environmental awareness questionnaires will be handy in establishing the benchmarks for assessing the impact of one of the INSET Modules on Teaching and Learning for Sustainable Development.

Look forward to more discussion.

Junaid.

In reply to Prof. Muhammad Junaid

Re: TFP evaluation for a baseline study

by Prof. Robert McCormick -

Thank you all for your thoughts on the evaluation, I have made quite a bit of progress in filling in some of the detail, but it will be later in the week before I have a fuller version of the framework for RM&E to share. I thought meanwhile it would be worth extending the brief I did last time a little more, and the attached outline is for your consideration.

Anouk and others have contributed to a discussion of PBL and how this could be investigated through classroom observation; please add to that discussion. I will try out some of the other instruments in the coming week (e.g. the environmental awareness questionnaire for learners; and questions to teachers on their professional development experience), to widen the discussion. It is important that you do input your comments as all these instruments are evaluating what you are trying to achieve, so your views are central.

I look forward to your comments in the coming days.

Best wishes

Bob