Environmental awareness questionnaire

Environmental awareness questionnaire

by Prof. Robert McCormick -
Number of replies: 12

Dear all

I have indicated that learners' environmental awareness would be measured as an impact, and that I had hoped we could use a well-validated instrument. I attach the barebones of the instrument published in:
Leeming, F.C., Bracken, B.A and Dwyer, W.O. (1995), Children’s Environmental Attitudes
and Knowledge Scale: Construction and Validation, The Journal of Environmental Education, 26 (3): 22-33.

The full instructions to students are not given here. If we can use this with minimal change, we can be sure this is a reliable instrument along with some could construct validity. The real issue is its face and content validity, i.e. is it measuring what is relevant in the environmental education being promoted in the Programme and for the countries involved. Obviously there will be some elements that need to be localised for each of Sierra Leone and Kiribati. For example, one question asks: ‘Which uses the most energy in an average house in the United States?’ Also some items may need to be updated. Indeed, the face and content validity will need to be checked for both countries. Changing one or two questions and adjusting a few will be ok, anything more will need a new instrument, unless someone knows of one that is as well validated.

While it is tempting to invent an instrument to suit the Programme, there is no time to ensure it will be worth using as an evaluation instrument with the reliability needed.

Best wishes

Bob

However, let's see what your first reactions are!


[1] .

In reply to Prof. Robert McCormick

Re: Environmental awareness questionnaire

by Anouk Janssens-Bevernage -

Hi Bob,

Thanks for this. I'll let Tekaribwa speak for the Kiribati group, but my first reaction is that a lot of questions would need to be changed to make sense in the Kiribati context. The environmental problems are very specific to the island. A lot of problems with waste on the sides of the road, but little air pollution. Dimmer light bulbs are probably not available (shopping is very limited - Tekaribwa?), there are very few animals, animal fur would be unknown in this climate, same with bird houses, I don't think there is recycling system (might be wrong but I haven't seen any), etc.

So this would mean adjustments to more than 2 questions. How about we remove the problematic (as in, too context specific) questions all together? The tool could be reduced to fewer questions and still be meaningful?

On the other hand there would be questions that need to be added to a Kiribati questionnaire to cover environmental issues that are specific to the islands (and will probably not be relevant to other countries in this programme). 

Kind regards,

Anouk

In reply to Anouk Janssens-Bevernage

Re: Environmental awareness questionnaire

by Prof. Robert McCormick -

Dear Anouk, Tekaribwa and all,

Yes I thought there would be these kinds of problems with the questionnaire, especially for Kiribati! I think I would rather substitute other questions that follow the same form as those that are replaced. Best first step would be to mark those that are inappropriate, and if possible suggest others to replace or that modifies them (choosing alternative options in an item - distractors is the technical term - is more difficult than you think), to see the scale of the problem. I wonder if there are any questionnaires in Australasia/Pacific region. One thing that has to be borne in mind is that, though of course the questionnaire must reflect with young people's lives. there are also planetary issues they should be aware of, but that depends on the environmental education that is offered to them.

Clearly more thought needed!

Best wishes

Bob

In reply to Prof. Robert McCormick

Re: Environmental awareness questionnaire

by Prof. Muhammad Junaid -

Dear Prof,

I share many of Anouk's observations on the environmental awareness questionnaire. Many of the questions need to be revised to suit the peculiar circumstances and context of Sierra Leone; others need to be expunged for lack of relevance and new and different questions need to be introduced. Plus, I think the questionnaire is too long. I will go through and mark those that I think will be more appropriate for Sierra Leone.

Best wishes,

Junaid.

In reply to Prof. Robert McCormick

Re: Environmental awareness questionnaire

by Prof. Muhammad Junaid -

Dear Prof,

I share many of Anouk's observations on the environmental awareness questionnaire. Many of the questions need to be revised to suit the peculiar circumstances and context of Sierra Leone; others need to be expunged for lack of relevance and new and different questions need to be introduced. Plus, I think the questionnaire is too long. I will go through and mark those that I think will be more appropriate for Sierra Leone.

Best wishes,

Junaid.

In reply to Prof. Muhammad Junaid

Re: Environmental awareness questionnaire

by Prof. Robert McCormick -

Dear all

Thanks for your comments so far on this questionnaire (but Prof. Junaid I would still like the marked-up version you mentioned in your last message). Given the initial reactions I have considered a few more environmental questionnaires and I attach my review to this message.

Unfortunately there is no obvious replacement, and I'm inclined to stick with the CHEAKS questionnaire. Nevertheless I have tried to give a good account of each of the new ones and to come up with some possible replacements, but I recommend we stay with a modified version of CHEAKS. On reading my review you may not agree, so please don't assume it is a closed choice! I have given full references for you to follow up if you want to.

Let us continue to consider this area!

Best wishes

Bob

In reply to Prof. Robert McCormick

Re: Environmental awareness questionnaire

by Anouk Janssens-Bevernage -

Hi Bob,

After looking at the alternatives, I agree with your suggestion to stick with the CHEAKS questionnaire, provided it is modified. However, I would suggest we stick to the 'spirit' of CHEAKS and change more than just a couple of statements. What stops us from making sure the questionnaire is adapted to the environments we work in? I'd be inclined to make sure the whole questionnaire makes sense to the respondents. I know from my own experience that when I complete questionnaires and as little as one item doesn't make sense, I get distracted and might not be motivated to continue. In a school context, people might feel uncomfortable when reading statements they don't know anything about ("Am I supposed to know this? Is it wrong not to know about this? Perhaps the questionnaire isn't supposed to be completed by people like me?") What do you think?

Anouk

In reply to Anouk Janssens-Bevernage

Re: Environmental awareness questionnaire

by Prof. Robert McCormick -

Thanks for that comment Anouk, it is helpful to get another view on this. I am now looking at Climate change questionnaires as this may be of more current interest and it is certainly the focus of the Sierra Leone module on this area. I'll let you all know what I find in the next couple of days.

As to changing any questionnaire to make it fit a particular situation the issue is that it addresses only a couple of types of validity, namely 'face' and 'content' validity. The former is the kind you drew attention to above Anouk, and the other you also hint at where it asks about a content that does not represent what students have come across in lessons.

Two other features are relevant. First construct validity, which is a little more difficult to explain, but basically it means that the way students respond to it reflects what you expect. Thus in the CHEAKS questionnaire has several dimension underlying it: ‘verbal commitment’, ‘actual commitment’, and ‘affect’. This implies that the items that relate to this should come out in a factor (when a factor analysis is carried out) that represents these. The second feature is the reliability of items, which is a measure of items that relate to each other showing a correlation with the overall score for that group of items, and indeed the overall score of the questionnaire.

The nearer that you can stick to the original questionnaire the more you can be assured that it carries the robustness of the original one in terms of validity and reliability. However, in all these things it is a trade-off! There is no point in sticking to one type of validity only to do great damage to another kind.

Prof. Junaid has suggested changes to CHEAKS (though he made similar comments to you, as you know) and I attach the modified version (it isn't changed so much). Kiribati will perhaps need other changes. I think we can tolerate different versions in different countries, provided the changes are not too great, and still have a score that is compatible.

However, as I said, I'll try to see what Climate change offers. Do you think this more specific issue would help Kiribati? I know you said things like recycling is not relevant there.

See what you think.

Best wishes

Bob

In reply to Prof. Robert McCormick

Re: Environmental awareness questionnaire

by Anouk Janssens-Bevernage -

Thanks Bob!

A quick clarification: there are some recycling efforts on Tarawa (the main island) but the situation is quite different to other countries (as well as very challenging) and therefore some of the items in the questionnaire would not be relevant. Also, in terms of "commitment", we'd need to be careful with asking  questions about certain actions when students aren't in a position to do these. For example, they could commit to buying energy-saving light bulbs, but what use is this when they aren't available in shops? What do we do with questions about public transport in a country where there isn't any? My concern is that there are more than just a few questions that will create confusion. It's not that students wouldn't know about these, it just doesn't apply to their realities (beyond what they might learn about in class or hear about from outside sources). And that will not only affect validity.

I do like the way the questionnaire is set up though, so perhaps we can ask the Kiribati team to work on this without any limitation as to the number of items they can adjust/change? You can then decide if the changes are too profound to be useful for our purposes here?

Yes, I'm keen to explore what we could do around climate change too, as this is an issue that directly affects the population in Kiribati, and it is a crosscutting theme across the new curriculum.

Kind regards,

Anouk

 

In reply to Anouk Janssens-Bevernage

Re: Environmental awareness questionnaire

by Prof. Robert McCormick -

Hi Anouk

Yes that is a good idea, the Kiribati team should try to make it relevant, then see what it looks like and what the implications are for the robustness of the instrument. Prof. Junaid it would be useful for you to do the same with the FTC team, unless you think that the work you have already done on the version I circulated yesterday reflects local views.

All we have to bear in mind is the time we have before it needs to be used in the field and the work that needs to be done to make a final version for this use.

Still working on the climate change questionnaires (more difficult than I thought!).

Best wishes

Bob

In reply to Prof. Robert McCormick

Re: Environmental awareness questionnaire

by Prof. Muhammad Junaid -

Dear Prof McCormick,

As per my earlier mails, I have already asked Brainard and his team to review the EA questionnaire and revert to you. I don't know why they haven't responded up to now; may be they are still studying it. I hope Betty will take them up on that as time is of essence.

Best wishes,

Junaid.

In reply to Prof. Muhammad Junaid

Re: Environmental awareness questionnaire

by Betty Ogange -

Dear Prof. Junaid,

Many thanks for this. Yes, I saw the e-mail where you made this request.  I will make a telephone follow-up with Brainard on this and related matters in the course of Friday October 5.

Best regards,

Betty

In reply to Betty Ogange

Re: Environmental awareness questionnaire

by Prof. Muhammad Junaid -

Dear Betty,

Many thanks for this and for your follow up call with Brainard. 

Best regards,

Junaid.