Methodology
Outline and Justification of Research Methodology:
Work Packages WP1, WP2 and WP3


1 Overview of methodology

Problems to be addressed

The Information Society is a site of rapid and fundamental change. The rapidity is extreme, with changes and developments on time-scales of less than a year. And the changes can be sufficiently fundamental to be deemed qualitative, for example placing at the disposal of every citizen world-wide communications, or providing them with tools of analysis and communication previously restricted to small technical elites.

Whilst the hardware and software for information processing and exchange can be readily purchased and are quickly becoming more and more widely distributed, the use of tools for analysing and representing information requires a rather advanced intellectual level together with high order critical skills.

Teachers are obliged to adapt to this new and continually changing situation. They need positive assistance in doing so. In the absence of positive intervention, education systems are generally rather slow in their response to change. Over a decade, what one can see going on in (for example) a secondary school science lesson may alter rather little. Once trained and with a few years' experience, teachers have acquired a 'stock' or 'repertoire' of knowledge, skills and strategies which they continue to draw upon for many years. If these are not well-adapted to the demands of a new situation, or if no new repertoire is provided, the result may be avoidance of the new situation, or transforming it into something more familiar.

Novel technologies can change not just physical but also intellectual landscapes. In particular, information technologies have produced (and will continue to produce) new forms of representing and dealing with information. These new forms do not just make learning or thinking 'easier' or faster; they may change their nature. One example is the use of visualisation to deal with and think visually about abstract structures or complex information, which is very rapidly increasing in use in scientific and technological communities. Another example of visualisation, together now with direct manipulation is educational software such as Cabri-Géomètre, in which the nature of geometrical proof changes for the student, becoming a matter of hypothesis forming and testing. Yet another is the way data representations which change in real time enhance understanding both of the process represented and the nature of the representation. A different kind of example is the possibility of hypertext structuring of information, which profoundly modifies 'linear' reading and learning. A final example is the increased scale on which mathematical operations can be 'chunked' for thought. For instance, with a spread-sheet the user can rather easily try the effect of altering various input parameters, and can see the effects directly, viewing the calculation as an integrated whole.

We adopt a transformative view of the nature of communication and change. Communications are not simply 'received' but are re-made, re-constituted, transformed by the receiver. Communication has to be seen as action; as minds acting on other minds which then act in response. Those acts of response are necessarily transformative, making new meanings relate to previous ones. The attentive reader is doing this at this very moment - trying to construct a meaning for this very paragraph which coheres with his or her own ways of thinking. That is one reason why examples are so important in communication.


Goals of the research

Three distinct but related areas of research have been identified for STTIS:

WP1 the nature of the use made by science teachers of informatic tools

WP2 difficulties in teaching and learning graphic representations

WP3 transformations when 'adopting' innovative teaching strategies

In each area we ask parallel or related questions, as follows:

For WP1:
What are some of the problems and opportunities for the use of (selected) informatic tools in science classrooms?

How do teachers transform expected uses of such tools - and what can be conjectured about difficulties and opportunities for using such tools in the classroom? What transforming mechanisms can one conjecture to be involved?

For WP2:
What are some of the problems and opportunities for the use of graphic representations in science classrooms, able to be anticipated on grounds of prior evidence and/or theory? Are these confirmed by students' readings of such representations?

How do teachers' understand the anticipated problems? How do they deal with such problems, in the context of innovative teaching involving essential use of images and graphic representations? What transforming mechanisms can one conjecture to be involved?

For WP3:
What specific requirements are expected of teachers in some selected well-defined curriculum innovations in science?

How do teachers understand these expectations? How do they act on these understandings in the classroom? What transforming mechanisms can one conjecture to be involved?


Nature of the research

The research methodology adopted is qualitative in nature, because we believe this to be the right way to address the essential problems in this case. As indicated below, quantitative methods might become appropriate at a later stage, after the qualitative diagnosis and description of the problems has been done. It is this first stage of diagnosis and description which the research in STTIS addresses.

The research is fundamentally applied in character. It is directed to informing and so to potentially improving practical decision-making and action. By 'informing' decision-making and action we do not (and could not) mean providing a base of data and theory from which decisions and action could be in some way completely derived. Such decisions and action in real time always have to go beyond available evidence. Instead, we mean providing data and interpretations which suggest priorities for action, indicate crucial factors which should be taken into account, and identify difficulties which action needs to overcome.

There are always two kinds of information relevant to practical action. They concern the two different questions of existence and of prevalence. For example, in the field of public health, it may be necessary to know both the nature of a problem (e.g. what is the disease?) and how extensive it is (how many have the disease?). It is important to note that the first question (existence) is logically prior to the second (prevalence). We cannot even enquire how widespread a problem is until we have diagnosed its nature.

To answer the second kind of question adequately requires large and carefully chosen samples. To provide useful answers to the first kind of question does not require large samples; instead it requires carefully cross-checked in-depth investigation. To use the previous analogy, one may only need a few cases to characterise a new disease, but one has to conduct extensive tests and investigations on those cases. One may also compare the way political parties use small 'focus-groups' of people to discover crucial aspects of the way policies are perceived.

Answers to the first kind of question (nature and existence) guide decisions in suggesting what should be done - the nature of the required action. Answers to the second kind of question (extent, prevalence) guide decisions as to the scale of the required action.

Broadly, the three researches (WP1, WP2, WP3) address the question of the nature and origin of difficulties, and only secondarily concern their prevalence. However, the design is such as to increase the likelihood that the problems which are found and characterised (diagnosed) will prove to be of general importance and significance; that is, that they are not trivial in nature and are not highly localised and specific, relevant only to a few contexts. In the given context (Science Teacher Training in the Information Society) this can be achieved by:

parallel investigations in different European countries

focusing on teachers who are willing to innovate.

The first - comparison of different European contexts - clearly helps to distinguish difficulties which are more, or are less, strongly related to local cultures and circumstances. The second is subtler. Much experience, in many countries, shows that although some difficulties for teachers in responding to innovations depend on (for example) inexperience, unwillingness or unfamiliarity, and so are not fundamental and are often transient, other more important difficulties are fundamental. That they are fundamental is indicated by the way they are present after 'transient' phenomena have died away. If willing and reasonably experienced teachers resist, or in transforming act to subvert, an innovation, we have reason to think that the problem is deep-rooted and may well appear in different circumstances. Thus it is a feature of the methodology to avoid studying transient effects, and to concentrate on those likely to appear when teachers are seriously trying to implement an innovation.

The methodology also has to take into account that education is a form of planned, purposive, communal action mediated by agents (teachers). Planners or legislators in education are obliged to act through teachers. Teachers have their own purposes and goals which may conflict with those planned. Teachers have their own understandings which may transform the understanding of the intended action. Teachers act under specific constraints (e.g. time, resources) which planned actions may not have taken into account. Thus it is necessary that the methodology provides for the investigation of plans and intentions, and of different understandings of - of different meanings given to - these plans and intentions. Further, the methodology, besides taking into account the intentions of teachers and other agents, has also to take into account the influence of the researchers themselves. It can never enough to study teaching in action alone.

In using informatic tools designed by others (for example, Computer Based Laboratory, or Interactive Physics) we need to distinguish the innovative intentions of the particular uses of the tool which are investigated, and the general intentions of the tool designer, which are often embedded in the tool's functionalities and characteristics. To do so is not simple, because innovative proposals are often very much based on what can be done educationally with the tool as designed. However, the possibility of such a 'second order' difference in intentions needs to be borne in mind.


Cross-checking - 'triangulating'

Such investigations place a heavy demand on cross-checking - on 'triangulating' - data and interpretations of these data, from a variety of sources. What people say they do is not always what they can be seen to do. The reasons they give for actions are not always consistent with what the situation appears to demand. Interpretations of evidence necessarily owe something to the interests of the interpreter. Actions are not always consistent or coherent.

For this reason, we intend to rely on multiple data sources. Interviews with teachers give some data about purposes. Interviews before and after teaching can cross-check stated purposes against reasons for observed actions which may seem to have different purposes. Class observation can notice both how purposes are translated into action, and actions which appear to serve unstated (even opposed) purposes. Analysis of tasks teachers give students to do (e.g. questions they set) can give evidence of teachers' conceptions of desirable outcomes, and give concrete content to aims otherwise only stated in general (perhaps not very clear) terms.

It is essential to provide for, but also to control, variety in the specific contents and contexts of teaching which are studied. A type of disjunction which appears between (say) a teacher's own stated aims and that teacher's actual practice, in the case of more than one type of informatic tool, is more likely to be general and less likely to be a function of that specific tool. A kind of conflict which appears between the intentions of planners and the intentions of teachers, in the case of more than one area of science, gives grounds for expecting it to be found elsewhere.

We require, therefore - for example - controlled variety in the informatic tools investigated. Those chosen are:

tools for simulation and modelling
tools for acquiring and representing data

In the case of graphic representations, defining the nature and scope of this controlled variety is itself problematic. In advance of the research identifying difficulties in learning and using such representations, it is necessary to use some a-priori scheme for identifying problems of representation. It becomes part of the research to check whether this a-priori scheme appears to be well-founded. Thus triangulation helps to test the relevance of the evidence and analysis on which the data collection is based.

There is also the matter of cross-checking or triangulating the terms and concepts used in analysing data and forming hypotheses or interpretations. We have (of course) already found in comparing previous researches amongst members of the STTIS project, that questions of this kind immediately arise. Are results obtained by one group and described in one way essentially 'the same phenomenon' as those produced by another but described differently? Or, despite similarities, are they actually different in some important way? If different, is this a difference in outcome due to different national contexts, or is it a difference in the questions asked, in the problem addressed? We rely on triangulation, within and between national data, to provide ways to decide such questions at least tentatively.

Thus the methodology will provide for cross-comparison not only of 'results' but also of methods used and conceptualisations formed to 'produce' those results. Data from one country needs to be re-inspected using the interpretative schemes used by another country, for example. It must also provide for cross-comparison of methods of investigation, which may either detect robust phenomena which appear despite differences in method, or may detect phenomena which appear fundamental but are actually likely to be an artefact of method.


Opportunity sampling

Since the research is to be grounded in observation of actual classroom work and analysis of existing innovations and plans, it follows trivially that these have to exist in the different countries involved.

This produces immediately two difficulties which can at best be only partially resolved.

The first difficulty is that not every required kind of innovation exists in every country. For example, WP3 calls for planned, documented innovations, a model for which is the planned introduction of compulsory work in optics on a national scale in France. There is of course no chance that the same thing will have occurred in another country. It may be that the nearest thing in common is a local innovation done on a voluntary basis in a quite different topic - for example introducing ideas about teaching qualitative thermodynamics in the UK. This could hardly be called 'controlled variation', but we are obliged to accept it or do nothing. Similarly, in France we can study the teaching of geometrical optics using an informatic tool, but geometrical optics is not always compulsory in the curriculum in several other countries, and is absent in some. Equally, however, we can and will look for the maximum coincidence in curriculum content and kinds of innovation as between countries, even if in some they are 'official' and in some not.

A further constraint, with the same consequences, arises from the examples of innovation actually available to researchers in the various partner countries. We are obliged to select those cases for which an opportunity presents itself or can be created.

The second difficulty, arising from the first, is that variations between national cultures and between content, kind and context of innovations, are inevitably confounded (mixed). This is of course a feature of reality in trans-European action of every kind. National cultures, structures and practices themselves help to determine the nature of innovations which are practicable or are thought desirable. Thus innovation can not be 'held constant' whilst national culture and context vary. It follows that analysis must treat this as a real phenomenon rather than as a methodological flaw. Results purporting to be about a kind of innovation need to be set in their national context, hypotheses about the effects of that context being made explicit. The corresponding advantage of cross-national research is that features of the local context, often rather 'invisible' to the local eye which takes them for granted, are thrown into relief when that context has to be understood from a different national context. (The fact that a UK teacher can legally not be told how to teach a given topic, may assume greater significance when set against a country in which this is expected and accepted.)


Variations in research designs

At first sight it appears obvious that research designs should be the same in different countries conducting parallel investigations. In practice this will not always be appropriate or possible. A simple example is that in some countries it is necessary to begin an investigation with identifying a range of difficulties students have in a particular context, whereas in others such work has already been done and can be by-passed.

A more fundamental difficulty is that in some cases the methodological strategy needs to be varied for reasons of principle. For example, there is a real difference between how one has to investigate representations which are familiar to students and ones which are novel. In the first case a questionnaire can elicit difficulties encountered in several examples, but in the second case the novel representation itself has first to be established, which generally cannot be done through a simple questionnaire.

They key criterion to adopt, is that methodologies should vary only for principled reasons, and that different methodologies adopted do address the same overall questions. Thus in the above case, in order that an investigation of a novel representation addresses the same questions as an investigation of a familiar one, it is necessary to vary the procedure so as to make the two cases more, not less, comparable. In this way, we are confident that common results will be meaningful.


Towards a common language of analysis

It is likely that cross-nationally phenomena will be seen which if described in common terms seem similar, but if described in different terms seem different. Thus the question of how to describe results (terms in which results are reported) will be a fundamental and essential part of reaching a trans-national report.

This will have to be reached by putting separate national reports together and comparing them. But before the national reports are finalised it will be essential to discuss and compare initial forms of analysis, so that unnecessary differences in terms of description are minimised. To do that will require access not only to draft parts of reports but also to some representative data, so that the discussion is not purely about words but is also about words in relation to evidence.

In doing analysis of such data one necessarily creates analytical concepts/descriptions (e.g. "serial causal reasoning"). Email can be used to circulate ideas for such concepts as they begin to emerge, so that others may try looking at their data in that light.

The aim is not to force reports into a common language, but to agree about which of the phenomena we think we see are probably related, and how. It is likely that the level at which there is a real commonality is quite deep, beyond the point where the local context is important. Thus it may go beyond the surface of actual evidence available, and have to be hypothetical rather than factual.

< go back >