[center]Saul Fisher
The Andrew W. Mellon Foundation
sf@mellon.org
——————————————————————————–[/center]
0、引言
尽管以教学为目的使用计算机已经有30多年的历史了，但是一个关于它在支持教学方面的价值和作用却是一个长期存在的争论。持久的关于如此广泛的教学媒体，例如粉笔或教科书的争论是令人难以想象的，另以方面， It is hard to imagine a comparable, sustained debate over other instructional media, such as chalkboards or textbooks. On the other hand, it is hard to think of another medium with such a sustained development cycle: despite the PC’s passage into early adulthood, only recently have we started to see high-tech instructional tools that have any common usage or popular acclaim. Chalkboards, by comparison, presumably were perfected-and so adopted-faster.
At the core of this Thirty Year War over the merits of instructional technology is this question: can we use computer technology to effectively enhance or replace traditional instruction-that is, to teach people? To answer this question, it is helpful to recall that traditional instruction is a face-to-face interaction where one person imparts or evinces some piece of knowledge or skill (or otherwise facilitates the learning of the same) to another person. Many who have wrestled with these issues have focused on the fact that traditional instruction in the face-to-face mode has many and varied flaws; a popular perspective among those promoting this last view is that the new instructional technologies represent an opportunity to make up for those flaws. I suggest that the pertinent claim against which the putative successes or failures of instructional technology should be gauged is rather that traditional instruction (that is, on the face-to-face model) somehow and occasionally manages to impart knowledge or evince or otherwise facilitate learning. This latter claim is a given for those who accept the possibility of successfully realizable constructivist learning models, and I suspect that this is most of us today, in some form and to some degree. But the lesson of constructivism’s successes in the classroom (as well as online) for measuring pedagogic effectiveness of instructional technologies-namely, that the teaching medium and pedagogic approach are wholly separable variables-has not been fully drawn out.
Indeed, the received view in the fields of educational psychology and instructional technology is either that the teaching medium or particular technology employed is ineluctably linked to a given pedagogic approach, or else that technologies and pedagogies are separable yet showing so indicates that only pedagogies merit studying for learning and teaching effect. The latter view is most closely associated with Richard Clark and the former, Robert Kozma. I suggest, against both Clark and Kozma, that in gauging the merits of instructional technologies, we can indeed distinguish between effects of pedagogies and technologies-and this distinction does not require that we write off the causal efficacy of teaching technologies.
1. Clark’s view of media and methods
Clark (1994) has argued that studies of instructional media [1] have sufficiently intertwined or ‘confounded’ instructional media and methods, so that in measuring differences in pedagogic outcomes between setups with and without technological interventions (such as the use of computers or networked data resources), it has been impossible to hold the methods constant and simply test the medium. There is no point to comparative studies of instructional media, then, if the intention is to determine the contribution of the medium to differential outcomes. Indeed, one argument Clark presents for this view consists in pointing to a wealth of data that he suggests shows that there is no significant difference in outcomes between such setups (that is, with variable technological enhancement), across multiple media. A further result of this large set of experimental results, Clark proposes, is that we ought to hold the media constant, so that we can find where, among such other factors as instructional methods, any significant differences lie-and we should direct our emmpirical research accordingly
Another one of Clark’s grounds for holding that media and pedagogic methods are confounded in media studies is his view that different media have non-defining, interchangeable attributes. From their interchangeability, Clark holds, it follows that we need not adopt any particular medium in order to realize a given method through the exploitation of one or another attribute. As a result, we should expect that any method of delivering instruction should be as good as any other. This expectation is borne out, he suggests, by the many studies indicating no significant difference between courses taught with and without technological enhancement, relative to pedagogic outcomes.
The primary consequence of confounding of media and method-according to Clark and his detractors alike-would be that such studies as were designed to show the effect of changing instructional media on learning outcomes naturally could not do so, given that outcomes might have changed by dint of methods yet this would go unnoticed in the flawed studies. Clark, for his part, takes the theoretical lesson here to be that there is nothing in instructional media per se that can cause pedagogic outcomes to rise or fall, and the practical lesson to be that studies that compare instructional media are most fruitfully focused on pragmatic considerations such as relative cost. The real test of change among modes of instruction, he proposes, consists in comparing pedagogic methods.
Interestingly, those educational psychologists, Human-Computer Interaction researchers, and instructional technology specialists who oppose Clark’s views generally don’t bother contesting what he takes to be the principal lessons of his primary claims concerning the confounding of media and methods. They take issue, instead, with those primary claims, maintaining that there are non-interchangeable attributes that individuate media by their special capacities, which vary in how well suited they are to particular teaching and learning tasks. Indeed, according to one view (Kozma, et. al.), in virtue of the differential suitability of media for achieving such tasks, media are thus individuated by their close fit with particular instructional methods. It follows that media and method aren’t confounded per se-they were linked to begin with, media serving as a proxy for method.
This particular debate has been pursued for over a decade, and the interlocutors have progressed not at all towards agreement-which is wholly unsurprising, given the fundamental nature of the disagreements over the fixity of attributes in media, and the desirability or necessity of distinguishing media and method in suitable experiments. And yet the debate continues, renewed in recent years by the introduction of instructional media of a far more sophisticated technological character than previously seen or employed in-or for that matter, outside-the classroom. So how to resolve the debate, particularly in light of widespread and growing use of computer-assisted and computer-mediated instruction? I argue that, whereas the individuating attribute view is incorrect (per Clark), broad functional classes may differentiate media. Contrary to Clark’s suggestion, media and method can be meaningfully distinguished in designing media experiments, which guarantees that we can measure the effect of media. The structure of my argument, and the rest of this paper, is as follows. First, given the nonessential nature of any individual media attributes, Clark is warranted in his doubts over their fixity. Yet nothing about the interchangeability of such attributes rules out special classes of media as determined by their (interchangeable) functions. One such special class-computational media-features functions reflective of capabilities not found in other, non-computational media. Indeed, these turn out to be capabilities of great interest to instructional technologists and HCI researchers, and they should be of great interest to anyone seeking to gauge the impact of computer-mediated instruction on learning outcomes. Second, the effects of different media on learning outcomes can be identified and distinguished from the effects of employing variable teaching methods: past studies indicating that changing media has no detectable effect have no bearing on the issue of whether media may have such effects as are detectable. Finally, the traditional opposition to Clark’s view has it that media and method are in any case inseparable, which is an untenable claim given (a) the nonessential nature of media attributes, and (b) the uncoupling of any given medium’s representational and procedural qualities from particular pedagogic methods.
2. The argument from the insignificance of media attributes.
I begin with Clark’s argument against the significance of media attributes. He argues thus: The view that choice of media can make a difference in learning outcomes depends on the claim that media have individuating qualities-known in the literature as "media attributes"-that make them better or worse for different teaching and learning tasks. But that claim is false, because there are no such qualities that characterize different media. Hence there are no grounds for thinking that media can affect outcomes.
What, precisely, are media attributes? Gavriel Salomon (1979), Richard Mayer (1997), Robert Kozma (1994), and others have suggested that media have particular attributes that correspond to specific elements of cognition with, accordingly, specific capacities to influence (enhance or diminish) learning. Such attributes-which can be broadly understood as the functional features of a medium that are designed to enable particular cognitive tasks-include zooming, constructing and taking apart represented elements, mediating sounds, and moving images. One view, promoted by Salomon, Mayer, and Kozma (among others) is that research on the effects of media is best focused on the effects of their attributes, given that they are causally efficacious relative to cognitive gains and that they can be consistently mapped, more or less isomorphically, to particular sorts of gains. Clark holds that while the claim regarding causal efficacy may be correct, the mapping claim is dubious-we cannot tie particular cognitive effects to particular media attributes. This is because different attributes can be realized in different media and in any case can give rise to the same cognitive effects. Hence, he reasons, it must be that something other than media attributes is responsible for differences in pedagogic outcomes.
A strong version of the media attributes view, posed by Kozma, rejects the view that attributes can be realized in different media. Rather, such attributes are essential and defining characteristics of media. Each medium is distinguished from every other in that
(a) each has a distinctive set of attributes-no two sets have identical membership (formally: for any two media M1, M2, M1 is not equal to M2 if
M1: {A | A1,…, An},
M2: {A’ | A’1,…, A’n},
and {A} is not equal to {A’}, and
(b) the attributes of a given medium Mi are always present in Mi.
Media attributes are, in short, fixed characteristics of a medium that define it by convention. If, for example, two computers did not allow us to interactively stream text data between its users, they would not be operating in a networked medium, given that the medium of a network as conventionally defined has the essential characteristic of allowing such streaming. What marks this view as ‘strong’ is the claim that such attributes are not merely characteristic of one or another medium-they are essential and definitive of particular media. Were this true, then Kozma would be warranted in his further suggestion as to the proper tasks for media studies:
i. identifying the causal mechanism whereby the attributes of a medium bring about cognitive or social processes, and
ii. saying how those same causal capabilities-isolated as variables (e.g. in education psychology experiments)-can influence learning [2].
What is the warrant for the strong view, though? Salomon follows theoretical discussions in Olson (1974) and McLuhan (1964) of symbol systems and their role in media-very 1960s!-in proposing an attribute ontology as a useful means of distinguishing media, and Meyer (1994) proposes that experimental evidence supports the utility of carving up the domain of media in this fashion. Successfully mapping attributes to particular teaching and learning tasks enables researchers to postulate augmented learning outcomes, or at least better performance on the constituent tasks, just in case the attributes in question are engineered to maximum enabling capacity. A better video replay or playback, it is thought, should make for better mental recall.
Clark takes a decidedly dim view of the strong media attribute view. There are no such attributes, he contends, if they are defined as particular to any given media. The claim that media attributes-of the sort Kozma and others describe-do not exist is not (or not clearly, at any rate) intended to suggest that there are no communicative features characteristic of different media. Rather, it is simply the claim that no such features are possibly found in one medium but never in another. Consider this contemporary example: students learn Spanish pronunciation through a variety of aural media, including live, face-to-face instruction, and analog and digital recordings (cd rom, web-based audio streaming, etc.). Whatever the medium, Clark would say here, there are in each instance the communicative features of, for example, playback, pause, and fast forward. These features are realized differently yet are functionally equivalent across media.
Given this functional equivalence, nothing distinguishes media relative to their general ability to transmit or otherwise communicate educational content. What distinguish them in this regard, Clark proposes, are incidental physical characteristics relevant only for the purposes of gauging maximally efficient delivery systems. The most appropriate role for media studies, accordingly, is studying the efficiency of different systems-in particular, Clark recommends that researchers pursue policy-oriented cost-efficiency studies.
The suggestion that the sole proper concern of media studies is efficiency raises some intriguing questions. For one, there could be other, non-efficiency concerns that stem from the particular physical characteristics of different media. These might include "user-friendliness", aesthetic appeal, degree of sensory stimulus, or even ethical propriety. Perhaps it is morally wrong, for example, to design educational websites that offer no aural interface for blind users (at any rate, under the ADA, it is probably not legal). Surely any such characteristics may run against efficiency concerns.
For another, perhaps capability is a matter of degrees, partly indexed to efficiency. If so, then in the limit, the maximal efficiency of one medium as compared with all others could also signal distinctive capabilities relative to delivering educational content. As a result, functional equivalence would fail to hold by dint of one or another differences in the way those functions are realized. As an example, consider voice-data transmission across networks. In the pre-Web era, it was perfectly possible to transmit text-data that encoded voice-data, though the lag-time for translating back and forth between text and sound might well have prohibited seriously entertaining the technology as a voice medium. In our brave new Web world, real-time transmission of voice-data across networks is a workable technology, owing to great efficiencies in digitally encoding sound. The function of transmitting voice-data has not changed yet the two scenarios suggest a shift from pre-Web networks to the Web where functional equivalence, in terms of practical capabilities, does not hold. Whether we want to count these as two truly distinct media, admittedly, is another question.
Now, using functional efficacy as the most fruitful way to define a set of attributes is of course a familiar strategy, from the philosophy of mind.[3] Functionalist accounts of the nature of mental states suggest that we understand such states (generally, as regards propositional attitudes, though possibly also with respect to ‘pure’ qualitative states) in terms of their causal efficacy, which is in turn defined in functional terms. A critical feature of such accounts is their neutrality on the issue of exactly how such states need be realized; indeed, a globally accepted tenet of functionalism is the suggestion that mental states can be multiply-realized. This tenet is intended to be an attractive feature of the view, allowing as it does various brands of physicalists and even dualists to agree on the functional role of mental states as definitive and crucial to any rich understanding of the way the mind works.
The famous and legion putative problems with functionalism in philosophy of mind include the suggestions that there are phenomena such as consciousness, or pain-states, which cannot be multiply realized, and that multiple realization leaves out intrinsic qualities (à la inverted spectrum arguments). The basic move of these sorts of anti-functionalists (leaving aside the very different concerns of eliminative materialists) is to suggest that, while some sorts of mental states, or some qualities thereof, might be multiply realized, there is always be one or another core sort of state, or quality of such states, which is inherently not capturable, under any possible alternate realization. The challenge for the functionalist, accordingly, is to deny or explain away the proposed cases of non-multiple realizability. Clark is in a similar position, though there is at least this difference. In the philosophy of mind, the debate revolves around in-principle equivalence between alternate realizations of mental states-even if, e.g. we could engage the population of China to interact in the "right" psycho-functional fashion (e.g. to perceive green), this will not compel anti-functionalists to accept that all true qualities of such mental states have been realized. Yet in judging the capacity of different media to deliver educational content in functionally equivalent fashion, practical considerations also must be introduced. First, there are typically practical considerations in differentiating among artifactual media: what the viewer experiences through or learns from a television screen and a computer monitor differ because of contingent differences in pixel count, screen refresh rates, screen size, and the like. While it is possible to imagine such differences disappearing, the extent to which they disappear is the extent to which artifactual media converge-and then we are talking about not merely functionally equivalent, but identical, media. Second, the delivery of instructional content has a pragmatically determined temporal cap-such content can grow stale, students’ interests may wander, lunchtime arrives, and so forth.
Putting such practical considerations aside, though, Clark’s broad media functionalism would fail if-as the strong view proponent typically suggests-there were individual media attributes, defined functionally, that were individually or jointly essential to and definitive of particular media. But his brand of functionalism would also fail if there were media the capabilities of which were special to particular classes of functions. In the former case, any single, or single group of, attributes might turn out to be characteristic of a given medium-whereas the latter case suggests that some broad classes of functions could be characteristic of general media types, including traditional and new sorts of media, with the particular nature of given individual media conceded to be causally undifferentiated by their attributes per se. If this picture is correct, then functionalism doesn’t fail on the theoretical grounds that there are individuating attributes, after all. It fails, rather, because it is not applicable-there are no like functions across the relevant groups of media. What are the particular groups of media here which differ so greatly in their classes of functions?
I posit classes of functions characteristic of some new media that may be radically distinct from those functions typical to traditional media, even if such new media also feature some of the same attributes of traditional media.[4] As an example, consider Web-based audio and video streaming, which features all the basic attributes of analog or even digital, non-Web audio and video media, yet bears other, special functions associated with Web technologies, including searchability and asynchronous transmission. Further, it is possible that such new, non-traditional classes of functions are attached to no particular attributes that differentiate in any significant way the media those functions characterize. Taking up the present example, Web-based streaming is characteristic of neither audio nor video in particular. So two media that have otherwise quite distinctive attributes-e.g. one cannot "zoom" sound-are much more similar relative to their Web-based qualities. Along the lines of this example, I propose that the most prominent candidate for a distinctive and non-traditional class of functions consists in those that are taken to characterize computational media.
At best, this is a sketchy proposal, because there is no canonical definition of "computational media". However, there are a few candidate definitions floating around, one of which suggests particular functions not characteristic of other media.
Naturally, nothing truly rests on the way we label the definition that successfully allows us to pick out the special class of functions not shared by other media-that’s the one I’ll call "computational media", but it could have some other suitable name, too. Nonetheless, it is helpful to distinguish this sense of the term from other uses that are prevalent, especially since proponents of the competing definitions have taken for granted that their candidates also enable us to distinguish a particular brand of media.
Thus, one relatively common way of talking about computational media is to define it in terms of an engineering discipline. For example, to slightly paraphrase the catalogue of one major research university, one view is that the discipline of computational media consists in the application of AI and HCI to the design of intelligent information navigation, task assistants, and learning environments.[5] It follows from this sort of view that computational media themselves are
(C1) whatever is designed as a result of research and scholarship in the computational media engineering discipline.
Accordingly, we define such media as certain sorts of computer tools designed to achieve particular tasks and with given foundational principles (e.g. borrowed from AI and HCI). A clear deficit of this definition is its narrow strictures on computational media as just whatever a particular group of engineers and allied researchers say such media are. But that opinion is quite likely to change over time-should our criterion of computational media change as well? Moreover, as is typical of one brand of badly-construed institutional theory, we define the criterion for the set of relevant entities by what are quite distinctly subsets of the range of institutions responsible for and involved in the creation of those entities. [6]
A more catholic definition, not tied to particular tasks of design principles used to guide realization of such tasks, is offered by Andrea DiSessa:
(C2) any representational formats based on computer resources that support human thought or expression [7]
This definition suggests that what makes the medium computational is that it utilizes a certain kind of resource, namely, computers or supra-computer structures. Thus, any medium that is, for example, mounted on or deployed over the internet should count. The same medium might have been deployed through the use of different resources-for example, digital video can travel through the medium of a network or through broadcast transmission. Yet just in case the medium can be realized through the use of computers, it counts as computational media. This definition has the merit of identifying computers as a possible resource or substructure for those media that admit of the right sorts of multiple physical manifestation. But this very point also makes the computational character of any media entirely inessential, and so (C2) fails to pick out any qualities of the medium itself that would constitute a distinct computational class.
A third alternative, more useful for identifying what makes computational media different from other media, suggests that such media consist in:
(C3) any representational formats which themselves have computational attributes.
This definition counts out much media that is computational on (C1) or (C2). It is not particularly discriminating to stipulate, per (C1), that those in the right discipline will recognize media as computational if they are so, and likewise for requiring, per (C2), that such media are fashioned and realized through the use of computers. After all, it is possible to network a VCR-an input/output device, though of clearly limited computational capacity-yet that should satisfy (C2).[8] And nothing on (C1) prevents those in a computational media engineering discipline from accepting networked gadgets of a non-computational character as computational media.[9] The idea of (C3), then, is to focus our attention on those media that themselves have functions (functional attributes) allowing the execution, representation, and communication of computational tasks.
There are any number of possible candidate functions, each of which characterize media that exceed the capacity of simple input-output systems. These include:
Data collection and synthesis / learning / knowledge-building
Data mining
Automation of tasks
Interaction of multiple agents, including automated agents [10]
Relation-mapping
What joins all these candidate functions is that they reflect the rule-governed behavior associated with programming. Programmability by itself is nothing special about any particular media-VCRs are supposedly programmable, and televisions may be wired to follow orders from, or give orders to, PCs. Rather, programmability for the types of functions described above marks media with them as wholly distinct from those media without them. Data synthesis, data mining-or anything else on that list-are simply tasks that cinema or radio, to take two examples, cannot muster. And if such other media could be retrofitted for the right sorts of programmability-think of digital television, as married to a database-it seems clear enough that one has transitioned to a different medium than the original.[11] Thus, on the test that non-computational media cannot be made computational, without changing media, it seems that we have landed upon a truly distinctive class of functions.
Let us assume, then, that this definition of computational media meaningfully demarcates an entire class of media, some of whose functions are not characteristic of any non-computational media. Then Clark cannot appeal to multiple realizability of functions across any media because there are special functions not realizable by any but computational media, whatever traditional media attributes are otherwise shared or functionally equivalent. The challenge, accordingly, is to conclusively demonstrate that there are, or are not, media that can be so classified.
I have not done this, but even lacking a canonical definition as would permit such demonstration, one point against Clark stands out nonetheless: it is possible in principle to devise functions for new media that do not correspond to any functions of previously existing media. Then it may be conceded that the particular attributes of media are insignificant to differential outcomes given their functional equivalence-yet this does not mean that different media cannot affect outcomes. The insignificance of particular attributes does not preclude the significance of special classes of functions characteristic of one sort of media and so not possibly multiply realizable in any other sort of media.
3. The argument from the ‘No Significant Difference’ literature: problems in isolating media effects
Clark’s other argument against our possibly capturing media effects suggests that, even if there were media attributes (for example, on some weaker conception), they would not be the sorts of things that make a difference in experimental outcomes. Clark offers the following line of reasoning:
(ME1) We would know that media effects are isolable if there were some evidence that they have been isolated.
(ME2) The best evidence of this would be that differences in outcomes have been detected.
(ME3) The evidence actually suggests no such differences.
(ME4) Hence there is no evidence that such effects have been isolated or are even isolable.
I suggest that this argument is flawed, for three reasons:
First, the burden of proof here consists in Clark showing that ME3 holds, and his reasoning for this claim draws on a faulty, if common, interpretation of the available empirical data. Clark puts his defense of ME3 in terms of a challenge to find significant differences (SD) in outcomes given different media-and then marshals the contrary evidence of Thomas Russell’s literature review (1999) and meta-analyses in the same vein (for example, as reviewed in Kulik (1994)). But whereas Clark surely wants to argue that it’s not the case that any studies show SD; hence there is no evidence of isolable media effects.he actually offers this instead:
some studies show no significant difference (NSD); hence there is no evidence of isolable media effects.
If we accept Russell’s claim at face value, then a great plurality of comparative media studies selected from over seven decades of research suggests a pattern of roughly equivalent (learning) outcomes across different media (Russell collects a total of 376 studies, of which 355 indicate NSD, and 21 indicate SD). This is perhaps impressive but not quite compelling. A great plurality of studies pointing towards the same claim fails, of course, to establish the claim definitively. Is it an unfair standard to demand definitive establishment of claims in the social sciences or education studies? To be sure, generally speaking-yet we hold the bar higher when trying to establish the NSD claim because of Clark’s proposed use of that claim. He suggests that one piece of evidence for his view that the choice of media can never make a difference to learning outcomes is that, overall, it has not made much difference to date. But lacking completely uniform results or a total data set-or even an idea as to what such a set would look like-it is hard to see why the existing set of comparative media studies is to be taken as evidence for the in-principle impossibility of media choice affecting outcomes.
Second, Russell’s NSD literature review and related meta-analysis studies (Kulik and others) are methodologically flawed, and so fail to support Clark’s claim that significant differences should be outside of media-and rather sought elsewhere, for example, among instructional methods employed. The principal (and well-advertised) flaw here in Russell’s report is that he makes no effort at all to distinguish among the types of media comparison studies that he includes. As a simple literature review, one need not hold up the standards of meta-analysis here. Yet this best known of literature reviews in the field does not even bother to assess commonalities and differences among its constituent studies, or attempt to explain why SD shows up in some cases.
That is all suitable for a simple collection of media comparison studies, though as such Russell’s compilation can carry no weight in interpreting or understanding more broadly the results of the collected studies. Are there more careful studies in this vein? Indeed there are. Most prominently and promisingly, Kulik (1994) focuses on computer-assisted instruction and presents measures of effect sizes across well over one hundred and twenty studies as judged in a dozen meta-analyses done over thirteen years (1978-1991).[12] He helpfully assesses these meta-analyses and their constituent studies by various grains of detail, and acknowledges that computer-assisted instruction is so overly diverse in nature that it is difficult to make too great generalizations from meta-analytic summaries of the available data across studies. Yet Kulik’s assessment-"Meta-analysts have demonstrated repeatedly that programs of computer-based instruction usually have positive effects on student learning" (p 26)-remains a bit too bright. It is hard to say whether the meta-analyses to which he refers demonstrate anything, much less repeatedly, given tremendous variation across, and under-defined parameters within, the many studies they characterize. Such variation and under-definition plagues the constituent studies, relative to precise type of media (except in the case of the numerous Stanford-CCC studies [13]), curricular and pedagogic design, experimental design, and conception of ‘learning outcome’. To properly code these variables-so as to judge the inclusion of the studies in viable meta-analyses and ensure maximal homogeneity-looks to be a task with low profitability. The failure to take such variegation and under-definition into account renders the grander, meta-analytic conclusions ill founded. [14] There is every good reason to pursue more rigorous meta-analyses, though that will surely require better and more uniform studies to begin with.[15] One piece of cheering news here is that the Andrew W. Mellon Foundation is supporting such careful research at a number of universities and colleges on the pedagogic outcomes and costs of using technology in teaching. In the meantime, however, from formal and informal literature reviews alike, there is to date no viable statistical evidence for (or, for that matter, against) ME3.
Third, in accepting that existing studies show NSD (and relying on said claims to establish ME4), Clark assumes that all the constituent studies have successfully isolated the media effect, after all, and have so demonstrated NSD across media. But if ME4 were correct, then Clark should not accept the NSD constituent studies as having valid experimental designs. Assume otherwise, that is, that ME4 is correct and the studies included in Russell’s survey have valid designs. Then media effects would be both non-isolable and isolable. Hence Clark’s argument is incoherent.
Finally, assume that Clark is right to suggest that we can hold the media constant and gauge the relative strengths of different instructional methods. By parity of reasoning, and barring any argument to the contrary, we should also be able to hold the methods constant too, which should give us some notion of whether pedagogic outcomes differ on the basis of media alone. Here the response has typically been-from Clark and many others-if you haven’t changed the method when you changed the media, then you have missed the point of changing the media. Even if this were so, this would be a normative, not a descriptive, claim. It is in any case a non-sequitur, for if one concedes that the methods can be held constant, then changing the media won’t change the methods. To defeat this point, Clark would need to suggest why it is that we cannot hold the methods constant. That is not a likely claim for him to make, though, for it is a close corollary of a strong media attribute view-a cornerstone of the view held by his opponent in the media comparison debates, Bob Kozma (I briefly look at that view at the end of my paper).
Given the failings of Clark’s central arguments, there is no apparent reason to think that we cannot isolate media effects experimentally, and that such effects as we may identify might not reflect differences among media relative to their causally efficacious features. This in turn suggests that there is one less reason to think we cannot gauge pedagogic effectiveness across instructional technologies, where such judgment consists in comparing effects among the various media those technologies represent. In particular, measures across computational and non-computational media appear to warrant careful attention. Yet other comparisons should prove fruitful, too, if there are no principled obstacles to comparing their performance. This result corresponds to an old intuition about instructional technologies and media, predating the use of the Web or even computers altogether. Just within the bounds of face-to-face instruction, we might use blackboards, overhead projections, or filmstrips-and it would likely have been astonishing were some media not better for presenting a given set of materials than were other media. Such is expectable given differences in the adaptability of learning material or curricula to audio or visual or tactile learning-or other sensation, learning, or comprehension modalities altogether. But if that is so, then by holding the curricula and teaching methods constant, we should have expected that the success rate of any given medium in furthering pedagogic outcomes might vary, perhaps greatly. Mutatis mutandis, the same is expectable in comparing media employing new technologies and of a computational character.
4. Separating media and instructional method
In closing, I return to the received alternative to Clark’s view, posed by Kozma (1994a, b), who suggests that instructional media and methods are inseparable and therefore can only be assessed as one. This inseparability follows from his strong version of a media attributes view, just in case the putative fixity of such attributes entails that they are suited to particular instructional methods. But what about that fixity entails the latter claim? Kozma suggests that representational and procedural qualities are what make media an intrinsic part of instructional design, and so wed media to methods. Given that the media attributes view ranges in particular over the capacities of media to represent and perform operations on mediated objects (voice, data, pictures, etc.), the unchanging identification of given media with particular attributes indeed suggests that those media should be well-suited to particular teaching methods that are strongly reliant on the capacities afforded by those attributes.
As an example of this relationship, Kozma invites us to consider interactive operations of media (that is, where media can react to inputs). Such capacities are part of a "…complementary process within which representations are constructed and procedures performed, sometimes by the learner and sometimes by the medium…." (Kozma, 1991, 1994, 11). As a consequence, Kozma proposes, new, interactive media are best able to take advantage of new, constructivist teaching methods. Yet even granting (as is commonly held in the HCI and instructional technology communities) that there is a good fit between interactive media and constructivist teaching and learning theory, Kozma gives us no reason to think that such media should be best-suited to such methods. That a given medium is any more than contingently optimal for any particular method would require much more knowledge about either media or methods than we are likely to come by.
In any case, one lesson from our assessment of Clark’s argument from the insignificance of media attributes is that they are in all probability not endowed with the fixity that Kozma and others conceive of. Without the strong media attributes view, though, we arrive at Kozma’s view that particular teaching methods cannot be attached to particular media in virtue of their making maximal use of capacities of the media which are definitive of the same media. The upshot is that media and methods can be distinguished, after all. But this should be as unsurprising for Kozma as it should be for Clark, for we could not even gauge what it would take to aassess the medium-as attached to a particular method-if we could not separate it out from the instructional design.
Computing and Philosophy (CAP) Conference, August 2000 Carnegie Mellon University, Pittsburgh
References
Clark, Richard E. (1983). Reconsidering Research on Learning from Media. Review of Educational Research, 53 (4), 445-459.
Clark, Richard E. (1994). "Media Will Never Influence Learning." Educational Technology Research And Development, 42 (2) 21-29.
Clark, Richard E. (1994). "Media and Method." Educational Technology Research & Development, 42 (3), 7-10.
Dillon, Andrew, and Ralph Gabbard. (1998). "Hypermedia as an Educational Technology: A Review of the Quantitative Research Literature on Learner Comprehension, Control, and Style." Review of Educational Research, 68 (3) 322-349.
DiSessa, Andrea. (2000). Changing Minds. Cambridge, MA: MIT Press.
Hannafin, Michael J., Kathleen M. Hannafin, Simon R. Hooper, Lloyd P. Rieber, and Asit S. Kini. (1996). "Research on and Research with Emerging Technologies", 378-402, Handbook of Research for Educational Communications and Technology, David H. Jonassen (ed.), New York: Simon & Schuster Macmillan.
Kozma, Robert B. (1987). "The Implications of Cognitive Psychology for Computer-Based Learning Tools", Educational Technology, 27 (11).
Kozma, Robert B. (1991). "Learning with Media", Review of Educational Research, 61 (2), 179-211.
Kozma, Robert B. (1994a). "Will Media Influence Learning? Reframing the Debate." Educational Technology Research and Development, 42 (2) 7-19.
Kozma, Robert B. (1994b). "A Reply: Media and Methods." Educational Technology Research and Development, 42 (3), 11-13.
Kulik, James A. (1994). "Meta-Analytic Studies of Findings on Computer-Based Instruction", 9-34, in Eva L. Baker and Harold F. O’Neil, Technology Assessment in Education and Training, Hillsdale, NJ: Lawrence Erlbaum Associates.
Lechner, Ulrike, and Beat F. Schmid. (1999). "Logic for Media – The Computational Media Metaphor", in Sprague, E.: Proceedings of the 32nd International Conference on System Sciences (HICSS 1999), Hawaii. (see also http://dlib.computer.org/conferen/hicss/0001/pdf/00015016.pdf.)
Mayer, Richard E. (1997). "Multimedia Learning: Are We Asking the Right Questions?" Educational Psychologist, 32 (1), 1-19.
McLuhan, Marshall. (1964) Understanding Media, the Extensions of Man. Cambridge, MA: MIT Press.
Olson, D.R. ed. (1974). Media and Symbols: The Forms of Expression, Communication, and Education. Chicago: National Society for the Study of Education.
Russell, Thomas L. (1999). The No Significant Difference Phenomenon. Raleigh, North Carolina: North Carolina State University (see also website at http://teleeducation.nb.ca/nosignificantdifference).
Salomon, Gavriel. (1979). Interaction of Media, Cognition, and Learning. San Francisco: Jossey Bass.
Soloway, Elliot and Amanda Pryor. (1996). "Using Computational Media to Facilitate Learning", Communications of the ACM, 39 (8)
Spencer, Ken. (1991). "Modes, Media, and Methods: the Search for Educational Effectiveness." British Journal of Educational Technology, 22 (1), 12-22.
Notes
[1] Some psychologists writing on instructional technology have distinguished between media and technology as follows: media are vehicles for communication, relative to delivery and representation of information, and interaction among users; whereas technologies are the means by which media are realized-physically or virtually (cf. e.g. Kozma (1994, 11)).
[2] Once we distinguish these tasks, Kozma suggests, we can see that Clark errs in taking attributes as non-necessary features of particular media. Clark wrongly runs (i,ii) together, according to Kozma, and so faults attributes for being contingent where one should never expect to find necessity-in the world of experimental variables. The causal mechanisms that such variables may occasionally represent can be essential elements of a medium though the variables, by definition, ought not share the quality of being essential. This argument is something of a red herring: Clark is not concerned with the possibility of any experimental variables being necessary-he is doubtful that the features such variables represent could be considered necessary to an adequate definition of the medium with which they are associated.
[3] Similarly, a de facto functionalism has guided microbiological thinking since Mendel offered his laws of genetics. The gene is a functional unit, physically composed of sequences of DNA but not corresponding to a consistent number, ordering, or structure of such units as those sequences may comprise. The multiple ways in which different genes are physically realized (all of them physical, to be sure, and all within a fairly narrow set of constraints) suggests another sort of philosophical lesson. For philosophers of biology such as Alexander Rosenberg, the critical role of genetics in biology and the centrality of the gene concept in genetics is constitutive of the very autonomy of biology from chemistry and physics. Biology cannot be reduced to the latter two sciences given that nothing in chemistry or physics equips us to talk of biological functionality.
[4] It may be suggested that my proposed distinction between classes of functions and attributes founders on the functional definition of attributes. My answer here is that the special classes of functions are defined by groups of functions with distinctive characteristics. Single attributes, by contrast, are generally defined relative to single functions.
[5] This is from the Computer Science Department at Northwestern University (the instructor is Christopher Riesbeck)-cf. http://www.cs.nwu.edu/academics/courses/c25/admin/intro.html; for a similar notion of computational media, aimed at the design of tools for "efficient communication and problem-solving artifacts", see the Mississippi State University School of Architecture website-http://www.sarc.msstate.edu/programs/grad/grad-curriculum.html.
[6] This is a familiar problem for architects who are told that their profession is simply civil engineering.
[7] Personal communication, July 5, 2000; DiSessa further suggested that computational media should be "relatively widely used", though what work this does in the definition is not clear. DiSessa, by the way, concedes that the alternate definition (C2) below is closer to the concept he develops in his Changing Minds (2000).
[8] So might a toaster, though its general mediating qualities may be questionable.
[9] Indeed, as the discipline borrows on AI research, it should be tempting to consider some devices with no computational features themselves as computational media just in case they serve as real-world physical extensions-physical media-of AI agents.
[10] This is a view of computational media proposed by Lechner and Schmid (1999).
[11] Of course, making something programmable is not necessarily making it a computational medium…one thinks of the MIT Media Lab’s programmable Lego Brick, which is not a medium at all.
[12] Other careful reviews of the literature include Spencer (1991) and Dillon and Gabbard (1998).
[13] Use of the computer-assisted instruction materials developed by Pat Suppes and R. C. Atkinson at Stanford was the subject of some two dozen controlled studies, which were incorporated in several of the meta-analyses Kulik examines.
[14] It might be contended that this is an instance of the "Apples and Oranges" argument offered by Eysenck against meta-analytic techniques during the late 1970s and early 1980s (Gene Glass, it will be recalled, responded that the wholly legitimate aim of meta-analysis by this token was to broadly characterize "fruit"!). Yet the variation and under-definition of significant parameters in these studies creates a conceptually prior problem, namely, that we lack a solid basis for judging whether the studies have enough in common-for example, by the standards of a rigorous homogeneity test-to warrant being assessed together in a meta-analysis.
[15] See Saul Fisher and Thomas I. Nygren, "Experiments in the Cost-Effective Uses of Technology in Teaching: Lessons from the Mellon Program So Far" (http://L2L.org/iclt/2000/papers/129a.pdf)..
——————————————————————————–
ITFORUM PAPER #65 – Medium, Method, and Message: Why we can measure the pedagogic effectiveness of instructional technology by Saul Fisher . Posted on ITFORUM on November 14, 2002. The author retains all copyrights of this work. Used on ITFORUM by permission of the author. Visit the ITFORUM WWW Home Page at http://it.coe.uga.edu/itforum/ http://it.coe.uga.edu/itforum/