Stanford

LINGUISTICS DEPARTMENT - STANFORD UNIVERSITY

An Invitation to CALL

Foundations of Computer-Assisted Language Learning

Home | Unit 1 | Unit 2 | Unit 3| Unit 4| Unit 5| Unit 6 | Unit 7 | Unit 8 | Supplement


Click here for PDF

An Invitation to CALL

Unit 6: CALL Theory and Research

Arguably, any applied field is defined by three dimensions: theory, research, and practice. The bulk of this course focuses on practice, but in this unit we will also briefly consider the role of theory and examine some of the types of research that have been conducted, along with a few example results.

The TESOL Technology Standard most closely allied to this unit is Goal 2, Standard 4: "Language teachers use relevant research findings to inform the planning of language learning tasks and activities that involve technology." While it is natural to extend face-to-face classroom research to the CALL domain, much more relevant sources can be found within the field itself. Scholars have been studying CALL technology, materials, tasks, environments, and interactions for more than 30 years, yet many teachers who use technology and even present their experiences with it at major conferences seem to know little or nothing of this literature. The purpose of this unit is to offer an introduction to CALL theory and research to provide you with a foundation and some resources for further exploration on your own.

THEORY    

What is CALL theory? Egbert & Hanson Smith (2007) claim that "CALL theory" is unneeded: “... educators do not need a discrete theory of CALL to understand the role of technology in the classroom; a clear theory of SLA and its implications for the learning environment serves this goal” (2007, p. 3). The key term for me there is "the learning environment". I argue that what the computer brings to the learning environment as a mediating actor is significant enough that any attempt to characterize such an environment without a deep understanding of how technology impacts that environment--and the learning process--will be incomplete. I have defined CALL theory elsewhere as follows.

Collectively, CALL theory is the set of perspectives, models, frameworks, and specific theories that offer generalizations to account for phenomena related to the use of computers and the pursuit of language learning objectives, to ground relevant research agendas, and to inform effective CALL design and practice.... a CALL theory is a set of claims about the meaningful elements and processes within some domain of CALL, their interrelationships, and the impact that they have on language learning development and outcomes (Hubbard, 2009: 3).

There is an interesting gap in the area of theory for CALL. Unlike the case of second language acquisition in general, CALL does not have a dedicated theory yet and based on current trends it is unclear whether it will ever have a comprehensive one. Instead, CALL theory comprises the "set" mentioned above, a set drawn from a number of sources including SLA theories, general learning theories, linguistic theories, and human-computer interaction theories. Hubbard (2009) proposes a typology for the relationship of theory to CALL research and practice. Much of CALL in the past has been atheoretical, produced without reference to any specific theory or framework, although that is changing. When theory is referenced, by far the most common approach is simple theory borrowing, where a theory from some other domain such as SLA or general education is used without any changes. A more interesting form is theory adaptation. In this case, a theory is modified or enhanced to accommodate the differences inherent in the computer versus the face-to-face environment. A relatively rare occurrence is theory synthesis, where two or more theories from different sources are combined to accommodate the special qualities of the computer-mediated language learning environment. CALL has witnessed only a few examples of theory creation, following the description for "a CALL theory" above. Examples of each of these may be found in Hubbard (2009). Two other categories are introduced in Hubbard & Levy (forthcoming). One is theory instantiation, where a theory like Activity Theory that explicitly has a place for both the technology and language learning is incorporated in a study. The second is theory ensembles, where perspectives from two or more theories are combined without synthesizing them into a unitary entity.

There is a great deal of variety in theoretical underpinnings to CALL. As an example, I researched all the instances of the word theory  in 25 years of CALICO journal articles (Hubbard, 2008). Across those several hundred CALICO articles (3-4 issues per year), there were 113 distinct theory references. Only 17 of these were mentioned in three or more articles, and there were no dominant ones. Of specific theories (as opposed to general types like "learning theory", which was mentioned in 20 articles), the two leaders were schema theory and item-response theory, the latter only applying to designing assessments.

In terms of general theoretical approaches these days, the influence of both cognitive theories (e.g., information processing) and socio-cultural theories (e.g., Activity Theory) is evident. Within that division, there is a tendency toward quantitative approaches for research in the cognitive tradition and qualitative approaches for socio-cultural studies, although mixed methods are increasingly valued.

RESEARCH

There are two kinds of research in CALL. One is descriptive and exploratory, looking to see what happens when language learners engage in CALL to see what, if anything, happens differently. The second kind, which represents the majority of CALL research, looks at what might be "better" about using particular instances of CALL. But what exactly is "better?" Here are some possible interpretations:

There are no doubt other ways of defining "better," but if we just consider these, a question arises: What are we comparing these to? Presumably, we are comparing a CALL activity to some corresponding non-CALL activity to see which gives us superior results for a given language learning target. This comparative approach has strong face value: language teachers and program administrators are reasonable in wanting evidence that CALL is worthwhile before putting time and expense into it, and comparative research seems the only way to provide definitive answers. Unfortunately, after two decades CALL researchers have not been able to provide those answers, and a number of influential researchers long ago came to the conclusion that in most cases the type of study that pitted CALL against non-CALL was a dead end, just as happened with "method comparison" (e.g., audiolingual vs. Total Physical Response) in the 1970s and 80s. Ultimately, the more interesting and answerable questions were not about the computer vs. its absence, but about specific applications, specific features of applications, specific types of activities, specific environments, and specific characteristics of learners.

As with other areas of second language learning, there are two ways for teachers to approach CALL research. One is  as a research consumer; the other is as a classroom or action researcher. Each of these is briefly discussed below. As with other units, the objective here is to just give a taste of what is an enormous and constantly growing area. Those with more serious research interests are encouraged to consult the reference list. It should be mentioned before continuing that not all CALL research is aimed primarily at improving language teaching and learning with technology. In some cases, researchers may simply want to observe how the technology environment influences or changes the way humans interact with one another, without necessarily judging whether or not it's "better".  Although such basic research is important to social scientists and may lead to more applied hypotheses, it does not directly impact teaching and learning and so will not be explicitly addressed here.

CALL RESEARCH TRENDS

As suggested above, most CAI (computer-assisted instruction) and early CALL research focused on comparing computer users with a control group typically using traditional methods. The results were mixed, often showing no significant difference, sometimes favoring the computer users, and occasionally favoring the traditional approaches (see Dunkel 1991).

Over time researchers began to argue against comparative research (see for example Chapelle and Jamieson 1989), stating that the number of variables was too great. There are now many areas being researched using a variety of quantitative and qualitative methods (although interest in comparative research remains, particularly in comparing face-to-face interaction with CMC). The results vary widely, and according to several research reviews (Felix 2005; Hubbard 2005; Huh and Hu 2005) the field is hampered by widespread problems with research designs and reporting.

Research has continued in all areas of CALL but recently has focused on several identifiable areas, such as:

Here are a few example studies. Keep in mind that these and other studies are generally conducted on small groups in specific settings and may suffer from methodological flaws of various sorts. As with all research, proceed with caution in connecting any findings to your own setting.

First, here are a few studies from the early 2000s

Study

Description

Some Results

Belz, J. (2001). Institutional and individual dimensions of transatlantic group work in network-based language teaching. ReCALL, 13 (2), 213-231. 

Investigated tandem learning with German & US university students

Of 3 tandem groups studied, 1 failed & 2 succeeded, showing individual & group differences are important, not just tasks & technology

Darhower, M. (2002). Interactional features of synchronous computer-mediated communication in the intermediate L2 class: A sociocultural case study. CALICO Journal, 19 (2), 249-277: https://www.calico.org/html/article_426.pdf

Explores use of chat in 2 4th semester Spanish classes: analyzed weeks 1,4 & 8.

Off-task discussion occurred when instructor gone; limited use of L1, social cohesiveness

De Ridder, I. (2002). Visible or invisible links: Does the highlighting of hyperlinks affect incidental vocabulary learning, text comprehension, and the reading process? Language Learning & Technology, 6 (1), 123-146: http://llt.msu.edu/vol6num1/pdf/deridder.pdf

Explores whether visible or invisible links are more effective in getting students to check unknown vocabulary and the effect of each on reading process.

Highlighted links are clicked more often than hidden ones, but without affecting speed, comprehension or learning of vocab.

Fernandez-Garcia, M. and Martinez-Arbelaiz, A. (2003). Learner interactions: A comparison of oral and computer assisted written conversations. ReCALL, 15 (1), 116-136. 

Compared negotiation patterns/numbers across different pairings and different treatments.

NS-NNS group had most negotiation, especially in oral setting.

Green, A. & Youngs, B. (2001). Using the Web in elementary French and German courses: Quantitative and qualitative study results. CALICO Journal 19 (1), 89-123. 
https://www.calico.org/html/article_453.pdf

Reports on study of replacing one of four language class days with independent study on the web

No significant difference in performance after replacing 1 of 4 class days with web.

Kötter, M. (2003). Negotiation of meaning and codeswitching in online tandems. Language Learning & Technology, 7 (2), 145-172: http://llt.msu.edu/vol7num2/pdf/kotter.pdf

Reports on a tandem MOO project looking for examples/patterns of negotiation and codeswitching

Logs showed different patterns from self reports, especially with respect to repetition. Notable differences between MOO & previous face-to-face studies

Schwienhorst, K. (2002). Evaluating tandem language learning in a MOO: Discourse repair strategies in a bilingual Internet project. CALL Journal, 15 (2), 135-145.

 

Reviewed MOO (an elaborated form of chat) logs for examples of repair strategies in a tandem setting.

Students said they used repetition requests a lot but logs disagreed; negotiation occurred and was more prevalent than avoidance or misunderstanding; Germans preferred paraphrases but their partners gave them translations

Sengupta, S. (2001). Exchanging ideas with peers in network-based classrooms: an aid or a pain. Language Learning & Technology, 5 (1), 103-134: http://llt.msu.edu/vol5num1/sengupta/default.pdf

Reports on peer exchanges in two partly networked-based classes: observing patterns

Found two common discourse moves were agreeing & praising--helped build community. Accountability gave feeling of heavy workload. Some students were more active, and one small group was consistently lukewarm.

Stockwell, G.& Harrington, M. (2003). The incidental development of L2 proficiency in NS-NNS email interactions. CALICO Journal, 20 (2), 337-359.  https://www.calico.org/html/article_326.pdf

Reports on a study of proficiency development via email messages: msgs 1, 5, 10 & 15 analyzed

Showed gains in error-free t-units. Recurring pattern was high performance on first msg, then drop on 5th followed by gradual increase to 15th.

Sun, Y.-C. & Wang, L.-Y. (2003). Concordancers in the EFL classroom: Cognitive approaches and collocation difficulty. CALL Journal, 16 (1), 83-94.

Reports on a study of inductive vs. deductive approaches to learning collocations using concordance software: 2 easy, 2 hard ones

Inductive approach was significantly better for easy collocations and almost significant (p=.05) for hard ones

Thorne, S. (2003). Artifacts and cultures-of-use in  intercultural communication. Language Learning & Technology, 7 (2), 38-67: http://llt.msu.edu/vol7num2/pdf/thorne.pdf

Reports on 3 case studies demonstrating artifact- mediated practices as cultural practices in US university students

Case 2: a dyad had an initial problem w/email but went to IM (instant messenger) on their own and started with a 6-hour session. Case 3: US students find email inappropriate for social interaction. Prefer IM.

Yoshii, M. and Flaitz, J. (2002). Second language incidental vocabulary retention: The effect of text and picture on annotation types. CALICO Journal, 20 (1), 33-58. 
https://www.calico.org/html/article_400.pdf 

Compared 3 groups for vocabulary acquisition during reading: glosses with picture only, text only, or combination. Surprise vocabulary tests at end and 2 weeks later.

Combination group outperformed others overall; picture group was best on 2-week delayed posttest.

Compare these with the following more recent studies, many of which point to the need for more learner training and/or support. (see Unit 7)

Study

Description

Some Results

Liang, M.-Y. (2010). Using synchrornous online peer response groups in EFL writing: Revision-related discourse. Language Learning & Technology 14 (1):  45-64. http://llt.msu.edu/vol14num1/liang.pdf Describes the interactions of three online peer groups in a Taiwanese undergraduate ELF writing class. Aim was to study the types of interaction and see how they contributed to subsequent revision. There was little in the way of meaning negotiation, error correction, or technical actions related to revision. Instead, social talk, task management and discussion of content dominated the discourse. Learner training is suggested.
Nielson, K. (20011). Self-study with language learning software in the workplace: What happens? Language Learning & Technology 15 (3):  110-129. http://llt.msu.edu/issues/october2011/nielson.pdf Studied effectiveness of independent study using tutorial CALL programs.  Government workers were provided with the commercial programs Rosetta Stone (150) or Tell Me More (176) . Participants agreed to use RS for 200 hours or TMM for 130 hours of self-study The attrition rates were so high that outcomes could not be determined. Only 21/50 completed the first 50 hours of RSS and only 1 did all 200 hours for a final assessment. For TMM, only 7/176 (none of whom were among the 82 beginners in the language studied) completed more than 25 hours and only 4 reached the level of exit assessment.
Smith, B. (2009). The relationship between scrolling, negotiation and self initiated self repair in an SCMC environment. CALICO Journal 26 (2): 231-245. https://www.calico.org/html/article_739.pdf Uses screen capture instead of just text chat logs to discover what students do during German jigsaw (information gap) activities. Captured self-initiated self-repairs (SISRs) by subjects prior to sending their text to their partners. Negative correlation between scrolling and negotiation of meaning; more SISRs with grammatical points. Argues for a methodological shift in data collection and analysis.
Stockwell, G. (2010). Using mobile phones for vocabulary activities: Examining the effect of the platform. Language Learning & Technology 14 (2):  95-110. http://llt.msu.edu/vol14num2/stockwell.pdf Reports on a 3-year study of trends in mobile vs. PC use for Japanese students doing English vocabulary learning activities. Students were allowed to choose which platform to do the activities on. Although the choice to use mobile phones for vocabulary learning increased over the three years studied, numbers remain low and students significantly spend more time to achieve the same results on mobile devices. There is value in allowing students to make informed decisions as to platform.

Winke, P. & Goertler, S. (2008). Did we forget someone? Students computer access and literacy for CALL. CALICO Journal 25 (3): 483-509. https://www.calico.org/html/article_713.pdf

Reports on a survey of over 900 students at Michigan State in foreign language courses to determine their facility with computer-based tasks and multimedia tool. Many students do not have access to or literacy in the specialized tools for CALL. Training is needed because the computer skills for personal purposes do not transfer to the language learning environment.

There are many areas of CALL that have been looked at, and we only cover a few of them here. Check the references at the end for resources to continue your CALL research review. It should be noted before continuing that CALL research has long been burdened by a problem which has not plagued most classroom-based SLA research: the technology adds a dimension of complexity and  it is constantly changing; consequently definitive answers in any area don't seem to exist.

SURVEY OF UNANSWERED QUESTIONS IN CALL RESEARCH

One of the major concerns that scholars seem to have upon entering this field, particularly if they are trying to develop a project for a master's or doctoral thesis, is what sorts of research questions to study. To address that problem, in the summer of 2002, I sent a survey to 120 CALL professionals around the world asking them to articulate one research question in the field that they would like to see answered. I received 64 responses. A writeup of the results and the actual questions proposed by the contributors can be viewed at www.stanford.edu/~efs/callsurvey. You may submit your own question or comment on those there.

SUBJECT CHARACTERISTICS IN CALL RESEARCH

In 2004 I carried out a study of research articles found in four CALL journals over a 2-3 year period, focusing on subject characteristics. The overall conclusion was that "CALL research as a whole is unbalanced in the direction of the study of novices working on novel tasks or using novel applications" (Hubbard 2005: 363). Among other recommendations, I suggested that more studies be done using experienced and/or trained learners (see Unit 7) so that we can get a more complete idea of the potential effectiveness of specific CALL software and tasks. This should not be taken as a general criticism of more basic observational research (i.e., what do students do naturally when left on their own in a CALL environment), which is also quite important--the point is that the CALL research domain should be more balanced than it currently seems to be. Results from an unpublished followup study looking exclusively at CMC research reached a similar conclusion: see www.stanford.edu/~efs/pacslrf06.

DOING RESEARCH

As noted in the introduction, this course is largely aimed at classroom teachers interested in beginning or expanding their use of CALL, and teachers can take the role of researchers themselves.  identifying a learning gap, creating a possible solution for it--in this case using technology--and then doing research on the effectiveness of that solution. There are several avenues available to teachers in the role of researchers of their own classroom or students.

LOCATING RESEARCH STUDIES

A search through Google Scholar (http://scholar.google.com) using appropriate keywords is one way to find research materials on a CALL topic. However, a problem is that many of the sources discovered in this manner will not be freely available. If your library does not carry journals such as Computer Assisted Language Learning, ReCALL, or System,  then the two most useful sites to search are CALICO (www.calico.org), which has CALICO Journal articles over three years old freely available, and Language Learning & Technology (http://llt.msu.edu), where all the journal articles are freely available. Both sites have internal search features.

REFERENCES

Chapelle, C. & Jamieson, J. (1989). "Research Trends in Computer-Assisted Language Learning." in Pennington, M. (ed.) Teaching Language with Computers. La Jolla: Athelstan.

Dunkel, P. (1991). “The Effectiveness Research on Computer-Assisted Instruction and Computer-Assisted Language Learning." In Dunkel, P. (ed.) Computer-Assisted Language Learning and Testing: Research Issues and Practice. New York: Newbury House/Harper Collins.

Egbert, J. & Petrie, G. (eds.) (2005). CALL Research Perspectives. Mahwah NJ: Lawrence Erlbaum

Egbert, J. & Hanson-Smith, E. (eds.) (2007). CALL Environments: Research, Practice, and Critical Issues, Second Edition. Alexandria, VA: TESOL.

Felix, U. (2005). "Analyzing recent CALL effectiveness research: Towards a common agenda." CALL Journal 18.1-2.

Hubbard, P. (2003). "A Survey of Unanswered Questions in CALL." CALL Journal 16.2-3.

Hubbard, P. (2005). "A Review of Subject Characteristics in CALL Research." CALL Journal 18.5

Hubbard, P. (2008). "Twenty-five Years of Theory in the CALICO Journal". CALICO Journal 25.3. https://www.calico.org/html/article_707.pdf

Hubbard, P. (2009). “Developing CALL Theory: A New Frontier.” Proceedings of the JALTCALL Conference, Nagoya, Japan, May 2008.

Huh, K and Hu, W. (2005). Criteria for effective CALL research. In J. Egbert & G. Petrie (eds.) (2005). CALL Research Perspectives. Mahwah NJ: Lawrence Erlbaum

Home | Unit 1 | Unit 2 | Unit 3| Unit 4| Unit 5| Unit 6 | Unit 7 | Unit 8 | Supplement


Last modified: February 21, 2014, by Phil Hubbard