Analysis and Results
After the portfolios were received, native speakers of each language examined the contents of the portfolios. The objective of the portfolio review was to determine whether the portfolios were useful in documenting language growth over the course of the school year. The portfolios contained student writing samples, quizzes, audio and video tapes, and special group projects of a written nature (e.g., family stories, history of a country, etc.). The portfolio examiners in each language recorded the contents of each student portfolio and made comments regarding overall organization/structure of the portfolio and each artifact's usefulness in terms of charting growth in the less commonly taught language. In addition, the native language reviewers compared the student portfolios with the teacher's initial portfolio plan.
To coordinate the portfolio review, the examiners of each language also met as a team frequently throughout the process of studying the contents of each portfolio. The purpose of these meetings was to compare notes and discuss the usefulness of the various artifacts found in the portfolios for assessing growth in the four language skills. These discussions proved to be useful because they highlighted the strengths and weaknesses of the student portfolios. For example, a frequently noted observation in most of the portfolio evaluators' comments was missing dates of when various materials (e.g., written samples) were completed by the students. Absence of dates hindered our analysis of documenting language development across the school year.
As another example, we found that even though all teachers included audio and video tapes in the portfolios, these artifacts varied widely in their usefulness for assessing students' proficiency in the less commonly taught language. In some cases, the audio or video tapes were of spontaneous speech samples which showed the student's ability to use the language communicatively. In other cases, the audio or video tapes were dialogues that had been memorized by the students and from which it was difficult to determine whether the students in fact had mastered the language beyond the simple rote memorization of the script. In these instances the audio or video tapes were useful in assessing accuracy of pronunciation in the target language, but not communicative competence.
Another problem was that teachers often included only a single audio or video tape for the designated student which had generally been prepared sometime during the second semester. The reason given for this generally had to do with student reluctance to be taped or the difficulty of locating suitable equipment or finding time within the instructional period to do the taping. Thus there was no way of judging growth in oral competency through the school year for some students. In those portfolios where there were at least two audio or video tapes and where the first had been done early in the school year and the second late in the year, it was possible to assess the student's growing oral skill in the target language. This was especially true in introductory foreign language classes (e.g., Japanese I). The portfolios also revealed that teachers at the advanced levels of foreign language instruction (e.g., Chinese IV) used a more thematic approach to their instruction. This made analyzing the portfolios not only more interesting because of the contents, but also more difficult to determine specific communicative growth across the academic year.
Another concern with respect to the review of the audio and video portions of the portfolios was that often times two or more of the case study students collaborated in preparing and presenting a skit or puppet show. Then when the production was taped, it was not possible to always identify each speaker or what the individual's contribution was to the material. Although these productions were entertaining and obviously very enjoyable for the students, they posed problems from an evaluation perspective. Thus, it was not always possible to adequately assess spontaneous communicative competence from these group presentations.
The written samples in the portfolios also provided important information regarding the growth in the target language. However, many teachers provided too few written samples to adequately judge writing skill development across the academic year. Portfolios which contained numerous and dated writing samples proved very useful in assessing growth in writing ability in the target language. Some teachers included in the portfolio various drafts of students' writing assignments. These were especially valuable since they showed the developmental stages of writing in the foreign language in response to their teacher's comments regarding their attempts to complete the written assignment.
As part of the process of examining the contents of each portfolio the reviewer prepared written commentaries on the contents that would be shared later with the teacher. These commentaries were first presented to the evaluation team during our meetings as we discussed the usefulness of the contents of all portfolios for documenting language growth in the target languages. The comments were useful in aiding team members formulate guidelines for foreign-language portfolio assessment. In addition, the teachers found the commentaries useful in planning for the following year's student portfolios. For example, in the case of one student portfolio the native language reviewer commented:
Later in the same commentary and referring to "Reading" the evaluator states:
As for writing the evaluator says:
In sum, these comments illustrate the strengths and weaknesses of a particular portfolio. The material in the portfolio allows the reviewer to determine both the level of proficiency attained by the student in the foreign language classroom and the strategies used by an exemplary teacher of Japanese to assess her students' acquisition of the language. The commentary also offers feedback to the teacher for improving the assessment process that will be useful for the multiple audiences for which the portfolio is prepared; that is, the teacher, the student, and the student's parents.
Home | Organization | Map | Events and Discussions | Resources | Standards and Frameworks