How does evaluation take place?
The simplest answer to this question is to say that, at the end of oral lessons that requires testing, you will find four highly similar forms of the test in question. These timed tests are preceded by a grading key, for the benefit of the teacher, and each form of the test is followed by a video of an instructor providing the correct answers. The students are encouraged to practice these four tests in advance of the actual testing day. On the testing day, the teacher spontaneously selects one of the four forms of the test for the students to perform, thus impeding them from simply memorizing one set of answers, but rather obliging them to master the structures contained therein. Written tests exist in three forms at the end of their lesson and contains links to the correct answers which, again, the students can practice in advance of the test.
Normally, the primary elements that contribute to assigning a student a grade are as follows:
1. Oral participation
2. Oral or written tests (focusing on discrete skills)
3. Extended oral presentations or extended writing assignments
However, a more in-depth response to this question depends upon who it is who is performing the role of the teacher. Is it a trained world language teacher, a classroom teacher who does not speak the language being taught or a parent who similarly does not speak the language in question? There will be a big difference between how many of these elements the parent, the non-language teacher and the trained language teacher will be able to take into consideration in light of their respective knowledge or lack of knowledge of the language.
SUGGESTIONS FOR PARENTS SUPERVISING HOMESCHOOLERS
Let’s start with the simplest case – that of the parent of a homeschooled student. Parents of a homeschooled student, who has no other language instructor, will only be able to evaluate the accuracy of their child’s performance on the second type of activities – oral or written tests that focus on discrete (individual, separate, distinct) skills. Within the ULAT, such tests are found at the end of certain lessons and are marked by the presence of a stop sign in the Table of Contents.
Each of these tests exists in four forms, which allows the student to study and practice them in advance of the test, but which are numerous and varied enough to impede the student from simply memorizing the correct answers. At the end of each of the four forms of the test, the correct answers are provided – in the form of a video for oral tests and in simple written form for written tests. In the case of an oral test, homeschooling parents can record their child’s voice while taking the test and then compare their child’s responses to those heard in the video containing the correct answers. In the case of a written test, they can simply print out their child’s responses and then compare them to the written correct answers found at the end of each form of the test. The video below shows how the testing process can be supervised by a non-speaker of the target language.
Proposed grade weighting: 100% attributed to oral and written tests
*****
This is a somewhat more complex situation because one must consider additional questions. How much time is being devoted each week in the classroom to language study, what is the age of the students and, consequently, what are the expectations for the students’ experience? Is the goal merely to “give them a taste” of the study of a new language or are they engaged in college preparatory language study with the goal of obtaining advanced placement credit upon completion of the program?
If the goal is simply to give the students a “taste” of language learning – and this is likely the case with elementary and possible middle school students – then teachers will want to give more emphasis to game playing and oral participation, at least until the introduction of written work becomes appropriate. In this case, only the first element above – oral participation – should be taken into consideration. Whereas teachers may want to design their own simplified means of recording and grading oral participation, be aware that a very complete and effective means of accomplishing
this task is presented in the following video:
Proposed grade weighting : 100% attributed to oral participation
In the unlikely event that classroom teachers, who are unable to speak the target language, are working with upper middle school and high school students, they will want to incorporate both of the first two elements above in their evaluation of their students. They will want an objective means of recording and awarding oral participation and they will want to employ the same approach to evaluate the testing that homeschooled parents must use. (See the video under “Parents Supervising Homeschoolers”.)
Proposed grade weighting system: 50% attributed to oral and written tests; 50% to oral participation
*****
SUGGESTIONS FOR TRAINED WORLD LANGUAGE TEACHERS
Trained world language teachers are able to take all of the above elements into consideration in evaluating their students’ progress, and undoubtedly will want to add another element or two of their own choice. Consequently, in their case, I highly recommend giving consideration to all of the following suggestions and reflections on student evaluation.
For giving our students a reason to apply themselves in their language studies, nothing beats a trip to the nation where the language that your student is learning is the official language. Other types of experiences are motivating as well, such as befriending a foreign student, hosting an exchange student or international online communication. Nonetheless, a far simpler and motivating kind of experience that you can regularly provide your students is…testing. Whereas a test may not generate the warm, fuzzy feelings of interpersonal relationships, provided that your students are concerned about their G.P.A., it is an event which focuses their attention, moves them to intense repetition and marshalls the very best of their capacities – that is, as long as the test rides that fine line between challenging and impossible.
Therefore, while awaiting the opportunity for international travel or communication, teachers do well to craft strategic exams that truly test that which they have as their objective for the student. The reason for this is simple. That ability which you test is that for which students will prepare themselves. If we claim that we believe in the primacy of oral communication, for example, and yet provide our students exclusively with text-based tests, which are far easier to structure and evaluate, we are kidding ourselves. What we esteem, and therefore teach, must line up perfectly with what we test. The weight we give to the various elements making up a student’s grade speaks volumes about what we really expect of our students.
In this section, I will be sharing perspectives on testing and evaluation that are not directly tied to the ULAT program. However, once I have spoken about each form of evaluation, I will share examples as the points in the ULAT at which those forms might be applied.
These are the different types of testing which I do in the first year of study, as well as the weight I attribute to each element:
1. Oral participation – follow-up drill and open-ended discussion (45%)
2. Oral PowerPoint or oral presentation testing (45%)
3. Miscellaneous assignment completion (10%)
Before coming back and explaining the items above, let me first show you the evaluation elements and weighting for the second year language student:
1. Oral participation – follow-up drill and open-ended discussion (30%)
2. Oral PowerPoint or oral presentation testing (30%)
3. Written test or composition (30%)
4. Miscellaneous assignment completion (10%)
By the third year, the balance shifts fully to the side of the written language. But why do we place so much emphasis, for example, on oral participation in the first two years? The reasons are two-fold. If students do not become comfortable with expressing themselves in the target language during the all-critical first year, my experience has shown that they likely never will. If students can first acquire oral skills in their new language, following some training in the phonics system of that language, and some elementary reading exposure, teaching them to read and write is a relatively simple task. The converse, however, is absolutely not the case. This should not surprise us as such an order (reading to writing to speaking to listening) bears no resemblance to how we naturally learn language. Additionally, if I can get students to open their mouths, prompted by much praise and by the realization that oral participation is required (45% of the grade) in my class, then I can begin to help them refine their speech.
The key, therefore, is that I must have an absolutely objective means of evaluating oral activities, otherwise students will be frustrated or argumentative and I will not want to conduct such tests. Yet, without an almost exclusive emphasis upon oral testing in the first year, students will not feel impelled to develop their oral skills. How critical is this? I would posit that a student who does not become comfortable with speaking the language in a first year classroom almost invariably never will.
One more point before explaining each form of testing mentioned above. I want to be sure that I am authentically testing the particular language arts skill that I claim I am testing. For that reason, all of my tests are extemporaneous – no use may be made of written notes or text. For example, if I ask students to give a 3-minute oral presentation summarizing a particular episode of a video we are watching, so that they not “freeze up” through nervousness while making their summary, I display a very brief outline of the main points of the episode that they may consult. However, they may not give their supposedly “oral” presentation by reading a pre-written text. That would be more comfortable for the student, but I would be measuring their writing and reading abilities, but not their oral expression.
ORAL PARTICIPATION
By far, oral participation is the most important tool we have to train our students. Oral participation activities are important for the students as they oblige them to speak the language, which thus makes them more comfortable with speaking and provides them with the chance to receive feedback from the instructor.
Oral participation activities are also important for the teacher for two reasons. First, as explained earlier, they allow us to hear from the students frequently enough to begin helping them refine their speech. (You can’t steer a car unless it’s moving.) Secondly, and this is a very important matter for a new teacher in particular, it takes the pressure off of the teacher to “carry the class” and places it squarely on the shoulders of the students. It all goes back to the issue of accountability.
Without the sense that their contributions in class are obligatory, particularly when February rolls around (and particularly for those of us in the north who are struggling with our students’ “cabin fever”), students can tend to sit back and expect you to put out all of the effort to “entertain” them. The attitude can almost become: “Teach me…I dare you.” On the contrary, the students in my classes have been so thoroughly trained to understand that their grade “sinks or swims” on the basis of their oral participation that there is a constant competition to be the next to speak. (A little sheepishly, I admit that they are almost like Pavlov’s dogs. Even on the very rare occasion when I might not be recording their participation, their hands fly up to speak as though they cannot help themselves!) Though it may seem that I am overwhelming the students with the constant burden of having to participate, it is clear that their doing so enlivens the class period and actually results in greater interest and more enjoyment for them.
Now, so that my evaluation of oral participation might be objective, fair and tightly tied to specific student behaviors, I confess that the system appears rather complex. Do not be deceived however. What was complex to create, is now very easy for you to use and I will explain the use of the oral participation spreadsheet in a moment. Watch the following video for an explanation of my system:
Parents and students may want to understand how such an important factor in a student’s grade is determined. With that in mind, I have written an explanation of my oral participation recording system that is so exhaustive that I am never again challenged once they have had the chance to read it. Do not bother reading it yourself or you may grow discouraged! It is there simply as a resource to hand to a parent or student. In reality, whereas explaining the system is very complicated, using it is actually simple enough that I have had people with minimal computer skills thoroughly versed in its use in less than 30 minutes – and vowing that they would never cease using it.
You will need a means of recording oral participation, as well as other daily class occurrences. Below you will find a link to download a seating chart grid to which you can add your students’ names. You will have to modify it, of course, to correspond to the layout of your own classroom.
Typically, I print this sheet and make about 30 photocopies of it to place on a clipboard on my desk. I do not make more than that quantity of copies, as students may be added or removed from a class list throughout the marking period. On that grid, I record a number of events. I write a capital “A” within the student’s rectangle to indicate an absence and a “T” for a tardy arrival to class. Recording absences here is very important to a fair and accurate oral participation score, as will be explained later. Positive oral contributions are indicated by a vertical hash mark and negative behaviors with a horizontal one. I also record there any test scores that were obtained during the class period. This makes the seating chart grid a very handy tool, as it contains all the data you need when placing your records online later in the day. You can modify the grid to any seating configuration. (By the way, you may wonder how I can arrange a classroom to be eight students wide. This is through the use of chairs, instead of desks, which will be explained later in the teacher training section dealing with classroom layout.)
Once you have downloaded and opened the oral participation spreadsheet as well, you will notice that passing your cursor over cells that have a red triangle will reveal comments regarding the use of that cell. You will also notice that columns H, J, K and L are missing. Those three columns contain formulae with which you never need to be concerned.
Complicated as this spreadsheet may appear, it is actually very simple to use. After entering some simple initial data at the beginning of the school year, you only need to do two elementary tasks on a daily basis and your students’ oral participation grade will appear automatically. When the school year begins, add the information requested by the comment in cells A6, A7 and A8, and then add your students’ names in column B, replacing “Student no. 1” with the actual name of your first student. By the way, you will find daily recordkeeping all the faster and easier to do if you place students in this list in the same order in which they are seated in your “Seating Chart Grid”, since that is where you will be recording their daily oral participation points.
Now, on a daily basis, or as often as you care to update your records, you only have to enter each day’s results and record the portions of a class discussion for which any students were absent. For example, after the first class discussion, in cell O5, record the day’s date, and then below it type in the number of oral participation points each student received. Remember to record your points as well in the black cell at the bottom of the column, if you were obliged to respond to your own questions. This is a very important factor in maintaining lively, sustained speech from your students. If they understand that the more you have to answer for them, the more negative is the impact on all of their oral participation grades (and the spreadsheet’s formulae are set up so that indeed will happen), they will make every effort to respond to all of your questions and instructions. Secondly, having recorded your students’ daily participation points, you merely need to give students credit for their absences, so that being absent from school does not unfairly impact their score. Simply look at the fraction of a class devoted to class discussion at the bottom of each day’s column (cell O46 for the first day, for example) and then add that fraction to any previous absence totals in column G. Thus, if a student has already missed 1.3 class discussions, and is absent for conversation corresponding to .4 of a class period, you will change his total in column G to 1.7. (The portion of a class period devoted to class discussion is expressed in a decimal because it is rare that you will actually spend 100% of a class period in conversation. For that reason, using 70 comments as the standard for a typical 50-minute class period, cumulating your students’ comments during a class period will indicate approximately the fraction of the period devoted to discussion. Do not be concerned by the actual length of your class period, as it is of no consequence as far as the sheet’s formulae are concerned.)
In sum, on a daily basis, all you need to do is to:
1. place the date at the top of the column in row 6 and record the total of student and teacher remarks
2. add the portion of a class discussion missed by absent students in column G.
ULAT APPLICATION: Now, where does all of this discussion of oral participation apply to the ULAT program itself? In a sense, it can apply to any classroom, regardless of the curriculum you are using. Nevertheless, with reference to the ULAT, you will want to conduct oral participation activities in two types of situations (and hopefully do so on an almost daily basis).
First of all, oral participation points should be awarded in the context of open-ended, spontaneous conversation. For example, after listening to a Skype conversation, such as one finds in lesson 1.46, or following one of the reading assignments or simply regarding a topic suggested by the instructor, the students can be offered the opportunity to summarize the conversation, answer questions about the reading or volunteer responses to the topic the teacher proposed. An example of this latter situation would be, in the course of Unit 2 during which the students learn to speak of their daily routine, to have one’s students share what they would change about their daily routine if they could. In the course of such open-ended discussions, in which there are no right or wrong answers, any comment which is on the proper topic, no matter how grammatically flawed, should receive credit.
The second type of oral participation situation deals with the review of a ULAT lesson’s structured vocabulary or grammar activities in which the students must form a statement on the basis of a series of images with which they are presented. Typically, this sort of a review is performed on the classroom screen immediately following an independent study session during which the students studied the lesson on the classroom or computer lab computers. When the students are asked to form statements regarding material that they have just studied at length, their responses must be completely accurate in order to receive credit. In this manner, they are held accountable for their use of independent study time and the teacher is able to insist upon structural accuracy.
ORAL PRESENTATIONS
All forms of speech can probably be subdivided into four levels of difficulty. Starting with the most elementary level, the easiest form of verbal interaction is to simply provide a “yes” or “no” answer to another’s questions. This is truly not speech, per se, and any test requiring such a level of response from the student is truly only measuring listening comprehension. The second level is to respond to another’s question with a single word answer. The third level is to make a one-sentence statement or response. The highest level involves extemporaneous extended discourse. This is the kind of speech in which I would engage when we lived in France and I would give a pep talk to the sports teams I coached, or a motivational talk to their parents about taking an active role in their kids’ lives or when I preached in the church I helped pastor. It is this level of speech to which oral presentations refer. We want to train our students not merely to be responders, parroting back single word or single sentence responses learned in class, but to be initiators – able to string together a series of extemporaneous sentences to inform, to motivate, to explain, etc.
Nonetheless, objectively measuring the value of a student’s speech in the form of a grade is a very challenging task. It cannot repose upon a teacher’s mere subjective impression, but needs to be determined on the basis of hard data. I arrived at a system which takes into consideration the talk’s content, the rate of speech and the grammatical accuracy of the student’s discourse.
When I assign an oral presentation to be done, I give the student a topic and a period of time to be respected for the talk. As I am not interested in measuring their oratorical skills, sang froid or memory, I provide them with an outline for the topics on which they should touch in their talk. For short presentations, the ULAT provides them with images that remind them of the topics about which they may speak. For lengthy presentations, three minutes or more, I typically place a brief written outline of the main points they will want to mention, but only in the target language. These measures are taken so that students will not be penalized by having their nerves cause them to “freeze up” during their talk and thereby forget what they wanted to say next.
Go to the 14:30 mark of the video below to watch an explanation of my oral presentation evaluation system and then download the guide found below the video:
ULAT APPLICATION:
Oral presentations are summative assessments in which the students “put it all together”. They combine the discrete elements of prior lessons to perform a wholistic activity in which the express themselves, extemporaneously and at length, in a series of complete sentences. Generally speaking, this type of activity will take place at the conclusion of the study of a particular topic. For example, in Unit 3, students learn to describe a person’s physical appearance, emotional state, clothing, health and physical condition. At the conclusion of those lessons aimed at each of those topics, a student can be asked to show the class an action picture of a family member or close friend and to describe the person in all of the aforementioned ways and then, to add the present progressive tense previously studied, to tell what the person is doing in the photo. After having seen an episode from a video, students can be given three to five minutes to summarize the events of that episode for a period of three minutes.
ORAL GRAMMAR AND VOCABULARY TESTS
The ULAT contains a great number of oral grammar and vocabulary tests. These are located at the end of every two or three lessons and summarize the main point on which those lessons were focused. The symbol of a stop sign in the upper right corner of a lesson means that the lesson will conclude with a test. By clicking on the stop sign, one can skip past all of the lesson’s activities and go directly to the tests.
A potentially surprising aspect of the ULAT’s vocabulary and grammar tests is that they are in full view online for the student at all times. In other words, no attempt is made to hide them and then to “surprise” the students with their content when the time comes from them to be tested. Quite the contrary is the case! The tests do not seek to measure the students’ recollection of analytic facts regarding the language, but rather their actual ability to perform a skill. With that in mind, it is entirely in the instructor’s interest that his or her students might practice the tests over and over again prior to taking them. In that each test exists in four forms, and that the students do not know which of the forms of the test they will be required to do, there is no danger of them merely memorizing a series of answers and not actually being able to perform the task the tests are requiring of them. Note that you will find instructions as to how to grade the tests in the daily lesson plan for the lesson containing the test.
When the student takes a test, he has a specified period of time during which he must form the statements. (If the images simply look confusing and meaningless to you, because you are looking at the ULAT for the first time, I can assure you that the student becomes extremely well-versed with their meaning as they are presented and then repeatedly reinforced throughout the lessons. In any case, insisting upon the timing element in these tests is important, as the students must be challenged and obliged to think and express themselves in a two-step process (concept to target language), as opposed to the typical four-step process (image-laden concept to native language thought to text-based thought in the native language to a translation to the target language) so commonplace as a result of
traditional text-based language instruction. The students need to understand that they must match the pace required by the test, thus
requiring significant repetition and practice, in order to succeed in your class.
Below you can see a video regarding how such a test is performed:
WRITTEN COMPOSITIONS
Written compositions are very similar to oral presentations and generally on the same subjects. You can see suggested subjects for oral presentations and written compositions for Units 1 through 7 within the Teacher’s Table of Contents. The student is given a topic about which to write, a minimum number of words to write and a specified period of time during which to do the writing. If the student is assigned to write a summary of a particular chapter of a book or video being studied, the teacher may choose to provide
him with a very brief outline, written in the student’s native language, of the main points in that chapter.
I evaluate my students in much the same manner as I do an oral presentation. Allowing time to reflect as they write, and to be able to proofread their work, I want my students to be able to write at a minimum rate of five points per minute. Thus, if given 30 minutes during which to write and proofread their work, I set the minimum point total for the assignment at 150 points. To avoid circuitous writing, in which the students are saying the same thing multiple times and merely seeking to accumulate points, I designate a certain number of topics or events which they must cover in their writing. Click on the link below to see the written composition evaluation guide:
WRITTEN VOCABULARY AND GRAMMAR TESTS
Written vocabulary and grammar tests are certainly the most traditional form of test in the language learning classroom and their grading does not require
any particular explanation. Such tests in the ULAT are found only at the conclusion of lessons in Units 9 and 10. Below is a video showing
how such written tests take place in the ULAT:
MASTERY LEARNING
One final explanation and one suggestion on evaluation are in order. You may be surprised to discover that the actual tests that students are to take are available to them online at the end of the lesson. Yes, I intentionally want students to see and practice taking their tests before they officially take them with me in class. If the tests are well made, they will correspond exactly to the skill I want my students to master. I want them to practice them…over and over. You may say to me: “But they can just memorize the test!” In a way, my response would be: “Great! They will undoubtedly learn something in so doing.” Nonetheless, I have a realistic side as well and realize that rote memorization will not necessarily result in authentic learning. It is for this reason that all tests exist in multiple versions. Therefore, the students do not know which version they will actually take when they get to class. Thus, they cannot simply memorize all of the tests (though I would be pretty pleased if they could).
As for my suggestion, it is this: allow your students to retake their oral tests for a limited period of time in order to improve their score. Oral expression is the foundation to language learning. You can teach an orally proficient student to read and write, but you will have great difficulty teaching a student to speak who learned a language exclusively by reading and writing. You do not want your students to move on to reading and writing until they have a strong oral foundation. You want to give them every opportunity to master each step of the oral language learning process.
For this reason, I allow my students to retake any oral test for a period of one week after they took it the first time in class. To protect instructional time in class, if they want to retake a test, they must do it immediately after school. I also only allow them to retake the same test once per day to ensure that they will have the time to put in some additional study before they try again. As the oral tests are very brief in nature, their retaking them does not impose any unusual burden on my workload. The students also know that I will never lower their grade if their second (or third or fourth…) score on the test is lower than their previous best. This assurance gives them the confidence to retake tests and thus to keep applying themselves to the skill I want them to master.
Stephen Nesbitt, 2013