Many techers feel comfortable setting pencil – and  – paper tests. Years of experience marking written work have made them familiar with the level of writ-ten competence pupils need in order to succeed in a specific standard. Howev-er, teachers often feel much less secure when dealing with tests which measurespeaking and listening even though these skills are regarded as essential compo-nents of a diagnostic test which measures overall linguistic proficiency. Althoughthe second-language English pupils often come from an oral rather than a writ-ten culture, and so are likely to be more proficient in this mode of communica-tion, at least in their own language, speaking in English may be a different mat-ter. In English medium schools in particular a low level of English may impedestudents’ acquisition of knowledge. Therefore, identifying the correct level ofEnglish of the student is all the more challenging and important.This article outlines some of the problem areas described by researchers whendesigning a test of oral production for beginning-level speakers of English andsuggests ways in which they may be addressed.

How does one set a test which does not intimidate children but encouragesthem to provide an accurate picture oftheir oral ability?

In replying to this question, one needs toconsider briefly the findings of researchersworking in the field of language testing. “Thetesting of speaking is widely regarded as themost challenging of all language tests to prepare,administer and score,” writes Harold Madsen,an international expert on testing (Madsen1983:147). This is especially true when exam-ining beginning-level pupils who have juststarted to acquire English, such as those apply-ing for admission to primary school. Theo-rists suggest three reasons why this type of testis so different from more conventional types of tests.Firstly, the nature of the speaking skill itselfis difficult to define. Because of this, it is noteasy to establish criteria to evaluate a speakingtest. Is “fluency” more important than “accu-racy,” for example? If we agree fluency is moreimportant, then how will we define this con-cept? Are we going to use “amount of infor-mation conveyed per minute” or “quickness ofresponse” as our definition of fluency?A second set of problems emerges when test-ing beginning-level speakers of English, whichinvolves getting them to speak in the first place,and then defining the role the tester will playwhile the speaking is taking place. Relevant elic-itation procedures which will prompt speakersto demonstrate their optimum oral perfor-mance are unique to each group of speakers andperhaps even unique to each occasion in whichthey are tested. The tester will therefore need toact as a partner in the production process, whileat the same time evaluating a number of thingsabout this production.A third set of difficulties emerges if onetries to treat an oral test like any other moreconventional test. “In the latter, the test isoften seen as an object with an identity andpurpose of its own, and the children taking thetest are often reduced to subjects whose onlyrole is to react to the test instrument” (Madsen1983:159). In oral tests, however, the priorityis reversed. The people involved are important,not the test, and what goes on between testerand testee may have an existence indepen-dent of the test instrument and still remain avalid response.

How can one accommodate these diffi-culties and still come up with a valid testof oral production?

In answering this question, especially inrelation to the primary school mentioned ear-lier, I would like to refer to the experience Iand one of my colleagues, Viv Linington, hadin designing such a test for the Open LearningSystems Education Trust (OLSET) to measurethe success of their English-in-Action Pro-gramme with Sub B pupils. This Programmeis designed to teach English to pupils in theearliest grades of primary school, using themedium of the tape recorder or radio.In devising this test, we decided to use flu-ency as our basic criterion, i.e., “fluency” inthe sense Brumfit uses it: “the maximally effec-tive operation of the language system so faracquired by the student” (Brumfit 1984: 543).To this end, we decided to record the totalnumber of words used by each pupil on thetest administration and to employ this as anoverall index to rank order the testees in termsof performance.To address the second and third set of prob-lems outlined above, we decided to use elicita-tion procedures with which the children werefamiliar. Figures 1 and 2 would require theteacher to find a picture full of images thepupils could relate to such as children playing.Students could participate in the followingtypes of activities:
• an informal interview, to put the childrenat ease by getting them to talk aboutthemselves, their families and their homeor school lives (See Figure 1).
• a set of guided answers to questionsabout a poster, to test their knowledge ofthe real life objects and activities depict-ed on the poster as well as their ability topredict the consequences of these activi-ties (See Figure 2).
• narratives based upon packs of storycards, to generate extended language inwhich the children might display suchfeatures as cohesion or a knowledge ofthe English tense system in an uninter-rupted flow of speaking.

The Interview

The tester should capture personal details byasking the following type of questions:What is your name?Where do you live?Do you have any brothers or sisters?Does anyone else live at home with you?Now tell me, what do you all do when you get up in the morning?How do you all go to school and work?Do you have any brothers or sisters in this school?What standards are they in?Which subject do you enjoy most? Why?What do you do at break?Tell me about your best friends.What does your mother/grandmother cook for dinner?Can you tell me how she cooks it?Why do you all enjoy this food most?Do you listen to the radio/watch TV in your house?What is your favorite programme?Why do you enjoy it most?What do you do when you are getting ready to sleep in the evening?What time do you go to sleep. Why?Now look at the picture and tell me what this littleboy is doing. Letʼs give him a name. What do you suggest?
Figure 1

“Arriving at School”

Questions for guided response:What are the children doing?Where are they?How many children are there?Are there more boys than girls?How do you know this?What is the girl in the green dress doing?What are the boys going to do when they finish playing marbles?Do you think the children are happy?Have you ever played marbles?(If yes)How do you play marbles?(If no)What other game do you play with your friends?How do you play it?Now look at the picture and tell me what this littleboy is doing. Letʼs give him a name. What do you suggest?
Figure 2

Instead of treating the situation as a “test,”we asked testers to treat it as a “game.” Bothpartners would be seated informally on theground (with, in our case, a recorder placed of each child’s experience, but there was anunderstanding that how and why questionswere more difficult to answer than other Wh-questions. A range of both types should there-fore be used. Story packs also provided for a range ofexperiences and could be used by the testertelling a story herself first, thus demonstratingwhat was required of the pupil. However, itwas anticipated that some pupils might be suf-ficiently competent to use the story packswithout any prompting from the teacher.Pupils could place the cards in any order theychose, as the sole purpose of this procedurewas to generate language. Story packs werecomposed of picture stories that had beenphotocopied from appropriate level books, cutup into individual pictures, and mounted oncardboard. Six pictures to a story pack wereconsidered sufficient to prompt the anticipat-ed length of a story pupils could handle. This test of oral production was administer-ed at both rural and urban schools to childrenwho were on the English-in-Action Programmeand those who were not. The comparative re-sults are not relevant here, but findings aboutwhich aspects of the test worked and whichdid not may be of assistance to those who wishto set similar tests. In summarising these find-ings, I will comment on the administration ofthe test, the success of each subtest in eliciting unobtrusively on the floor between them be-cause of the research nature of our test). If theoccasion was unthreatening to the pupil withthe tester acting in a warm friendly way, weanticipated the child would respond in a simi-lar way, and thus produce a more accurate pic-ture of his or her oral productive ability. Wesuggested the tester act as a Listener/Speakeronly while the test was being conducted, and asAssessor once the test administration was over.To maintain a more human approach tothe testing situation, we decided to allow thetester a certain flexibility in choosing ques-tions to suit each particular child, and also inthe amount of time she spent on each subtest.The time allowed for testing each pupil wouldbe limited to 8 minutes, and all three subtestswould be covered during this period, but theamount of time spent on each could vary.Question banks were provided for testers toselect questions they felt were within the range language, and, finally, on the criteria we usedfor evaluating the test outcomes.Firstly, both testers commented that thistype of test was more difficult to organise andadminister than other kinds of evaluation teststhey had used. This was caused by the need tofind a quiet and relatively private place to ad-minister the test and record the outcome andbecause the procedure could be done only ona one-to-one basis. We had anticipated thistype of feedback but were also not surprisedwhen told that subsequent administrations“were much easier and the children were moreenthusiastic about participating than the pre-vious time.” The testing procedure was new toboth tester and testee, but once experienced, itgave children greater freedom of expressionthan other kinds of tests.Secondly, while the test as a whole did elic-it oral language production, the amount andtype of language varied from subtest to sub-test. The interview produced rather less lan-guage than the other two subtests; it also elicit-ed rather learned chunks of language, whichwe called “patterned responses.”The guided responses, on the other hand,produced a much greater variety of answers,couched in a fairly wide range of grammaticalstructures. But even these responses consistedon the whole of single words or phrases.Open-ended questions evoked longer respons-es from the more able students, but seemed toconfound less able students. For example, thequestion “What can you see in the picture?”produced the answer “I can see a car and awoman going to the shop and a boy had abicycle and the other one riding a bicycle,”from a bright pupil, but only “Boy and bicy-cle” from a weaker pupil.Higher order Wh- questions such as “Whatdo you think is in the suitcase?” or “What willhappen next?” seemed to produce only “Idon’t know” responses from even the mostcompetent pupils. They seemed to lack thelinguistic resources, or perhaps the cognitiveresources, to predict or suggest answers.The narrative subtest, based on the storycards, elicited the best display of linguisticability from the testees, both in terms ofamount of language produced and range ofgrammatical structures used.Competent pupils were able to respondwell to the tell/retell aspect and constructedsentences of 7 to 10 words in length, joined by a variety of coordinating devices. They alsoemployed past tense forms in retelling thestory such as the following:The boys they played with the cow’s what…… what …… a …… bells three bells…… then they got some apples and wentto swim …… the monkey saw them swimand putted them shirts and shorts ……some they said hey …… I want my shirts…… wait I want my shirts …… but mon-key she run awayLess competent students could describeisolated images on each card without usingnarrative in any way to link them together.From these results we therefore concludedthat the story packs were the most successfulof the three elicitation procedures we used instimulating optimum language output.The final issue from the findings of theOLSET test that are relevant here are the cri-teria used for assessing the language output.Our decision to count “number of words pro-duced” as a measure of speaking ability was amixed blessing. Initially it did seem to rankorder the pupils in terms of ability and gave usa base for comparison at subsequent testadministrations, but non-verbal factors such asself confidence, familiarity with the tester, andpresence of the teacher may have affected eventhese results. In the second administration ofthe test, it was not at all accurate becauseimprovement in ability to speak and respondin English was reflected more in the quality ofhow the testees spoke, rather than in the quan-tity of language they produced. Several of themore competent pupils spoke the second timein round 1 but displayed knowledge/featuresnot present in their own home languages suchas prepositions and articles, used correctly sub-ordinating and coordinating conjunctionsthey had been introduced to only in the courseof conversation, and employed a variety oftenses in their story telling. We therefore usedthis data to develop a number of assessmentlevels, or descriptive band scales, based uponthese various grammatical competencies,when evaluating the pupils’ output (a bandscale outlines a set of linguistic features andskills a pupil needs to display in order to beplaced in that category).In response to our discussion, some schoolshave begun to introduce two components intheir diagnostic test. The first is a multiple-choice comprehension test and the second anoral test based upon a set of story cards.The same test will be used for pupils at alllevels of the primary school, using the lead pro-vided by a test produced by the Human SciencesResearch Council for the same purpose. How-ever, the expected proficiency levels to enter aparticular grade or standard will be different.In conclusion, let me summarise the adviceI would give to teachers who need to designspeaking tests but who are afraid to take theplunge into this area of assessment:
• Do not be afraid to set such a test in thefirst place.
• Draw on your own materials to set a testappropriate for your group of testees.
• Keep the factor of time constant for eachtest administration.
• Give the testee the opportunity to leadonce he or she is at ease.
• Do not allow factors such as accent tocloud your perception of linguistic com-petence.
• Rely on your own instinctive judgmentwhen assigning a value to performanceon such a test.
• Try and think of this value in terms ofwords rather than marks.

References
Brumfit, C. 1984. Communicative methodology inlanguage teaching.Cambridge: Cambridge Uni-versity Press.
Madsen, H. S. 1983. Techniques in testing.NewYork: Oxford University Press.