You are here: Home » Content » Oral Language Testing at Tay Nguyen University
Content Actions

Oral Language Testing at Tay Nguyen University

Module by: Nhi Le

Summary: Assessment of oral language proficiency at Tay Nguyen University (TNU), where the author of this thesis works, has been claimed to be extremely problematic. This thesis takes a critical look at the reality of oral English language testing at this institution to point out its strengths and weaknesses and the cause(s) of the existing drawbacks or problems.

STATEMENT OF AUTHORSHIP

I certify that the minor thesis entitled ‘Oral Language Testing at Tay Nguyen University: Current Practices and Recommendations for Improvement’ and submitted in partial fulfilment of the requirements for the degree of Master of Arts in TESOL is the result of my work, except where otherwise acknowledged, and that this minor thesis or any part of the same has not been submitted for a higher degree to any other university or institution.
The research reported in this thesis was approved by Hanoi University of Foreign Studies.
Signed: Le Thi Phuong Nhi. DH Tay Nguyen
Dated:24 February 2008

table of contents

STATEMENT OF AUTHORSHIPi
table of contentsiii
ACKNOWLEDGEMENTSvii
glossaryviii
ABSTRACTx
list of figures and tablesxi
Table 1.1: The second-year students’ oral test results 2xi
Table 1.2: The third-year students’ oral test results 2xi
Figure 2.1: Continuum of Spoken Language Production 8xi
Figure 2.2: Conditions of Communicative Stress in a Task 8xi
Figure 2.3: Success of Meaning Negotiation 10xi
Figure 2.4: The Model of Test Development 15xi
Table 2.1: Level Scale of Language Proficiency Based on the Global Scale by Council of Europe 21xi
Table 2.3: Oral Test Types and Elicitation Techniques 26xi
Table 4.1: A checklist for Oral Test Development 44xi
Table 4.2: Summary of Oral Test Types Used in the Achievement Speaking Test for the Second-Year Students (School Year 2002-2003) 45xi
Table 4.3: Summary of the Students’ Oral Test Performance in the Achievement Speaking Test for the Second-Year Students 47xi
Table 4.4: Correct Answers for the Questions in the Questionnaire 56xi
Table 4.5: Teachers’ Assessment Priority Perception of Interactional and Transactional Short Turns 58xi
Table 4.6: Teachers’ Assessment Priority Perception of Transactional Long Turns 58xi
Table 4.7: Teachers’ Choice of Number of Tasks for a Speaking Test 59xi
Table 4.8: Teachers’ Choice of Elicitation Techniques for Levels of Proficiency 59xi
Table 4.9: Teachers’ Choice of Specific Test Tasks for Level of Proficiency 60xi
Table 4.10: Teachers’ Choice of Steps to Be Considered in Oral Test Design and Operationalization 60xi
Table 4.11: Teachers’ Confidence in Students’ Test Results 60xii
Table 4.12: Teachers’ Lack of Confidence in Students’ Test Results 61xii
Table 5.1: The Marking Scales for Task 1 of the Sample Term 1 Achievement Speaking Test 76xii
Table 5.2: The Marking Scales for Task 2 of Sample Term 1 Achievement Speaking Test 78xii
Table 5.3: The Marking Scales for Task 1 of Sample Term 2 Achievement Speaking Test 80xii
Table 5.4: The Marking Scales for Task 2 of Sample Term 2 Achievement Speaking Test 82xii
CHAPTER 1: INTRODUCTION1
1.1 The Problem1
1.1.1 Theoretical Perspective1
1.1.2 Practical Perspective2
1.2 Aims and Overview of the Thesis3
CHAPTER 2: LITERATURE REVIEW6
2.1 Typical Features of Spoken Language6
2.2 Communicative Approach to Testing Oral Language Ability11
2.3 Theoretical Framework for Oral Test Development14
2.3.1 Design Stage15
2.3.2 Operationalization Stage16
2.3.3 Administration Stage18
2.4 Major Considerations in Operationalization of Speaking Tests18
2.4.1 Level Scale19
2.4.2 Oral Test Types and Elicitation Techniques21
2.4.2.1 The Direct Interview Type21
2.4.2.2 The pre-arranged Information Gap Tests22
2.4.2.3 Tests Where the Learner Prepares in Advance23
2.4.2.4 Mechanical/Entirely Predictable Tests24
2.4.3 Marking Key27
2.5 Qualities of a Good Test29
2.5.1 Validity29
2.5.2 Reliability30
2.5.3 Practicality31
2.6 Summary32
CHAPTER 3: methodology33
3.1 Research Questions33
3.2 Data Collection Instruments33
3.2.1 The Checklist34
3.2.2 The Observation36
3.1.3 The Questionnaire36
3.3 Procedures37
3.4 Summary39
CHAPTER 4: RESULTS AND DISCUSSION40
4.1 Evaluation of TNU Current Development Process of Oral Language Tests 40
4.1.1 Review of TNU Current Development Process of Oral Language Tests40
4.1.2 The Observation Results 45
4.1.3 Analysis of the Results47
4.2 Evaluation of TNU staff’s Perceptions of Oral Testing55
4.2.1 Results56
4.2.2 Analysis of the Results61
4.3 Summary64
CHAPTER 5: RECOMMENDATIONS AND CONCLUSION66
5.1 Recommendations for TNU Oral Testing Practices66
5.1.1 Recommendations for TNU Development Process of Achievement Speaking Tests67
5.1.1.1 Rating/Level Scale67
5.1.1.2 Blueprint for Development of Achievement Speaking Tests at TNU70
5.1.1.3 Standardisation Meeting71
5.1.1.4 Supportive Test Taking Environment 72
5.1.1.5 Use of Test Results for Teaching Evaluation72
5.1.2 Practical Applications to the Operationalization Process of Speaking Tests for First-Year Students73
5.1.2.1 Suggested Tasks in the TLU Domain for Inclusion in Speaking Tests for First-Year Students 73
5.1.2.2 Two Sample Achievement Speaking Tests for First-Year Students 75
5.2 Conclusion82
REFERENCES85
APPENDIces89
Appendix 1: Three Achievement Speaking Tests Used at TNU89
Appendix 2: 95
Achievement Speaking Test for the Second-Year Students 95
(Term 2 – School Year 2002-2003)95
Appendix 3: The Tapescript of the Test Recorded96
Appendix 4: PHIẾU KHẢO SÁT104
109
Year 1 Year 2 Year 3109
b. Oral report    109
h. Role-play   109
i. Reading aloud   109
Year 1 Year 2 Year 3 109

ACKNOWLEDGEMENTS

I would like to show my greatest gratitude to my thesis supervisor, Ms. Nguyễn Thị Thanh Hà, who assisted and encouraged me much by providing insightful discussions, valuable comments and criticisms in the preparation and completion process of this thesis.
I would wish to send my special thanks to the organisers of this master course, Ms. Phạm Kim Ninh, Head of the Department of Post Graduate Studies of Hanoi University of Foreign Studies, Ms. Nguyễn Thái Hà, Ms. Phạm Thu Hương, the staff of this department, and the leaders of Tay Nguyen University.
I am also grateful for the permission to attend this master course given by the leaders of Tay Nguyen University, and especially to the staff members of the English Section for their assistance and participation in the research project.

glossary

This glossary is intended to give working definitions of terms used frequently in this thesis in order to help readers understand the author’s intended meaning.

Communicative stress This term means the difficulty degree of a task that a speaker has to carry out. This difficulty refers to all the conditions under which the speaker is put to perform the task.

Elicitation technique An elicitation technique involves the procedure of performing a task that inferences of a speaker’s language ability are based on.
Interactional function This term refers to one of the two functions of spoken language. A speaker producing an interactional instance of spoken language wants to make the interaction atmosphere pleasant.
Level scale or rating scaleA level or rating scale used in this thesis is a document displaying classified levels of learners’ language knowledge and what learners can do at each level.

Long turnA long turn is a string of utterances that a speaker produces

Oral test type A type of oral test refers to the way a test task requires test takers to do.
Short turnA short turn is speech of one or two utterances that a speaker produces.
Test administration Test administration involves the delivery of a set of test tasks to a group of test takers under specified conditions.
Test designTest design in this thesis refers to the production of a principled statement as a basis for writing actual tests and administering them.
Test developmentTest development is the entire process of creating and using a test. It involves test design, test operationalization and test administration.
Test operationalizationTest operationalization is the production process of actual tests. It involves developing test task specifications and test structure.
Test structure Test structure refers to the number of test tasks included in a test
Test task specificationsTest task specifications tell in detail what a test is designed to measure and how it will be tested
Transactional functionThis term refers to one of the two functions of spoken language. A speaker producing a transactional instance of spoken language means to convey his intentions and messages.

ABSTRACT

Assessment of oral language proficiency at Tay Nguyen University (TNU), where the author of this thesis works, has been claimed to be extremely problematic. This thesis takes a critical look at the reality of oral English language testing at this institution to point out its strengths and weaknesses and the cause(s) of the existing drawbacks or problems.
In order to achieve this, a study was carried out to evaluate TNU current practices and TNU staff’s perceptions of oral language testing. The methods employed in the study include: (1) an detailed analysis of the current oral test development process based on Bachman & Palmer’s theoretical framework for test development, and (2) a survey on how well the staff know about oral skill assessment.
The results of the study show that (1) the current oral testing practices at this institution are far from being consistent with the language testing theory, and (2) the staff have gained limited and insufficient knowledge of oral language testing. These findings serve as the basis for seven practical recommendations made for the improvement and standardisation of TNU current oral testing practices.
The seven recommendations are as follows. Recommendations 1,2,3,4 & 5 are made as an effort to make relevant applications which are based on Bachman & Palmer’s theoretical framework for test development. These recommendations can be considered as guidelines for developing speaking tests in general. Recommendations 6 & 7 are particularly intended for the sample operationalization of speaking tests for first-year students.

list of figures and tables

Table 1.1: The second-year students’ oral test results2
Table 1.2: The third-year students’ oral test results2
Figure 2.1: Continuum of Spoken Language Production8
Figure 2.2: Conditions of Communicative Stress in a Task8
Figure 2.3: Success of Meaning Negotiation10
Figure 2.4: The Model of Test Development15
Table 2.1: Level Scale of Language Proficiency Based on the Global Scale by Council of Europe21
Table 2.3: Oral Test Types and Elicitation Techniques26
Table 4.1: A checklist for Oral Test Development44
Table 4.2: Summary of Oral Test Types Used in the Achievement Speaking Test for the Second-Year Students (School Year 2002-2003)45
Table 4.3: Summary of the Students’ Oral Test Performance in the Achievement Speaking Test for the Second-Year Students47
Table 4.4: Correct Answers for the Questions in the Questionnaire56
Table 4.5: Teachers’ Assessment Priority Perception of Interactional and Transactional Short Turns58
Table 4.6: Teachers’ Assessment Priority Perception of Transactional Long Turns58
Table 4.7: Teachers’ Choice of Number of Tasks for a Speaking Test59
Table 4.8: Teachers’ Choice of Elicitation Techniques for Levels of Proficiency59
Table 4.9: Teachers’ Choice of Specific Test Tasks for Level of Proficiency60
Table 4.10: Teachers’ Choice of Steps to Be Considered in Oral Test Design and Operationalization60
Table 4.11: Teachers’ Confidence in Students’ Test Results60
Table 4.12: Teachers’ Lack of Confidence in Students’ Test Results61
Table 5.1: The Marking Scales for Task 1 of the Sample Term 1 Achievement Speaking Test76
Table 5.2: The Marking Scales for Task 2 of Sample Term 1 Achievement Speaking Test78
Table 5.3: The Marking Scales for Task 1 of Sample Term 2 Achievement Speaking Test80
Table 5.4: The Marking Scales for Task 2 of Sample Term 2 Achievement Speaking Test82

CHAPTER 1: INTRODUCTION

This thesis reports the results of a study carried out to investigate the current practices of oral testing at Tay Nguyen University (TNU) in order to point out the existing problems and to make some practical suggestions for improvement. This introductory chapter describes in detail the problem the thesis attempts to solve, states the objectives of the study, and provides an overview of the thesis.

1.1 The Problem

1.1.1 Theoretical Perspective

Theoretically, as has become clear through empirical studies in language testing, there has been ‘a shift from using assessment as a way to keep students in their place to using assessment as a way to help students find their place in school and in the world community of language users’ (Cohen, 1996, p. 3). In this popular tendency of treating language tests, language tests have been considered extremely helpful for both students and teachers, and even for administrators. Madsen (1983, p. 4-5) points out the importance of language testing by demonstrating that properly made tests can
‘1. help create positive attitudes towards instruction by giving students a sense of accomplishment and a feeling that the teacher’s evaluation of them matches what he has taught them.
2. help students learn the language by requiring them to study hard, emphasizing course objectives, and showing them where they need to improve.
3. help teachers and administrators by confirming progress that has been made and showing how they can best redirect their future efforts.’
Therefore, being competent in language testing, particularly in oral language testing under review in this thesis, is claimed to be crucial for language teachers to properly develop language tests. This thesis is an attempt to provide a clear discussion of how to become competent in oral language testing. An answer to this question will explicitly help to evaluate TNU current oral testing practices.

1.1.2 Practical Perspective

Apart from the above theoretical concern, this thesis also grows out of a practical consideration regarding the researcher’s work at TNU as an English teacher and assessor of students’ oral test performance. The problem identified in this thesis has taken root from the existing oral testing practices at TNU. The following are two tables of oral test results of the second-year students (School Year 2001-2002) and of the third-year students (School Year 2002-2003).
Students with Term 1 Term 2 Average
Mark 4 0% 6,5% 3,25%
Marks 5 and 6 38% 41,5% 39,75%
Mark 7 33,5% 27% 20,25%
Marks 8 and 9 28,5% 25% 26,75%
Table 1.1: The second-year students’ oral test results
Students with Term 1 Term 2 Average
Mark 4 0% 0% 0%
Marks 5 and 6 26% 39% 32,5%
Mark 7 40% 33% 36,5%
Marks 8 and 9 34% 28% 31%
Table 1.2: The third-year students’ oral test results
The two tables reveal that nearly half of the second-year students (47%) and more than half of the third-year students (67,5%) get high marks (7,8 and 9). However, in a talk with the researcher about their speaking ability, preferably their results of the former speaking tests, the majority of those students who got high marks in the tests seemed very reluctant to agree that their test results really revealed their actual ability to use English for communication. For example, they still found it hard to use English to either satisfactorily communicate their ideas or make themselves fully understood in a real instance of communication. So why did they get such high scores?
The same question was put forwards for discussion with the teachers and the assessors of speaking skill. Most of them believed that those students deserved high scores because they actually performed their tasks fluently and were able to answer the questions of the examiners. Nevertheless, when the author asked these teachers what particular speaking abilities they expected to see in the students’ test performance, what exact criteria their assessment was based on, and what the detailed procedure for test design was, they did not all provided any specific and clear answers. They could not point out speaking abilities expected, they have assessed students’ test performance intuitively, and they have constructed oral tests in their own way. Thus, as can be concluded that there seems to have never been any official detailed guidance for the construction and administration of oral tests at this institution.
Most of the staff members whom I have discussed this matter with have shared the same worry and showed interest in how to gain a scientific approach to assessing our students’ oral ability properly and fairly. In other words, we all would like to find out how to appropriately measure the students’ speaking ability and how to write useful speaking tests. This aims at helping to ensure fairness for the students, and improve and maintain the training quality of the institution.
All the concerns described above indicate an urgent need to evaluate the reality of oral language testing at TNU in the light of language testing theory. A thorough study is carried out as an attempt to help the staff give the oral testing an adequate position in their training program.

1.2 Aims and Overview of the Thesis

This thesis is carried out with the two main aims: firstly, to investigate the existing oral testing practices at TNU; and secondly, to make suggestions for improvement. Based on Bachman and Palmer’s theoretical frammework for developing language tests, the author recommends a procedure for speaking test development as an attempt to provide a profound understanding of how to properly develop an oral test. Therefore, these recommendations are hopefully used as guidelines for oral language test development not only at TNU but also at other institutions throughout Vietnam. The data used for the two purposes above are collected from two different sources: (1) a detailed analysis of current oral testing practices at TNU, in particular, the development procedure of an achievement speaking test; and (2) a questionnaire survey given out to 12 TNU teachers of English to investigate their perceptions of oral testing and thus to find out the cause(s) of the current practices.
The thesis consists of five chapters as follows:
Chapter 1 identifies the problem and provides an overview of the thesis.
Chapter 2 reviews the literature related to major issues in oral language testing such as essential features of spoken language, a theoretical framework for test development in general and development of oral tests in particular, and major qualities of a test.
Chapter 3 describes the methodology employed in the study. In order to evaluate current practices, the study involves describing the existing practices of spoken language testing at TNU, and investigating the staff’s perceptions of oral testing by delivering questionnaires to 12 staff members.
Chapter 4 presents the results of the study, and analyses the results to point out findings.
Chapter 5 makes some practical recommendations for standardisation of TNU oral testing practices, and provides a summary of the main details of the whole thesis with a conclusion ending the thesis.

CHAPTER 2: LITERATURE REVIEW

Chapter 1 presents the background information of the study. This chapter looks at main issues of oral testing. The discussion of the issues is meant to give a theoretical foundation on which to develop a framework for developing oral tests. The chapter discusses the following issues: (1) typical features of spoken language, (2) communicative approach to testing oral language ability, (3) theoretical framework for test development, (4) major considerations in construction of oral test tasks and tests, and (5) qualities of a good test.

2.1 Typical Features of Spoken Language

Spoken language had been ignored in language teaching long before it was noticed to be as essential as written language as well as other aspects of this science. From this time learners of a foreign language have been encouraged to learn how to produce spoken language forms spontaneously, not simply to utter written language sentences.
The features of spoken language reviewed here will help to specify typical and important areas of language knowledge to be involved in the process of testing speaking skill.
The most special feature of spoken language is its functions. Brown and Yule (1983) demonstrate that spoken language encompasses two functions in terms of a speaker’s intention. These two functions are defined as Interactional function and Transactional function. The former refers to the kind of spoken language speakers use to make their interaction atmosphere pleasant whereas the latter is concerned in interactions where speakers want to mainly convey their intentions and messages. Therefore, Brown and Yule (1983, p.13) assert that interactional language is listener-oriented while transactional language is message-oriented.
In interactional situations the participating speakers do not challenge each other to communicate information, and tend to end up feeling friendly and comfortable with each other. In transactional situations information transmission requires language exchanges between interlocutors to be understandable and appropriate. Obviously, ‘all foreign learners of English, who wish to learn the spoken form of the language, need to be able to express their transactional intentions’. They must know how to make clear the ideas to be communicated, even in their own mother tongue environment, yet it is easier to make themselves understood in their own language than in a new language.
Another crucial feature of spoken language is length of its production, that is the language is orally produced at length or not. Speech consisting of only one or two utterances is defined as a short turn, and that of a string of utterances is defined as a long turn by Underhill (1987, p.16). Taking short turns is of course less demanding than taking long turns. When in position of taking a transactional long turn, a speaker is immediately ‘responsible for creating a structured sequence of utterances which must help the listener(s) to create a coherent mental representation of what he is trying to say’.
As regards these two features a product of spoken language can be considered in such a continuum as interactional short turns – transactional short turns – transactional long turns. The difficulty of spoken language production ranges from the one extreme to the other extreme of the continuum, and the level of difficulty is shown in the figure 2.1 below. Clearly teaching as well as testing speaking skill should gradually follow this continuum according to learners’ level of language proficiency.
Interactional Transactional
Short turns
Long turns
Figure 2.1: Continuum of Spoken Language Production
The above figure indicates that content to be taught or assessed should be graded according to the difficulty of tasks intended for the course purposes. The degree of this difficulty is determined by communicative stress, which involves three conditions under which a speaker feel more or less comfortable in producing what he has to (Brown and Yule, 1983, p.34). The less stressful a task is, the easier it is for speakers to carry out. These three conditions are features of the context, state of knowledge of the listener and type of task shown in the figure 2.2 below.
Communicative StressState of knowledge of the listenerType of taskFeatures of the context-The listener-The situation-The language-The information-Status of knowledge-Structure of the task
Figure 2.2: Conditions of Communicative Stress in a Task
The listener refers to the relationship between the speaker and the listener, or the number of the listeners he is talking to. The situation is concerned with the speaking environment (is it familiar or unfamiliar, and private or in public?). The language relates to the listener(s)’ language proficiency in comparison with the speaker’s, and the information is what the listener wants or needs. Status of knowledge mentions the degree of familiarity of the task’s topic, and structure of the task refers to the purpose of the task or the difficulty of the task itself. This difficulty ranges from the static relationships to the abstract relationships between what is being talked about and what is going to be said. Obviously, tasks involving ‘abstract relationships are more difficult than those involving the description of static and dynamic relationships’ (Nunan, 1991, p. 48). O’Malley & Pierce (1996, p. 76) state these relationships correspond to an increase in difficulty levels. The tasks intended for the purpose(s) of teaching or testing should thus be graded according to these relationships as follows:
  1. Static relationships
Describing an object or photograph
Instructing someone to draw a diagram
Instructing someone how to assemble a piece of equipment
Describing/instructing how a number of objects are to be arranged
Giving route directions
  1. Dynamic relationships
Story-telling
Giving an eye-witness account
  1. Abstract relationships
Opinion-expressing
Justifying a course of actions
(Brown & Yule, 1983, p.109)
The difficulty of tasks additionally depends upon the number of relationships, elements, factors or characters within each task. For instance, ‘a short narrative involving a single character and only two or three events may be easier than a lengthy description covering many details and relationships’.
Linguistic KnowledgeSociocultural KnowledgeCooperative PrincipleProduction/Interpretation of Spoken LanguageFurthermore, how to ensure their production of spoken language in a new language to be appropriately interpreted is extremely demanding on the part of speakers/learners. In order to achieve this confidence, learners must first process their acquired knowledge of language and then produce utterances linguistically acceptable and socioculturally appropriate, and the utterances must conform to the cooperative principle (Celce-Murcia and Olshtain, 2000, p.168-171). This principle refers to the general rules of how to maintain the exchange flow between interlocutors, which means that ‘the speaker wants to be understood and interpreted correctly and the hearer wants to be an effective decoder of the messages he receives’. A speaker’s ideas successfully communicated are illustrated in figure 2.3 below.
Figure 2.3: Success of Meaning Negotiation
Linguistic knowledge, or Organizational knowledge (Bachman and Palmer, 1996; and Bachman, 1990), includes grammatical knowledge (ie. knowledge of vocabulary, morphology, syntax, and phonology/graphology), and textual knowledge (ie. rules of cohesion and coherence, and knowledge of rhetorical organisations). Sociocultural knowledge, or Pragmatic knowledge (Bachman and Palmer, 1996; and Bachman, 1990), is associated with ‘(1) characteristics of the individuals who take part in the communicative exchange, (2) features of the situation in which this exchange takes place, (3) the goal of the exchange, and (4) features of the communicative medium through which the exchange is carried out.
The assessment of learners’/students’ production of spoken language or their oral test performance is entirely based on the two major features of spoken language - interactional and transactional functions, and production length. Therefore, the criteria for assessment must be formed and founded on the basis of these two features, and these criteria vary according to learners’ language proficiency level or the difficulty of test tasks. In particular, the criteria for assessing learners’ interactional and transactional short turns are to focus more on learners’ or test takers’ communicative reaction and successfully negotiated ideas rather than on content, size, cohesion or coherence like in taking transactional long turns.
To sum up, the two functions and length of spoken language production are deeply associated with what to be tested in a test of oral ability, and how to ensure success of spoken language production is primarily related to how to test or assess learners’ oral ability. The next section will discusses the suitable approach to making right inferences from learners’ oral test performance.

2.2 Communicative Approach to Testing Oral Language Ability

Testing the oral ability in a language is one of the most important aspects of language testing. This ability is an extremely difficult skill to assess as Heaton (1988) and Brown & Yule (1983) suppose. Partly because of the difficulty of treating speaking tests in the same way as other more conventional tests, testing of speaking skill has generally received little attention. In a genuine speaking test, real people meet face to face, and talk to each other. Hence, it is the people and what passes between them that are important whereas the test instrument is secondary. To put it more closely, oral tests should be designed around the people involved so that they can be encouraged to talk to each other as naturally as possible.
For several decades, a new theory of language and language use has exerted a considerable influence on language teaching and potentially on language testing. For example, Hymes’s theory of communicative competence is concerned with not only language forms but also the ability to use language in socio-cultural context. Communicative competence in oral language ‘requires control of a wide range of phonological and syntactic features, vocabulary, and oral genres and the knowledge of how to use them appropriately’ (Butler et al., 2000, p.2) Although the relevance of this theory to language testing was recognized more or less immediately, it took quite long for its actual impact on practice to be felt in the development of communicative language tests. McNamara (2000, p. 16-17) characterises communicative language tests to have two features:
  1. they are performance tests, requiring assessment to be carried out when the learner or candidate is engaged in an extended act of communication;
  2. they pay attention to the social roles candidates are likely to assume in the real world settings, and offer a means of specifying the demands of such roles in detail.
The communicative approach to spoken language testing involves assessment of how language is used in real communication. Accordingly, Heaton (1988) states that most communicative language tests aim to ‘incorporate tasks which approximate as closely as possible to those facing the students in real life’. Success in actual language performance is judged in terms of the effectiveness of the communication which takes place rather than formal linguistic accuracy. Consequently, the assessment of learners’ production of spoken language or test performance should relatively concentrate more on interaction efficacy than on accuracy of language forms.
In addition, the four following characteristics of communicative language tests mentioned by Brown and Gonzo (1995, p.421-422) include a broad basis for both the design and use of language tests.
First, such tests create an ‘information gap,’ requiring test takers to process complementary information through the use of multiple sources of input. .... The second characteristic is that of task dependency,with tasks in one section of the test building upon the content of earlier sections .... Third, communicative tests can be characterized by their integration of test tasks and content within a given domain of discourse. Finally, communicative tests attempt to measure a much broader range of language abilities – including knowledge of cohesion, functions, and sociolinguistic appropriateness – than did earlier tests, which tended to focus on the formal aspects of language – grammar, vocabulary, and pronunciation.
To put it narrowly for oral testing, all speaking tests that encompass the same purpose of measuring test takers’ speaking ability in real interactions are expected to be used to assess authentic language use in context and the ability to communicate meaning, that is to include all the characteristics mentioned above. As previously discussed, the ability to communicate meaning is assured by success of meaning negotiation (figure 2.3 above) in actual acts of interaction.
Speaking tests aim at eliciting test takers’ ability of communicating ideas, and how to do this depends upon the content of test tasks or questions that fit students’ level of language proficiency. Test takers’ different levels of language proficiency can be reflected in the difficulty degree of test tasks. As reviewed in the previous section 2.1, this degree of task difficulty called communicative stress should be taken in account in the teaching and testing of speaking skill, especially on the part of teachers or test developers and assessors. A thorough understanding of the issue helps testers to make informed judgements of ‘what type of speaking activity the student would find reasonably ‘unstressful’ at a particular point in his course’ (Brown and Yule, 1983, p.107). Obviously, tasks of oral testing are to be graded mainly in accordance with the degree of communicative stress.
To sum up, the adequate approach, in my viewpoint, to assessing learners’ production of spoken language is to measure the extent to which they are able to successfully convey and achieve the intended purposes of a particular test task. In other words, learners’ performance on an oral test task should be examined in terms of communicative effectiveness or success of meaning negotiation. However, this assessment way, if a real success, is greatly related to the communicative stress under which test tasks are designed. Therefore, the next two sections will closely review more factors that must be taken into account during the construction process of speaking tests.

2.3 Theoretical Framework for Oral Test Development

The two previous sections have discussed the major features of spoken language taken into consideration in assessing production of spoken language, and have considered the communicative approach as the most adequate one to assessing spoken language production. This section describes in detail the theoretical framework for developing language tests which is intended for the following interpretation into the development of speaking tests.
Whether a test is useful or not much depends on test development process. Bachman and Palmer (1996) divide test development into three stages such as design stage, operationalization stage and administration stage.
This process of test development is illustrated in the figure 2.4
Test Development
Design stageAdministration stageOperationalization stage
-Purpose(s) of the test-Tasks in the TLU domain-Characteristics of the test takers-Construct to be measured-Plan for evaluation of test usefulness-Resources- Test task specifications- Blueprint-Giving the test-Collecting test results -Analyzing the results
Figure 2.4: The Model of Test Development
The design stage involves describing and identifying all the factors related to the test. These factors are the purpose(s) of the test as a whole, target language use tasks, test takers’ characteristics, language ability to be measured, usefulness of the test and resources.
The operationalization stage consists of two sub-stages. The former is the development of test task specifications referring to the purpose of individual test tasks, the construct to be measured, the setting, time allotment, instructions for responding to the task, characteristics of test input, and scoring method. The latter is the development of a blueprint – a description of ‘how test tasks will be organized to form actual tests’. The blueprint is therefore the structure of a test including the number of test tasks/parts and the relative importance of tasks/parts intended for the purpose(s) of the whole test, and the specifications of each test task.
The administration stage involves giving the test to a group of specific test takers, collecting test results, and analyzing the results.

2.3.1 Design Stage

The Design stage involves six activities all aiming at producing a design statement as a principled basis for the other two stages (Bachman & Palmer, 1996, p. 88). The six activities are as follows:
(1) Description of the test purpose(s). First specific inferences about language ability and capacity for language use made from the test takers’ performance are explicitly stated, and then specific decisions based on these inferences are provided.
(2) Identification and description of test tasks in the target language use (TLU) domain. A set of the TLU task types is characterized as the basis for developing actual test tasks.
(3) Description of the characteristics of the test takers. The characteristics to be believed to be particularly relevant to test development involve personal characteristics, topical knowledge, general level and profile of language ability, and predictions about test takers’ potential affective responses to the test.
(4) Definition of the construct/ability to be measured. The components of language ability to be assessed through the test task(s) are critically determined.
(5) Development of a plan for evaluating test usefulness. This plan consists of three parts as follows:
an initial consideration of the appropriate balance among the six qualities of usefulness and the setting of minimum acceptable levels for each,
the logical evaluation of usefulness, and
procedures for collecting qualitative and quantitative evidence during the administration stage.
(Bachman & Palmer, 1996, p.133-134)
(6) Identification of resources and development of a plan for their allocation and management. The resources refer to the people, material and time involved in test development. The balance between the resources available and required for test development should be taken into account in order to provide a good plan for how to allocate and manage them.

2.3.2 Operationalization Stage

The Operationalization stage need to be closely examined with the purpose of helping the concerned staff of TNU to equip themselves with a thorough understanding of the stage and then to gradually improve their practices of oral testing.
As mentioned above, this stage focuses on the structure of a test – the blueprint involving the number of test tasks/parts and specifications of each test task.
Test task specifications are described as follows:
1. The purpose of the test task.
2. The definition of the construct to be measured.
3. The characteristics of the setting of the test task
4. Time allotment.
5. Instructions for responding to the task.
6. Characteristics of input, response, and relationship between input and response.
7. Scoring method.
(Bachman & Palmer, 1996, p.172-173)
These test task specifications can be interpreted in the context of oral testing as:
  1. The purpose of the test task.
  2. Specified components of oral ability to be tested.
  3. The place where the task or language act occurs.
  4. Expected duration of task performance.
  5. Specified and understandable instructions.
  6. Areas of linguistic, pragmatic and topical knowledge adequate.
  7. Marking key
In order to well operationalize or produce test task specifications, test writers should make informed judgements of major considerations in oral test operationalization reviewed in detail in Section 2.4

2.3.3 Administration Stage

The administration of a test mainly involves three activities such as giving the test to a particular group of test takers, gathering test results and analyzing the results (Bachman & Palmer, 1996, p. 91). In particular, procedures for administering a test include preparing the testing environment, communicating the instructions, maintaining a supportive test taking environment, and collecting the test papers. These all aim at ‘guiding test takers through the process of taking the test in accordance with the procedures specified in the test blueprint.’

2.4 Major Considerations in Operationalization of Speaking Tests

The communicative approach is considered as one of the most adequate way to measure learners’ oral language ability. In order to successfully apply this assessment approach, oral test tasks intended for specific tests must be designed in terms of difficulty degree or communicative stress fitting test takers’ language proficiency level. Operationalizing a speaking test that fits the test takers’ level of language proficiency, test writers or teachers must (1) know the exact level of the test takers, then (2) choose suitable oral test types or elicitation techniques for test tasks, and finally (3) design the method of marking each task. The following three sub-sections discuss these factors, which should be taken into sound consideration during the operationalization of speaking tests.

2.4.1 Level Scale

The explicit classification of test takers’ language knowledge levels helps to grade test tasks according to the communicative stress. It is displayed on a formal document including established ‘criterion levels of oral language proficiency based on the goals and objectives of classroom instruction’ (O’Malley & Pierce, 1996, p.65). This document, called a level scale or rating scale by Underhill (1987), is a series of short descriptions of different levels of language ability in terms of test takers’ or students’ language knowledge. It describes in brief what a typical learner at each level can do so that teachers and assessors can analytically select or grade test tasks that best fit each level, and can easily decide on the score to give each student in a test.
The following is an example of a level scale with four major levels (Table 2.1 on page 17) based on the level scale introduced in ‘Hệ thống Định chuẩn Trình độ Ngoại ngữ’ của Hội đồng Châu Âu by Vũ Thị Phương Anh and Nguyễn Thị Kim Thư (2003).
Elementary Pre-intermediate Intermediate Upper-intermediate
-introduce oneself and others.-ask and answer questions about personal details such as where he/she lives, people he/she knows and things he/she has.-interact in a simple way provided that the other person talks slowly and clearly and is prepared to help.-use very simple expressions related to areas of the most immediate relevance (e.g. very basic family information, shopping, local geography, employment). -communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters.-dsecribe in simple terms aspects of his/her background, immediate environment and matters in areas of immediate need.-simply talk about familiar matters regularly encountered in work, school, leisure, etc. -use the language in most situations likely to arise when travelling in an area where the language is spoken.-make a simple connected presentation on topics which are familiar or of personal interest.-describe experiences and events, dreams, hopes and ambitions.-briefly give reasons and explanations for opinions and plans. -interact with a degree of fluency and spontaneity that makes regular interaction with native speakers without strain for either party.-make a clear and detailed presentation on a wide range of subjects.-give opinions on topical issues and explain the advantages and disadvantages of various options.
Table 2.1: Level Scale of Language Proficiency Based on the Global Scale by Council of Europe

2.4.2 Oral Test Types and Elicitation Techniques

When test takers’ proficiency level is explicitly identified on the level scale, adequate oral test types and proper elicitation techniques must be critically selected to fit the test takers’ level and the testing situation. This sub-section reviews types of oral test in combination with elicitation techniques of this kind of testing, for these two aspects have an interrelated relationship. An elicitation technique involves the procedures of performance for each test task, and a test task itself represents a test type.
Underhill (1987) classifies oral tests/test tasks into four main types: (1) the direct interview type, (2) the pre-arranged information gap tests, (3) tests where the learner prepares in advance, and (4) mechanical/entirely predictable tests. Each type requires some specific techniques to elicit test takers’ language performance named elicitation techniques. The four following sub-sections respectively summarize these four oral test types in combination with the involved elicitation techniques

2.4.2.1 The Direct Interview Type

The direct interview is the most common and authentic type of oral test; there is no script and no preparation on the test taker’s part. The assessor or interviewer, of course, has quite a careful preparation, but not so rigid as to control exactly what the test taker says. This may result in difficulty in assessing the test performance consistently and reliably. (Underhill, 1987, p. 31)
The assessors should be flexible in choosing suitable and feasible techniques to well elicit the task of this type in a specific testing situation. The most common elicitation techniques used in this case are discussion/conversation, interview, form-filling and question and answer.
Discussion/conversation is associated with interaction between two or more people in which the assessor should create the right atmosphere in a very short time so that the test taker can respond to it. The topics discussed and the directions taken by the conversation are the result of this interaction. (Underhill, 1987, p. 45)
Interview, to some extent, is quite similar to discussion/conversation, but an interview is structured. That is to say, the assessor or interviewer maintains firm control and keeps the initiative; whatever the test taker says is in more or less response to the interviewer’s questions or statements. (Underhill, 1987, p. 54-56)
Form-filling is a technique in which the test taker and interviewer work together to fill in a form or questionaire. The questions is usually related to the test taker’s personal details, professional situation or language needs. Question and answer refers to a set of disconnected questions raised by the tester. The questions are graded according to difficulty to elicit the test taker’s opinions on certain topics. This technique may involve using different question types, giving cues for question formation, and naming. (Underhill, 1987, p. 58-59)

2.4.2.2 The pre-arranged Information Gap Tests

In such tests, an information gap between two test takers, or between a test taker and the assessor, is deliberately created by the test designer. The test taker’s success and speed in bridging that gap are taken as an indication of his oral proficiency. (Underhill, 1987, p. 32)
The elicitation techniques proposed for this type of oral test are learner-learner description and re-creation, picture story and role-play.
Learner-learner description and re-creation technique requires one test taker to describe a design or construction of model building materials to another test taker who has to reconstruct the model from the description alone, without seeing the original. The technique consists of reporting description to partner, map-reading, and comparing models. (Underhill, 1987, p. 56-58)
Picture story is widely used with more advantages than disadvantages. Before performing the test task, the test taker is given a picture or a sequence of pictures to look at. Then the test taker describes the picture(s) or story freely before being asked questions related to the story. This technique includes using several similar pictures, ordering pictures to create a picture story, using live action, and vocabulary naming from pictures. (Underhill, 1987, p. 66-69)
Role-play technique involves two people, each of whom takes on a particular role in a given particular situation. A few minutes just before the test the test taker(s) is given a set of written instructions to get prepared, and then he carries out his role in the given situation. This technique can be used between an assessor and a student, and between students. (Underhill, 1987, p. 51-52)

2.4.2.3 Tests Where the Learner Prepares in Advance

Tests of this type give the test taker a sufficient amount time to prepare the task. The preparation time will range from a few minutes for a blank dialogue to several hours or days for a presentation. (Underhill, 1987, p. 33)
The underlying techniques may be oral report, reading blank dialogue, and re-telling a story.
Oral report technique requires the learner to give an oral presentation on a given topic lasting from five to ten minutes. He or she can refer to the notes, but reading aloud is strongly discouraged. The use of such aids as an overhead projector, a board or flipchart diagrams is encouraged if appropriate. At the end of the presentation, the test taker has to answer all the questions raised by the tester. This technique can be applied by making a mini-presentation with limited preparation time, and by identifying a topic of personal interest at a previous stage. (Underhill, 1987, p. 47-49)
Reading blank dialogue is used in the test context in which the learner is provided a dialogue with only one part written in and prepares the missing lines in a few minutes. The interviewer reads through the given lines and the test taker fills in the blanks aloud. (Underhill, 1987, p. 64-66)
Re-telling a story technique requires the test taker to re-tell a story in his own words after reading it. The test taker is not allowed to refer back to the written text once he has begun to re-tell it. This can be carried out by using notes, using a set text, and using an unseen text. (Underhill, 1987, p. 73-75)

2.4.2.4 Mechanical/Entirely Predictable Tests

Mechanical-type tests determine in advance what the test taker is expected to say, for there is always a single correct answer. This complete predictability makes such tests unauthentic and non-communicative. Hence, they cannot be used to measure the test taker’s oral fluency but to measure grammatical knowledge or the mechanical aspects of speech such as pronunciation, stress and intonation patterns. (Underhill, 1987, p. 33)
Tests of this type encompass such elicitation techniques as reading aloud, sentence transformation, sentence repetition, translating/interpreting, sentence completion, and sentence correction.
Reading aloud technique requires the test taker to read aloud to the tester, either a passage of text, or part of a dialogue in which the tester or another testee reads the other part. This technique may consist of reading scripted dialogue with someone else reading the other part, reading text with phonetic markers, reading sentences containing minimal pairs, spelling aloud, and reading from a table. (Underhill, 1987, p. 76-78)
Sentence transformation is the technique in which the test taker is given a stimulus sentence and is asked to orally transform it into a different grammatical pattern. This technique allows rapid testing of particular structural areas and an estimation of the test taker’s ability to correct himself. (Underhill, 1987, p. 84-85)
Sentence repetition technique is used in a test in which the test taker listens to a set of sentences or utterances, and then repeats them as accurately as possible. The technique may include repeating sentences of increasing length and repetition of sentences with specific language areas. (Underhill, 1987, p. 86-87)
Translating/interpreting technique involves the test taker’s target language translation of a short passage of a native-language familiar text. This technique may have such variations as translating in both directions, translating an unprepared passage, translating test in the language laboratory, and translating disconnected words or phrases. (Underhill, 1987, p. 79-81)
Sentence completion technique is associated with test context in which the test taker is asked to complete a series of sentences with the last few words missing from each. The technique may consist of using written tests, using gapfill to check discourse reference, text completion, using spoken cues, and completing a well-known saying. (Underhill, 1987, p. 81-83)
Sentence correction technique presents the test taker with a sentence containing an error. The test taker’s task is to identify the error and to correct it. The test taker can also be given a chance to correct his own errors. (Underhill, 1987, p. 84)
The four types of oral test/test task combined with different elicitation techniques are summarized in table 2.3 below.
Test types the direct interview type The pre-arranged information gap Tests where the learner prepares in advance Mechanical/entirely predictable tests
Elicitation techniques -Discussion/conversation-Interview-Form-filling-Question and answer -Learner-learner description and re-creation-Picture story-Role-play -Oral report-Reading blank dialogue-Re-telling a story -Reading aloud-Sentence transformation-Sentence repetition-Translating or interpreting-Sentence completion-Sentence correction
Table 2.3: Oral Test Types and Elicitation Techniques
No test type as well as no single elicitation technique is said to be the best for an oral test task or an oral test as a whole, for each of them has its own advantages and disadvantages. One elicitation technique may be suitable in a testing situation, but inappropriate in other ones. For example, reading aloud technique may be well used to measure elementary learners’ pronunciation, intonation and stress, but may be improperly used to measure intermediate learners’ speaking ability as this technique is considered to be uncommunicative. Therefore, it is advisable to combine various test types and elicitation techniques in a test of overall oral ability (Underhill, 1987, p. 37-38). This combination depends on the test purpose, and the areas of language competence and ability that intend to be seen in test takers’ performance.
Additionally, oral test tasks ‘differ with regard to whether they call for the use of static relationships, dynamic relationships, or abstract relationships’ (O’Malley & Pierce, 1996, p. 76). These relationships are mentioned in Section 2.1. The selection of oral test types for test tasks is therefore necessarily related to the difficulty degree corresponding to these relationships. Consequently, O’Malley & Pierce (1996, p. 69) propose that the test tasks selected and designed can challenge the language proficiency level(s) of test takers without frustrating them.

2.4.3 Marking Key

During the operationalization process of oral tests, the classification of test takers’ level of language proficiency is to be carefully considered with the purpose of choosing adequate test types and elicitation techniques in which the test takers’ language performance can be best shown. While designing particular test task(s), test writers or teachers should consider and decide how to mark each test task, and therefore build up a guideline helping the assessors to mark each task. This guidline is called a marking key or marking protocol by Underhill (1987).
A marking key is a set of procedures specified in advance that tells assessors what they are supposed to do step by step in the process of marking each test task/question. Test writers can make the marking quicker and more reliable by drawing up a detailed marking guide that tells the marker how to mark each question.
Underhill (1987, p. 95) identifies the aims of a marking key as follows:
- To anticipate problems that the marker is likely to face, and to suggest how to cope with them.
- To maintain the aims of the test by directing the marker’s attention to the language areas that are most important, and by giving general guidelines for dealing with unusual responses.
- To describe the purpose of each question/task.
A marking key revealing such aims thus surely helps to increase the consistency of measurement, that is reliability (See 2.5.2). In fact, oral tests are a kind calling for subjective judgement on the part of assessors, and thus do not have as a high degree of reliability as those that require objective judgement such as multiple-choice or cloze tests with either completely right or completely wrong answers. In order to help assessors achieve the highest possible degree of reliability, it is essential to provide them with a comprehensible marking key conveying the three aims identified by Underhill.
The most important factor concerned in a marking key is the distribution of marks to specific speaking sub-skills that are intended to be measured. These speaking sub-skills are named mark categories by Underhill (1987). The kind of categories defined in a test should be based on the teaching program and be cited by the way in which the teaching syllabus expresses the aims of the program (Underhill, 1987). There are two models that mark categories base on: the traditional model of language components and the more recent model of performance criteria. The former refers to the components of language proficiency (grammar, vocabulary, pronunciation and intonation, style and fluency, content, etc.) while the latter mentions the components of language performance or performance criteria (flexibility, accuracy, appropriacy, independence, hesitation, etc.).
The focus on a number of different language sub-skills or categories can also help to improve marker reliability; the assessor is supposed to give each test taker a separate mark for each category. All these separate marks are then combined to give the overall score, which is related to the process of weighting. In most oral tests or test tasks some categories are more emphasized than others according to the test purpose(s), so a weighting system is used as shown in the following example taken from Underhill (1987, p. 97).
Grammarmarked out of 10 then multiplied by 3
Vocabularymarked out of 10 then multiplied by 3
Pronunciationmarked out of 10 then multiplied by 2
Fluencymarked out of 10 then multiplied by 1
Contentmarked out of 10 then multiplied by 1
Total score 10
In sum, it can be asserted that the marking key plays a very essential role in the design of language tests in general, of oral language tests in particular to ensure the quality of reliability. It must be involved in the whole process of test development from the beginning. Language teachers or test developers should thus take the ways to mark test performance into sound consideration throughout the test construction process.
In the oral test operationalization process, consequently, language teachers or test designers must take great care over not only the selection test types and proper elicitation techniques for the intended test tasks but also the design of a marking key for each test task.

2.5 Qualities of a Good Test

The previous three sections are concerned with the techniques and procedures for developing oral language tests whereas Section 2.5 is related to the qualities of a good test, i.e. whether the test results can reveal test takers’ actual ability to orally use the language. A test used to elicit test takers’ actual language proficiency must reveal such qualities as validity, reliability and practicality.

2.5.1 Validity

Test validity generally is concerned with the degree to which a test actually measures what it is supposed to measure. In other words, it refers to the correspondence between abilities to be assessed and real indication of these abilities in a test, so a test is said to be invalid when there is no relationship between them. The concept of validity includes such detailed aspects as content validity, construct validity and predictive validity. A test is said to have content validity if its content represents a sample of the language skills, structures, etc. with which it is meant to be concerned ( Hughes, 1989). When embarking on the test construction, a test writer should first draw up a table of test specifications, describing in very clear and precise terms the particular language skills and areas to be included in the test. Not less important is the construct validity of a test. A test with construct validity is capable of measuring certain specific characteristics in accordance with a theory of language behaviour and learning. In other words, construct validity ‘examines whether the instrument permit inferences about underlying abilities.’ (Cohen, 1996). According to Hughes (1989), the word ‘construct’ refers to ‘any underlying ability or trait which is hypothesised in a theory of language ability’. This ability or trait is defined by Bachman and Palmer (1996) as ‘the domain of generalisation to which our score interpretations generalize’. Certain learning theories or constructs are believed to underlie the acquisition of abilities and skills. Another approach to test validity is to measure the degree of the agreement between results of the test and those provided by some important task at some future point.

2.5.2 Reliability

If test validity is defined as accuracy of measurement, test reliability is related to consistency of measurement. A reliable test score will be consistent across different characteristics of the testing situation. Unless test scores are relatively consistent, they cannot give any information at all about the ability measured. Another aspect of overall test reliability is rater reliability. Raters must maintain consistency in their own marking standards. This kind of reliability is called intra-marker reliability (Underhill, 1987). Or the same work marked by different raters should produce similar results, which is named inter-marker also by Underhill. If some raters rate more severely than others, the ratings of different raters are not consistent, and the scores obtained could not be considered to be reliable. Oral tests belong to the kind calling for subjective judgement on the part of the marker, so the scores awarded in an oral test cannot be believed to always have such high reliability.
It is also necessary to recognize that inconsistencies cannot be eliminated entirely. Nevertheless, it is possible to minimize the effects of the potential sources of inconsistencies under control in test design (Bachman and Palmer, 1996). Amongst factors affecting test performance, the characteristics of the test tasks are partly under control. In language test design and development, thus, it is possible to minimize variations in the test task characteristics that do not correspond to variations in target language tasks.
Test administration also involved in the concept of reliability has not been given proper attention at some universities at the present time. Administrating a test involves exam invigilators and such test conditions as classrooms, equipment, materials, exam rules and procedures dealing with test takers’ cheating.

2.5.3 Practicality

Test practicality pertains to ‘the ways in which a test will be implemented, and, to a large degree, whether it will be developed and used at all’ (Bachman & Palmer, 1996, p. 35). It concerns practical matters such as the amount of time, human and material resources available for constructing a test, administering it, marking it, and interpreting the results. If the test resources required for implementing a test exceed the resources available, the test will be impractical. Human resources are a crucial component of test construction and administration involving such individuals as test writers, scorers or raters, and test administrators as well as clerical and technical support personnel. In fact, not all institutions have sufficient staff to be in charge of all these well-defined roles. One person may be in charge of several functions. Test writers, key personnel in the process of test development, are involved not only in writing tests but also in collecting materials, editing and recording. Material resources include space (the number of classrooms, language labs needed), equipment (typewriters, computers, cassette players, overhead projectors), test materials (test booklets, answer sheets, audiotapes). Time consists of test development time and the time required to complete the parts of each stage of the test development process.
Moreover, the specific types and amounts of resources required may differ according to the design of a specific test, and available resources may vary from one situation to another. Test practicality thus can only be determined for a specific testing situation. Obviously, to determine the practicality of a given test, test developers must take into account of the resources required to develop a test, and the management and allocation of the resources available.

2.6 Summary

Chapter 2 has considered the main features of oral testing, particularly spoken language and considerations on how to elicit students’ overall speaking ability. Production of spoken language is examined in a continuum of the language functions and a success of meaning negotiation. Next, the communicative approach to assessing production of spoken language is considered one of the best ones. And Bachman & Palmer’s theoretical framework for test development is reviewed as the basis for description and evaluation of current oral testing practices at TNU. Also, this framework is used as the main foundation on which suggestions for improvement of TNU oral testing problems are based. In addition, major considerations in operationalizing speaking tests include a level scale, selection of test types, elicitation techniques, and a marking key. Finally, for a test to be valid, it must also be reliable and practical. Validity is associated with accuracy of measurement, and reliability refers to the consistency of measurement. Practicality, more or less important, concerns the ways in which the test will be implemented in a given situation, or whether the test will be used at all.

CHAPTER 3: methodology

Chapter 2 has reviewed major issues in oral language testing in order to provide an adequate understanding of its theory, which serves as the basis for investigation of the current practices at TNU. In particular, the discussion of Bachman and Palmer’s theoretical framework for developing tests presents the basis for the evaluation of oral testing practices at TNU, and for suggestions for improving its drawbacks. In order to investigate TNU current oral testing practices, the researcher (1) analysed the present process of oral test development at this institution, and (2) surveyed the staff’s perceptions of oral language testing. This chapter consists of three sections. The two research questions are first raised. The second sub-section respectively presents the data collection instruments used to carry out (1) & (2). The other sub-section describes the procedure for conducting the study.

3.1 Research Questions

The study aims at answering the two following questions.
1. What are strengths and weaknesses of the oral test development procedure at TNU?
2. What are the English teaching staff’s perceptions of oral language testing?

3.2 Data Collection Instruments

In order to answer the two research questions, the researcher collected information from two sources: (1) a situation analysis and (2) a questionnaire. Firstly, the situation analysis was carried out with a checklist based on Bachman and Palmer’s framework for test development. To ensure the reliability of the information from the situation analysis, one end-of-term speaking test was observed and tape-recorded. Secondly, the questionnaire was developed to elicit the staff’s perceptions of oral language testing. Therefore, this sub-section respectively describes these three instruments

3.2.1 The Checklist

The situation analysis with a checklist (as seen on page 31) that has been developed based on Bachman and Palmer’s framework for test development reviewed in 2.3 involved such four factors as (1) the test design stage, (2) the test operationalization stage, (3) the test administration stage, and (4) the use of test results.
1. Test Design Stage
Are the purposes of oral tests explicitly identified?  
Which kind(s) do the oral tests include? Selection PlacementDiagnosisAchievement  
Is a set of the TLU tasks presented?  
Is there an official document including detailed instructions on students’ language proficiency levels?  
Is there an official document including detailed instructions on construct or language ability to be measured?  
Are there any criteria set for evaluating test quality?  
2. Test Operationalization Stage
Are there any official guidelines on the number of test tasks to be included in a particular speaking test?  
Are the specifications of each test task provided?  
3. Test Administration
Are the assessors informed of how to mark the test tasks before the test is administered?  
Is the testing environment well prepared?  
Is a supportive test taking environment maintained?  
Are the instructions for each test task made clear to the students?  
4. Use of Test ResultsIs the information from test results used for
...grading the students?  
...evaluation of the effectiveness of instructional programmes?  
...the teachers’ modification of teaching methods and materials?  

3.2.2 The Observation

Observation of a particular achievement speaking test is expected to help collect evidence that supplements the analysis of the current oral test development process, namely the operationalization and administration stages. The observation enables the researcher to directly collect data firsthand, and the data gathered describes the observed phenomena as they take place in their natural settings (Nachmias, 1996, p.206); therefore, this kind of information is surely of great reliability.
The observation focused on the oral test type(s) with underlying elicitation techniques being used (See Appendix 2), time spent on the students’ test performance, and interaction between the assessors and test takers recorded in a tape and then transcribed (See Appendix 3).

3.1.3 The Questionnaire

In order to elicit TNU staff’s perceptions of oral language testing, questionnaires were formulated, and then delivered to the staff members. Questionnaires, in this thesis, are chosen as an adequate way to well elicit respondents’ knowledge, for the respondents are not put under pressure of time, i.e. they answer questions in their own time and at their own pace, and in an anonymous style of responding they undoubtedly feel free and comfortable to answer questions (Gillham, 2000; Nachmias, 1996).
The respondents include 12 English language teachers of TNU, who are involved in the development process of oral tests. They are at the age of 27 to 50. They all had tertiary training in language teaching in different educational institutions throughout Vietnam.
The questionnaire consists of 10 questions, 8 of which are used to elicit the teaching staff’s perceptions of oral testing. These 8 questions are developed based on the theory in oral language testing reviewed in Chapter 2. The other two concern the staff’s working experience, and their qualifications in language testing. The questions are as follows:
Questions 1 and 2 relate to the functions of spoken language and the communicative approach to measuring speaking skill.
Questions 3 and 4 relate to oral test types and elicitation techniques used in a test of speaking ability.
Question 5 relates to grading test tasks and validity.
Question 6 relates to the design and operationalization process of oral tests, and reliability.
Questions 7 and 8 relate to reliability of a test.
In general, questions 1 to 8 are close questions, and questions 9 & 10 are open-ended.
For questions 1 and 2, the respondents are required to arrange the options according to their level of priority. Question 3 requires 1 out of 4 options. Questions 4 and 5 require the respondents to tick the level(s) of language proficiency that the elicitation techniques and particular questions fit. Question 6 and question 8 allow more than one choice. For question 7, the respondents are required to choose 1 out of 3 options.
Questions 9 and 10 are meant to investigate possible sources of the differences in the subjects’ perceptions of oral language testing.

3.3 Procedures

The study meant to investigate TNU current oral language testing practices involves two steps. Firstly, the reseacher analysed the development process of speaking tests at this institution from School Year 1998-1999 (,when the researcher started working at this institution,) up to now. The current practices are evaluated in order to point out the strengths and weaknesses based on Bachman and Palmer’s theoretical framework for test development reviewed in 2.3 (Chapter 2).
To ensure the reliability of the information from the analysis, one end-of-term speaking test was observed and tape-recorded. The information from this observation is particularly meant to confirm the evaluation of the test operationalization and administration stages. The test under observation was used for the second-year students at the end of Term 2 - School Year 2002-2003 (See Appendix 2). This test was administered in the morning on June 25th, 2003 at TNU. 10 students were chosen at random to be audio-recorded. Their performance was recorded in a tape and then transcribed (See the tapescript in Appendix 3).
There were 60 test takers altogether, and they were divided into 3 groups of 20 in 3 separate rooms. However, because of the time limitation and the simultaneity of the 3 groups, only 10 of them were chosen at random for recording. Each group was conducted by 2 assessors. The students drew lots for the test task or topic to prepare for 5 to 10 minutes before performing it. The test consisted of 8 topics altogether (See Appendix 2). Both the assessors and the students were unaware of being recorded.
Secondly, the investigation of TNU current oral testing also involved a survey of the English teaching staff’s perceptions of oral skill assessment. The survey was conducted in the form of questionnaires. The questionnaires were delivered to the respondents and collected one week later. The respondents were clearly informed of the purpose of the questionnaire.
The questionnaire (See Appendix 4) is written in Vietnamese to make sure that the respondents’ different extent of familiarity of some technical terms in language testing does not affect their understanding of the questions and thus distort their responses.

3.4 Summary

Chapter 3 has raised the two research questions and described the research methods employed, particularly the subjects of the study, the data collection instruments used to serve the purposes of the study, and the way the study was conducted. The data collection instruments include a checklist for oral test development summarizing the oral test development process at TNU described, tape recordings of actual test performance used to supplement the operationalization and administration processes of achievement speaking tests, and questionnaires formulated to investigate the staff’s perceptions of oral testing. The data gathered from the study will be thoroughly analysed in the next chapter.

CHAPTER 4: RESULTS AND DISCUSSION

Chapter 3 has outlined the two research methods used to carry out the study: the situation analysis and the questionnaire survey. The situation analysis is meant to evaluate the current oral test development at TNU while the survey is aimed at investigating the staff’s understanding of oral language testing. Chapter 4 is thus divided into two sections evaluating (1) the current development process of speaking tests and (2) the staff’s perceptions of oral skill assessment.

4.1 Evaluation of TNU Current Development Process of Oral Language Tests

As previously mentioned, the analysis of TNU current development process of speaking tests and the observation of one real end-of-term speaking test are intended for the evaluation of the practices of developing speaking tests. This section (1) starts with a detailed review of the present development process of speaking tests summarized in the checklist, (2) presents the results gathered from the observation of the particular achievement speaking test, and (3) analyses the results in order to reach a conclusion of whether TNU current procedures for oral test development are consistent with the theoretical framework or not.

4.1.1 Review of TNU Current Development Process of Oral Language Tests

Information regarding the existing practices of developing English speaking tests at TNU described in this sub-section plays an essential role in the critical evaluation of these practices. TNU current oral test development process is described in relation to such four main factors as (1) the test design stage, (2) the test operationalization stage, (3) the test administration stage, and (4) the use of test results.
First of all, the oral tests used at this institution have been formally administered at the end of each term in order to measure what the students have actually achieved after one particular time of learning. Such kind of test is called a final achievement test by Hughes (1989) and McNamara (2000), progress and grading test by Bachman & Palmer (1996), course test by Davies (2000), or final or attainment test by Heaton (1990). All the teachers or test writers as well as assessors have always known that they should elicit the students’ actual ability to use the language in real communication, and especially their language knowledge and ability they are required to grasp by the end of one particular term. Therefore, oral tests at this institution are explicitly identified as achievement ones from the very start, and both the teachers and the assessors have been aware of this.
However, neither the Department nor the English Section has produced a formal document including explicit classification of students’ language proficiency levels, particular areas of language ability or construct to be assessed and sets of TLU tasks identified for all the levels. Also, they have not established any criteria for test quality evaluation.
Secondly, as regards the operationalization process, the test designers have not kept in hand the official document mentioned above with descriptions of different levels of language proficiency in terms of the students’ language knowledge, i.e. a level scale (discussed in 2.3.1) and with areas of language ability to be tested. Additionally, the teachers or test writers have not received any detailed guidelines or instructions on number of test tasks included in the test(s) as well as test task specifications. Nevertheless, they have been informed about the administration time of the test(s) in advance in order to ensure punctual test production and submission.
Therefore, the teachers or test designers have freely produced the speaking tests in their own way, and most of the oral tests conducted make use of merely one oral test type – Tests where the learner prepares in advance – and one elicitation technique – oral report – and consist of one test task/part (See Appendix 1- these three tests were used for the same class for three terms in succession). Furthermore, as shown in these three tests, none of the test task/question is attached with neither external prompts helping the students make a structured presentation nor explicit instructions quantifying language knowledge and ability needed to perform the task.
Thirdly, concerning the administration process, there has been no detailed guidance in the form of either a meeting among the group of administrators and assessors or an official document regarding the students’ language proficiency level, i.e. level scale mentioned above, and no guidelines on method(s) of marking students’ performance on each test task. In short, the assessors are never informed of or provided with these two important things before test administration.
At the end of one term the oral test administration takes place with three classes of 60 students on average. The test administration for one class is allotted half a day and each class is usually divided into two groups of about 30 students in two separate rooms. The time for test performance of each student is about 4 to 5 minutes, hence. During test administration every 5 students are called into to draw lots for test questions or tasks to make a preparation for 5 to 10 minutes. Then each student presents his/her preparation in front of two assessors. The two assessors often raise some questions related or perhaps unrelated to the student’s presentation. Sometimes the assessors do not ask any questions. The student’s final score is taken from the average of marks given by the two assessors. Meanwhile, the other students waiting for their turn are standing along the corridor and talking, that is to say a supportive testing environment is not maintained.
Last but not least, after all the students finish their performance, test results are analysed to grade the students. However, the teachers are not provided with and are not allowed to keep the list of students’ test scores. The oral test results at this institution are not thus used to either evaluate the effectiveness of the instructional programs or modify the teaching methods and materials.
TNU oral test development described above can be reviewed by means of the checklist (Table 4.1 on page 39), specially designed with the purpose of highlighting the strong and weak points of the current practices. The answer ‘Yes’ is ticked ‘’ and ‘No’ is crossed ‘×’.
1. Test Design Stage
Are the purposes of oral tests explicitly identified?
Which kind(s) do the oral tests include? Selection PlacementDiagnosisAchievement
Is a set of the TLU tasks presented? ×
Is there an official document including detailed instructions on students’ language proficiency levels? ×
Is there an official document including detailed instructions on construct or language ability to be measured? ×
Are there any criteria set for evaluating test quality? ×
2. Test Operationalization Stage
Are there any official guidelines on the number of test tasks to be included in a particular speaking test? ×
Are the specifications of each test task provided? ×
3. Test Administration
Are the assessors informed of how to mark the test tasks before the test is administered? ×
Is the testing environment well prepared? ×
Is a supportive test taking environment maintained? ×
Are the instructions for each test task made clear to the students?
Are the test tasks in use attached with any limitation of knowledge? ×
4. Use of Test ResultsIs the information from test results used for
...grading the students?
...evaluation of the effectiveness of instructional programmes? ×
...the teachers’ modification of teaching methods and materials? ×
Table 4.1: A checklist for Oral Test Development
Table 4.1 partially speaks for TNU current oral testing practices with many ‘×s’, which reveals impropriety in the speaking test development at this institution and undoubtedly indicates a big gap between practice and theory.

4.1.2 The Observation Results

Table 4.2 displays oral test types used during the administration of the end-of-term speaking test for the second-year students (Term 2 – School Year 2002-2003 – Appendix 2). The information from this table is intended for evaluation of oral test types in use in the next sub-section 4.1.3
Students
Oral test types
Direct interview Pre-arranged information gap Tests where the learner prepares in advance Mechanical /entirely predictable tests
1      
2      
3      
4      
5      
6      
7      
8      
9      
10      
Table 4.2: Summary of Oral Test Types Used in the Achievement Speaking Test for the Second-Year Students (School Year 2002-2003)
Table 4.2 indicates that only one test type was in use, yet as discussed in 2.4.2 (Chapter 2), a speaking test, namely an achievement one intended to measure overall oral proficiency, that can be believed to be valid should be a combination of various oral test types, at least two.
The following is Table 4.3 presenting elicitation technique(s) employed to elicit the 10 students’ ability during the achievement test mentioned above, their topic number or test question, duration of their test performance, their interaction with the assessors. All these details were recorded and transcribed in Appendix 3.
Students
Elicitation techniques involved in tests where the learner prepares in advance
Oral report Reading blank dialogue Retelling a story
Topic number Time(minutes) Interaction focus
1     8 5 Student’s presentation without any questions from the assessors
2     2 4.5 Student’s presentation without any questions from the assessors
3     1 5 Student’s presentation with 1 question raised by the assessor
4     8 3 Student’s presentation without any questions from the assessors
5     6 3 Student’s presentation without any questions from the assessors
6     2 2.5 Student’s presentation without any questions from the assessors
7     3 3.5 Student’s presentation without any questions from the assessors
8     5 2.5 Student’s presentation without any questions from the assessors
9     1 5 Student’s presentation with 2 questions raised by the assessor
10     4 3 Student’s presentation without any questions from the assessors
Table 4.3: Summary of the Students’ Oral Test Performance in the Achievement Speaking Test for the Second-Year Students
As can be seen in table 4.3, oral report, one of the three main elicitation techniques used to elicit test takers’ speaking ability through their performance on this kind of test – Tests where the learner prepares in advance (See 2.4.2.3, Chapter 2), was the only elicitation technique employed throughout this achievement test. Additionally, after most of the students finished their presentation, the assessors did not raise any questions except for students 3 and 9.

4.1.3 Analysis of the Results

The evaluation of TNU current oral testing practices is carried out in relation to four factors described in 4.1.1: (1) test design stage, (2) test operationalization stage, (3) test administration stage, and (4) use of test results.
  • Test Design Stage
As can be easily seen in Table 4.1, oral tests are explicitly identified as achievement ones from the very start. Obviously, clear identification of test type at the beginning of a course proves to be beneficial because the teachers can integrate the test content into the teaching program. As pointed out by Brown (1994), Heaton (1990), Hughes (1989) and Ur (1996), achievement tests should be integrated into the teaching program and related directly to the classroom lessons or units, the syllabus or curriculum. Therefore, information or indication of students’ performance on an achievement test reveals their achievement or progress at the end of a course of study (Bachman & Palmer, 1996), and an achievement test of speaking skill is of course a means of eliciting students’ progress in overall speaking ability after a course/term of study.
However, a product of this stage involving such four crucial things as students’ profile of language ability, construct/ability to be measured, sets of test tasks in the TLU domain and a plan for test quality evaluation, as described in 4.1.1, has never been produced and presented to the teachers as a principled basis or guidelines for the other two stages. This undoubtedly indicates that the first stage of oral test development at TNU is far from being consistent with the theoretical framework reviewed in 2.3.1 – Chapter 2. As a result, this big mismatch leads to the staff’s improper practices in the other two stages.
  • Test Operationalization Process
Apart from the mismatch between practice and theory at this institution mentioned above, a remarkably essential fact shown in Table 4.1 is that the Department and English Section have not provided any specific guidance, i.e. a blueprint, for speaking test construction process, namely (1) the number of test tasks to be included in a speaking test, and (2) specifications of each test task. These two factors are critically analysed respectively.
Firstly, as previously discussed, an achievement test of speaking skill is a means of eliciting students’ progress in overall speaking ability after a course of study, yet most of the achievement speaking tests in use at TNU can be asserted to fail to serve this purpose because they make use of merely one type of oral test or one test task – Tests where the learner prepares in advance (Tables 4.2) - combined with only one elicitation technique - Oral Report (Table 4.3). This is partially because no blueprint is presented. Underhill (1987) points out that an oral test rarely consists of only one elicitation technique but it is usual that it involves several techniques placed in a sequence. The reasons he provides for including more than one technique in an oral test are as follows
  1. It is more authentic to use a mix of techniques, with the learner doing different things with the language....
  2. An oral test that consists only of Question and Answer, for example, will naturally favour learners who are good at answering questions....
  3. To help improve the consistency of assessment, a change of tasks during a test can be used as an opportunity to swap interviewers and so combine multiple tasks with multiple assessment....
  4. A live test with several different parts is more flexible and can be adapted quickly to meet changing circumstances or different needs....
(Underhill, 1987, p.38)
Probably, such test tasks have been carefully discussed in class, and the students are expected to produce ‘well-prepared’ talk, even predictable questions can also be prepared in advance. Of course, ‘the task(s) on which the student has to perform may be generally familiar in form to the student, but the student cannot ‘prepare’ a written version of what he will say’ (Brown & Yule, 1983, p.120). He must prove to the assessors that in his test performance he has learned to use, not to repeat, what he has been taught. What we as examiners want to know when testing a students is not whether the students has learned what to have been taught, but whether he is able to produce an extended piece of spoken English appropriate to the communicative situation he encounters (Brown & Yule, 1983, p.120).
Obviously, this popular kind of oral test at TNU is far from being useful in measuring the students’ overall language oral proficiency, and can be said to be lacking in construct validity and reliability (See 2.5, Chapter 2).
Secondly, no specifications of particular test task(s), especially specified components of oral ability to be tested, areas of language knowledge adequate and a marking key, to some extent, results in the teachers’ or test designers’ inadequate and useless tests. It can be said that there is lack of consideration of communicative stress in the oral test construction.
As can be seen in four achievement speaking tests (See Appendices 1 & 2), all the test questions/tasks – topics- are never accompanied with any external prompts helping the students make a structured presentation, and any explicit instructions quantifying language knowledge and ability needed to perform the tasks.
It is extremely necessary for test writers to provide clear instructions helping test takers to organise a spoken presentation for test performance because students are always encouraged to produce effectively organised speech so that the listener finds it easy to catch up with what is being said (Brown & Yule, 1983, p.119).
Also, in order to write test tasks fitting students’ proficiency levels, test writers need really give explicit instructions quantifying language knowledge and ability. The quantification of performance on a particular task much depends on the grading of tasks according to cognitive difficulty (Brown & Yule, 1983, p.121). To put in another way, the same task type can be made easier or more difficult. For example, describing a room with 8 elements is apparently more difficult than a room with 5 elements. Inevitably, test designers or teachers of speaking skill should always bear in mind informed judgements of the degree of this cognitive difficulty or communicative stress (Figure 2.2, Chapter 2) during test operationalization process.
Besides, no official instructions on criteria for marking students’ test performance are presented; thus, the test writers/teachers are unaware of the importance of scoring method(s) for each test task, and they never design a marking key (See 2.4.3, Chapter 2) instructing assessors how to assess students’ performance on test tasks. As discussed in 2.4.3, in a marking key, language and skill categories are identified and awarded separate marks according to test purpose(s). As Underhill (1987, p.94) points out the aim of a marking key is ‘to save time and uncertainty by specifying in advance, as far as possible, how markers should approach the marking of each question or task’. With help of a marking key and a level scale mentioned above, assessors can mark a test more quickly and reliably, for each language or skill category is expected to be separately marked.
  • Test Administration Process
Table 4.1 and 4.3 indicate that TNU speaking test administration reveals many a shortcoming. These weak points include (1) lack of test administration standardisation, (2) lack of reliability in marking test takers’ test performance, and (3) lack of supportive testing environment.
First, before test administration there has been no official meeting - named ‘the standardisation meeting ’ by Alderson, Clapham & Wall (1995)- for discussion and agreement on how to mark each question/task among the group of assessors. Perhaps the administrators here tend to think the assessors, as language teachers, must obviously know how to fully elicit the students’ oral proficiency, so they do not need to be informed of what to do during the test. Even when the assessors can be aware of the importance of this meeting, they are unable to hold it. It is partially because the staff’s insufficient knowledge of oral testing cannot help them to design an appropriate marking key and a reasonable description of mark categories with a mark criterion.
Therefore, before test administration, a marking key and a mark criterion for mark categories are first needed from test designers, and then a considerable amount of time must be spent on discussion to reach agreement on the way to mark each question/task. Alderson, Clapham & Wall (1995, p.112) maintain, ‘although this is likely to be expensive, it is the safest way of ensuring that enough discussion will take place for all examiners to understand thoroughly the level scale and the procedures for scoring.’ All these things aim at assuring reliability of an achievement speaking test.
Second, Table 4.3 reveals that, during the students’ test performance, interaction hardly existed between the assessors and the test takers or students apart from 2 students out of 10. These two students were asked 1 or 2 questions. Moreover, the duration of these 10 students’ test performance varies 2 minutes on average. As discussed in 2.1- Chapter 2, spoken language has two functions, interactional and transactional, which are both necessarily incorporated into a speaking test. In fact, in most of the oral tests in use at TNU, namely the achievement test mentioned above, the students are expected to merely produce transactional instances of the language. Can such tests be considered to be able to measure test takers’ or students’ overall oral proficiency? The answer is surely no because they reveals no interactive communication. This also means that the assessors gave scores just on the students’ presentation, which also surely indicates a lack of validity and reliability (See 2.5, Chapter 2).
Last but not least, as regards a supportive testing environment the oral tests were almost administered in noisy rooms. Students should be put at ease before and during their performance, which can increase their confidence. Bachman & Palmer (1996), hence, demonstrate that it is crucial to maintain a supportive environment throughout the test, that is to avoid distractions due to temperature, noise, excessive movement, etc. In order to do this, test administrators and assessors are to be in control of techniques and create an atmosphere which will help each student to feel at ease (Alderson, Clapham & Wall (1995, p.116). For those students waiting for their turn should be sitting in a comfortable room, not standing along the corridor and talking so as not to affect the others’ performance.
  • Use of test results
The last factor under evaluation involves ways of how test results or students’ final scores in test performance are used. As previously described in 4.1, students’ oral test scores are used to grade them in terms of their progress or achievement after a term/course of study. This is the most popular and common purpose of all achievement tests, that is to make the final decision on students’ proficiency kept in their study record in the form of grades. Furthermore, teachers and students are really interested in receiving feedback on students’ progress which helps students ‘guide their own subsequent learning’, and helps teachers ‘modify their teaching methods and materials so as to make them more appropriate for their students’ needs, interests and capabilities’ (Bachman & Palmer, 1996, p. 98). However, TNU students’ test scores have never been used to either evaluate the effectiveness of instructional programs or make any improvement in teachers’ teaching methods and materials. In other words, oral testing at this institution has no effect on the teaching and learning of speaking skill which is named negative washback or backwash by Hughes (1989), Heaton (1988) and McNamara (2000). Information regarding inferences about students’ proficiency made from test performance can be really useful for assessing the efficiency of a teaching program as well as teachers (Bachman & Palmer, 1996, p. 98).
In conclusion, the analysis of TNU current practices of developing oral language tests reveals a number of weaknesses as follows:
There is no principled basis for oral test operationalization and administration
Oral tests in use lack construct validity and reliability
There is lack of consideration of communicative stress in oral test operationalization
There is lack of test administration standardisation
There is lack of a supportive test taking environment
These current practices are thus far from being consistent with the theoretical framework for test development.

4.2 Evaluation of TNU staff’s Perceptions of Oral Testing

As previously mentioned, the questionnaire survey was carried out on 12 members of TNU English teaching staff with the purpose of investigating their perceptions of oral language testing. The questionnaire consists of 10 questions, 8 of which help to elicit the staff’s perceptions of oral testing, and the other two indicate the staff’s working experience and their qualifications in language testing. Concerning information elicited about the staff’s perceptions, in particular, questions 1 & 2 reveal their awareness of the two functions of spoken language, questions 3 of the number of oral test types and elicitation techniques involved in a test, questions 4 & 5 of communicative stress in tasks suitable to students’ proficiency levels, and question 6 of the procedure for oral test design and operationalization. Meanwhile questions 7 & 8 elicit the teachers’ reliability degree within their own inferences about students’ oral test performance. This section (1) presents the data collected from the teachers’ responses to all the questions of the questionnaire, and (2) analyses the data to evaluate their understanding of this kind of assessment.

4.2.1 Results

This sub-section first provides the list of the correct answers to the questions and the results collected from the survey.
List of the correct answers to the questions
The correct answers for the questions of the questionnaire are provided in Table 4.4 below.
Questions Answers
1 d-f-h-e-b-c-a-g
2 i-h-f-d-g-e-b-c-a
3 b, c or d
4 Year 1: b, c, d, e, h, i Year 2: a, b, c, d, f, g, h Year 3: a, b, c, f
5 Year 1: b. e, i Year 2: a, d, gYear 3: c, f, h
6 All the options must be selected
7 Selected in the respondent’s own opinion
8 Used when option b or c of question 7 has been chosen
Table 4.4: Correct Answers for the Questions in the Questionnaire
As far as the two functions of spoken language concerned, criteria for assessing test takers’ interactional and transactional short turns, as discussed in 2.1 (Chapter 2), are more based on their communicative reaction and successfully negotiated ideas rather than on content, size, cohesion or coherence like in taking transactional long turns. Thus, the correct order for Question 1 is d-f-h-e-b-c-a-g, and the correct order for Question 2 is i-h-f-d-g-e-b-c-a. However, every two positions of the last fourth (Question 1) and of the last fifth (Question 2) positions can be exchanged, but the first fourth positions cannot be put in the last fourth (Question 1) or the last fifth (Question 2) positions. That is to say, the order d-f-h-e of question 1, and i-h-f-d of question 2 cannot be changed, and these options must be given the highest priority based on the major features of communicative language testing, that is, interaction efficacy is the assessment prerequisite not accuracy of language forms.
For Question 3, related the number of test tasks/parts included in a test of overall oral proficiency, it is necessary to make use of at least 2 elicitation techniques representing 2 oral test types as demonstrated in 2.3.2 (Chapter 2). The correct answer to this question, hence, is either b, c, or d.
Concerning grading test tasks according to degree of communicative stress reviewed in 2.1 (Chapter 2), elicitation techniques suitable and adequate for each level of proficiency in Question 4 are as follows: Year 1: b, c, d, e, h, i; Year 2: a, b, c, d, f, g, h; and Year 3: a, b, c, f. Similarly, specific tasks in Question 5 used to measure speaking ability for each level are identified as follows: Year 1: b, e, i; Year 2: a, d, g; and Year 3: c, f, h.
As reviewed earlier in 2.4 (Chapter 2), all the steps, in Question 6, must be involved in the oral test design and operationalization process. Therefore, all the options of this question are to be chosen.
The data collected from the questionnaire
Table 4.5 on page 49 presents information regarding the subjects’ own criteria for assessing test takers’ performance on interactional and transactional short turns (Question 1).
OptionsLevel of priority a b c d e f g h
1 0 0 0 6 3 1 0 2
2 0 2 0 4 1 3 0 2
3 3 2 1 1 2 1 0 2
4 1 1 6 0 2 0 0 2
5 1 3 3 0 2 0 1 2
6 4 2 1 0 1 2 0 2
7 3 2 0 1 1 5 0 0
8 0 0 1 0 0 0 11 0
Table 4.5: Teachers’ Assessment Priority Perception of Interactional and Transactional Short Turns
Most of the respondents (11/12), as shown in Table 4.5 give option (g) the lowest priority in their assessment criteria for interactional and transactional short turns.
Table 4.6 displays the respondents’ perception of priority in assessing students’ test performance on transactional long turns (Question 2).
OptionsLevel of priority a b c d e f g h i
1 0 1 0 3 2 0 0 0 6
2 3 1 0 2 1 2 0 2 1
3 0 3 1 1 4 1 0 1 1
4 1 2 5 1 1 0 0 2 0
5 1 1 2 1 1 1 1 3 1
6 3 2 1 0 2 1 0 1 2
7 3 0 2 4 0 1 0 2 0
8 1 2 0 0 1 6 0 1 1
9 0 0 1 0 0 0 11 0 0
Table 4.6: Teachers’ Assessment Priority Perception of Transactional Long Turns
Table 4.6 reveals most of the respondents (11/12) also give option (g) the lowest priority in their assessment of transactional long turns.
Table 4.7 displays all responses the respondents have chosen for Question 3, concerned with the number of test tasks/questions included in an achievement speaking test.
Number of tasks included 1 2 3 4
Number of respondents 4 4 3 1
Table 4.7: Teachers’ Choice of Number of Tasks for a Speaking Test
Table 4.7 indicates that 8 out of 12 respondents think a speaking test should make use of two or more tasks/parts.
Table 4.8 reveals the subjects’ choice of elicitation techniques for their tests of speaking (Question 4). All the elicitation techniques in the questionnaire are popularly used in tests of oral ability. The appropriate elicitation techniques for each proficiency level, as previously asserted in Table 4.4, are provided in the brackets right below each option.
OptionsLevels a(2,3) b(1,2,3) c(1,2,3) d(1) e(1) f(2,3) g(2) h(1,2) i(1)
Year 1 5 4 6 2 7 3 6 8 11
Year 2 5 5 2 8 5 11 5 6 1
Year 3 6 11 8 4 0 4 2 2 1
Table 4.8: Teachers’ Choice of Elicitation Techniques for Levels of Proficiency
As shown in table 4.8, only 2 out of 12 respondents choose (d) for year 1 and (c) for year 2, and half of them choose (g) for year 1.
Table 4.9 on page 51 displays the respondents’ selection of specific tasks used to measure test takers’ speaking ability at each proficiency level (Question 5). The particular tasks adequate to the three levels, as given in Table 4.4, are presented in the brackets next to each option. For example, a(2) – a refers to one option of the question and (2) to the second level of proficiency (Year 2).
OptionsLevels a(2) b(1) c(3) d(2) e(1) f(3) g(2) h(3) i(1)
Year 1 4 9 2 0 2 1 3 3 9
Year 2 6 3 9 3 6 1 6 8 3
Year 3 3 1 4 10 8 11 5 2 3
Table 4.9: Teachers’ Choice of Specific Test Tasks for Level of Proficiency
Table 4.9 reveals that only 2 out of 12 respondents use (e) for year 1, 3 use (d) for year 2, and 2 use (h) for year 3. In addition, up to 9 of them use (c) for year 2, and 10 use (d) for year 3.
Table 4.10 presents responses selected for Question 6, associated with the procedure for oral test design and operationalization. The correct answer, as given in Table 4.4 is selection of all the options.
Options a b c d e f
Number of respondents 8 9 10 10 6 7
Table 4.10: Teachers’ Choice of Steps to Be Considered in Oral Test Design and Operationalization
Table 4.10 shows that only half of the respondents (6/12) consider option (e) to be essential in the construction process of oral tests.
Table 4.11 reveals information regarding the respondents’ reliability degree within their own inferences about students’ oral test performance (Question 7).
Marks given on test takers’ performance Sure Not very sure Not sure
Number of respondents 2 10 0
Table 4.11: Teachers’ Confidence in Students’ Test Results
As shown in table 4.11, 10 out of 12 respondents do not believe that their marks given on the students’ oral test performance can reveal the students’ actual interactive ability.
Table 4.12 presents responses selected for Question 8, used to elicit the reason(s) why the respondents are not or not very sure of the marks they have given on the students’ oral test performance.
Options a b c d
Number of respondents 10 5 7 5
Table 4.12: Teachers’ Lack of Confidence in Students’ Test Results
A remarkable thing shown in Table 4.12 is that those who are not sure of the reliability of students’ test scores all believe that their uncertainty results from the students’ prior preparation for test tasks.
Personal details about the respondents from questions 9 and 10 provide important facts:
  • Half of the respondents have taught English for over 8 years, 5 – over 5 years, and only one – over 11 years.
  • Only one of the subjects has attended a course or a workshop on language testing in Vietnam.

4.2.2 Analysis of the Results

This sub-section analyses the data gathered from the questionnaire survey. The information revealed in Tables 4.5 to 4.12 above can be generalized as follows: (1) most of the teachers are unable to distinguish the two functions of spoken language, (2) the majority of the teachers do not have a sufficient understanding of communicative language testing, (3) quite a number of the teachers fail to recognize the difficulty level of test tasks graded according to communicative stress, (4) the teachers have an inadequate understanding of oral language testing, and (5) the speaking tests in use lack authenticity.
Firstly, most of the teachers fail to distinguish the two different functions of spoken language in their own assessment criteria as discussed in 2.1 (Chapter 2). In particular, for Questions 1 & 2 (Tables 4.5 & 4.6), 11 out of 12 subjects give the lowest priority to option (g), that is the ability to speak at length. When taking transactional long turns, students are required to show their ability to express and convey their ideas at length. Nevertheless, the teachers at TNU hardly expect to see this feature in their students’ oral test performance.
Secondly, the notion of communicative language testing is not fully grasped and soundly engraved on many of the teachers’ mind. In this preferably applied approach to language testing as mentioned in 2.2 (Chapter 2), accuracy of language forms has turned out to be given a lower priority than communicative effectiveness in assessment. As revealed in Tables 4.5 and 4.6, the majority of the teachers, in their assessment criteria, give a high priority to options (a), (b) and (c) of both Questions 1 & 2. In particular, for Question 1, option (a) is given such a high priority by 4 subjects, (b) by 5 and (c) by 7; and for Question 2, option (a) by 5, (b) by 8 and (c) by 8. This implies that on average half of TNU English teaching staff give linguistic accuracy a high priority when evaluating their students’ production of spoken language.
Thirdly, as shown in Table 4.7, for Question 3 option (a) has been selected by only 4 subjects, which means that the majority are able to realize the need to include more than one task or elicitation technique in a test of overall speaking ability as discussed in 2.3.2 (Chapter 2). However, in their selection of proper elicitation techniques or tasks for each level, they reveal their inability to recognize the difficulty level of the given elicitation techniques according to communicative stress (Figure 2.1 – Chapter 2), particularly the difficulty of the tasks themselves. To prove this, as shown in Table 4.8, only 2 out of 12 subjects have chosen (d) (the correct answer) for the first level and (c) (the correct answer) for the second level while 6 subjects have chosen (g) (the wrong answer) for the first level.
Besides, this failure is more strongly confirmed by the information gathered from Question 5. As shown in Table 4.9, only 2 out of 12 subjects have chosen (e) (the correct answer) for the first level, 3 have chosen (d) (the correct answer) for the second level, and 2 have chosen (h) (the correct answer) for the third level whereas up to 9 of them have chosen (c) (the wrong answer) for the second level, and 10 have chosen (d) (the wrong answer) for the third level. This result suggests that most of TNU English teaching staff are unable to recognize the difficulty level based on the relationship within each specified task as mentioned in 2.1 (Chapter 2).
Fourthly, as regards all the steps to be sufficiently considered in the development process of speaking tests as described in 2.4 (Chapter 2), only half of the subjects (6/12) have selected option (e) (Table 4.10), and the other options have not been chosen by all the subjects. This partly reveals the subjects’ incomplete grasp of language test development. As shown in Table 4.7 (Question 3), there is a great agreement on combining more than one kind of elicitation techniques in a test of overall speaking ability (8 subjects/12). However, in relation to the requirements in the development process of a speaking test, half of them have failed to recognize the importance of identifying the number of test tasks or elicitation techniques included in a speaking test.
Lastly, concerning the usefulness of the oral tests in use at TNU, 10 out of 12 subjects, as revealed in Table 4.11 (Question 7), do not entirely believe that their marks given on the students’ oral test performance could reveal the students’ actual speaking ability. These 10 teachers have all chosen option (a), as shown in Table 4.12, which implies that in most of the oral tests the students were informed of the topics or test questions during the class time, and they were well prepared for what they were going to speak before the real tests actually occurred. Such speaking tests can thus be said to lack authenticity, defined as ‘the degree of correspondence of the characteristics of a given language test task to a target language use task’ (Bachman & Palmer, 1996, p. 23), because they do not represent real life language use. Therefore, many of the teachers (7/10) think that the elicitation techniques commonly used in most of the tests are unable to elicit the students’ actual speaking ability (option c). In addition, half of them (5/10) have selected the other two reasons for their low confidence in their own given marks. For the first reason, perhaps they think the students could even guess and prepare what they could be asked about as in many cases the topics or test questions were given beforehand. For the second reason, they have had no specified assessment criteria or guideline to base their marking on as described in 4.1.1. As a result, they have had to ‘design’ their own criteria.
To sum up, the analysis of the data collected from the survey shows that TNU English teaching staff have obtained a limited understanding of oral skill assessment. Obviously, their limited and insufficient grasp of oral language testing may probably lead to their low confidence in their scores given on their students’ oral test ability. Also, their improper practices of developing speaking tests critically evaluated in 4.1.3 must be an inevitable consequence of their incomplete perceptions of speaking skill assessment.

4.3 Summary

Chapter 4 has described the current process of speaking test development at TNU, presented the results collected from the observation of an end-of-term speaking test and from the survey of the staff’s perceptions of oral language testing. Then the evaluation of the current oral test development process and the analysis of the survey result help to reveal that TNU staff’s superficial knowledge of oral language testing surely results in their inappropriate practices of developing oral language tests. The study has proved to be beneficial as it helps TNU staff to find out their strengths and weaknesses, and they can therefore identify room for improvement.

CHAPTER 5: RECOMMENDATIONS AND CONCLUSION

In chapter 4, the results of the study have been presented and analytically discussed in order to find out strengths and weaknesses of TNU oral language testing. The discussion indicates two main findings: first, the current practices are far from being consistent with the theoretical framework of test development, and second, TNU teaching staff have gained limited and insufficient knowledge of oral language testing. These findings serve as the basis for following recommendations regarding standardisation of TNU oral testing practices. This chapter (1) makes several practical recommendations for TNU oral testing practices, and (2) provides a conclusion ending the thesis.

5.1 Recommendations for TNU Oral Testing Practices

The findings of the study presented above imply that current oral testing at this institution really need to be improved and standardised in order to gradually increase the training quality of the institution as a whole and of the English Section in particular.
TNU staff’s lack of sufficient competence in speaking skill assessment is one of the main drawbacks resulting in their improper practices of oral test development. Since it is TNU English teaching staff who are, at this institution, the most proficient in English teaching, and directly involved in both the teaching and the testing of speaking skill, it is essential that they be aware of the need to become more competent in developing speaking tests. This thesis recommends using Bachman & Palmer’s framework for test development (See 2.3 – Chapter 2) for the context of oral ability assessment as a theoretical basis for developing speaking tests at TNU.
Based on Bachman & Palmer’s theoretical framework for test development discussed in 2.3 – Chapter 2 – and on the weaknesses of TNU current oral testing practices analysed in Chapter 4, this thesis makes seven particular recommendations, 5 of which are meant to be used to improve the whole present development process of speaking tests, and the other two of which are directly involved in the test operationalization, namely a set of TLU tasks for TNU first-year students and two sample achievement tests for first-year students. This section starts with suggestions for improving the test development process as a whole and ends with practical applications to the operationalization of speaking tests for first-years students.

5.1.1 Recommendations for TNU Development Process of Achievement Speaking Tests

The Pedagogy Department or the English Section should produce an official document including the following suggestions this thesis attempts to make as the first effort to standardise TNU current oral language testing. These recommendations for improving the current development procedure include:
  1. A rating/level scale
  2. A blueprint for development of TNU achievement speaking tests
  3. A standardisation meeting
  4. A supportive test taking environment
  5. Use of test results for teaching evaluation

5.1.1.1 Rating/Level Scale

At this institution, as described in 4.1 – Chapter 4, there has never been an official level scale in English Section training programme in general, for speaking skill in particular. Now a specified level scale is the prerequisite for oral test development, therefore. Regarding to the level scale, the training of speaking skill at TNU includes 240 contact hours explicitly distributed to 6 terms, the first 4 of which are each allotted 45 contact hours, and the rest 2 of which have 30 contact hours. At the beginning of the course the students are supposed to be false-beginners, i.e. at post-elementary level in terms of speaking skill, since most of them, from rural areas and small towns, had no chance of exposure to English, and their English learning at high school just focussed on grammar, structure, vocabulary and reading skill. After 240 contact hours of training, the freshmen are expected to reach upper-intermediate level.
This thesis suggests using the level scale introduced in 2.4.1 on page 17 for TNU speaking skill training course as well as its oral language testing. This level scale, as maintained in 4.1.3 (Chapter 4), helps teachers and examiners to identify the level that best fits their students’ proficiency level, and thus design adequate test tasks that are valid for their designated purpose. Obviously, use of this level scale helps to increase test validity.
Elementary Pre-intermediate Intermediate Upper-intermediate
-introduce oneself and others.-ask and answer questions about personal details such as where he/she lives, people he/she knows and things he/she has.-interact in a simple way provided the other person talks slowly and clearly and is prepared to help.-use very simple expressions related to areas of most immediate relevance (e.g. very basic family information, shopping, local geography, employment). -communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters.-dsecribe in simple terms aspects of his/her background, immediate environment and matters in areas of immediate need.-simply talk about familiar matters regularly encountered in work, school, leisure, etc. -use the language in most situations likely to arise when travelling in an area where the language is spoken.-make a simple connected presentation on topics which are familiar or of personal interest.-describe experiences and events, dreams, hopes and ambitions.-briefly give reasons and explanations for opinions and plans. -interact with a degree of fluency and spontaneity that makes regular interaction with native speakers without strain for either party.-make a clear and detailed presentation on a wide range of subjects.-give opinions on topical issues and explain the advantages and disadvantages of various options.

5.1.1.2 Blueprint for Development of Achievement Speaking Tests at TNU

A blueprint which has been discussed in 2.3.2 (Chapter 2) includes the number of test tasks/parts and specifications of each task. The blueprint is suggested as major guidelines for TNU staff to construct their speaking tests.
As discussed in 2.4.2 (Chapter 2), a test of speaking ability that enables assessors or examiners to elicit a test taker’s overall oral proficiency should consist of at least two tasks or elicitation techniques. Undoubtedly, a speaking test making use of two or more tasks or elicitation techniques is said to have construct validity and reliability. Regarding the speaking tests of several popular published exams, a test of overall oral ability always has two or three tasks/parts. For instance, BEC (Business English Certificates) and IELTS (International English Language Tests) have a three-task speaking section, or Let’s Talk presents speaking tests involving two tasks/parts. Thus, it would be advisable to design a blueprint for speaking tests at TNU including two tasks or elicitation techniques.
The following are the suggested components of the blueprint for an achievement oral test.
1. Test structure
1.1 Number of tasks/parts:2 tasks
Language, as discussed in 2.1 (Chapter 2), has two functions, so assessment of students’ ability to use the language orally is to involve these both functions. Furthermore, length of spoken language production is also the basis for this kind of assessment (See Figure 2.1, page 6).
Task 1:Interactional and transactional short turns
The purpose of this task is to evaluate students’ progress in taking interactional and transactional short turns.
Task 2:Transactional long turns
The purpose of this task is to evaluate students’ progress in taking transactional long turns.
1.2 Relative importance of the tasks
This importance ranges on the continuum of spoken language production (Figure 2.1) according to students’ levels of language proficiency. For instance, in a test for first-year students, Task 1 is more important than Task 2.
2. Test task specifications
The purpose of the task
The specified components of oral ability to be tested
The place where the task occurs
Specified and understandable instructions
Expected duration of task performance
Areas of linguistic, pragmatic and topical knowledge adequate
Marking key
Concerning criteria for scoring, based on the discussion on a marking key in 2.4.3 (Chapter 2), the researcher suggests combining the two models of mark categories, i.e. the traditional model and the model of performance criteria, since doing so means that linguistic accuracy is not neglected and objectives of the course or the teaching are not thus neglected either. That is to say, we as language teachers never want to neglect linguistic forms when instructing students. Therefore, this thesis recommends the marking scales adapted from PET Speaking Test by University of Cambridge Local Examinations Syndicate, representing a combination of these two model. These marking scales are used in the marking key of two sample achievement tests introduced in the next sub-section 5.1.2

5.1.1.3 Standardisation Meeting

As discussed in 4.1 – Chapter 4, before TNU oral tests are actually administered, no discussion taking place to reach agreement on how to mark each question/task among the group of assessors is proved to also affect test reliability. It would thus be strongly advisable that the English Section should potentially hold just a short standardisation meeting in order to ensure that enough discussion will take place for all examiners to understand thoroughly the level scale and the procedures for scoring. Of course, at this meeting the level scale (discussed in 5.2.3) and the marking key for each test task (discussed in 5.2.5) are really needed. As such a meeting is possible, it will help to increase test reliability.

5.1.1.4 Supportive Test Taking Environment

As mentioned in 4.1 – Chapter 4, most of the speaking tests have taken place in noisy rooms, which surely affects students’ test performance and thus reduces reliability of the tests. Therefore, in order to ensure that a test is reliable, it is crucial to maintain a supportive environment throughout the test. In particular, examiners and administrators should avoid distractions due to temperature, noise, excessive movement, and so on, and provide a comfortable room for those students waiting for their turn. In my opinion, it would be feasible for TNU to maintain such a testing environment.

5.1.1.5 Use of Test Results for Teaching Evaluation

At TNU, as previously described and analysed, students’ test scores have never been used to either determine the effectiveness of instructional programs or make any improvement in teachers’ teaching methods and materials, which reveals that TNU staff do not exploit test potential and usefulness to improve their teaching as well as their testing. This sub-section hence aims at helping the concerned staff to develop a plan for teaching evaluation based on test scores collected.
In particular, teachers should first keep a list of scores in order to evaluate students’ achievement in general, i.e. to find out whether the instruction has helped students develop this skill. Then they should specify typical problems impeding the majority of students’ performance during test administration in order to find out suitable strategies to promote effective learning, i.e. to modify teaching methods and materials.
With the test scores, as Madsen (1983, p. 5) maintains, teachers might well ask themselves whether their teaching is effective, which is particularly suggested using the following questions:
‘1. Are my lessons on the right levels? Or am I aiming my instruction too low or too high?
2. Am I teaching some skills effectively but others less effectively?
3. What areas do we need more work on? Which points need reviewing?
4. Should I spend more (or less) time on this material with next year’s students?’
And test administration can ‘provide insights into ways that we can improve the evaluation process itself.’
‘1.Were the test instructions clear?
2. Did the test cause unnecessary anxiety or resentment?
3. Did the test results reflect accurately how my students have been responding in class and in their assigned work?’

5.1.2 Practical Applications to the Operationalization Process of Speaking Tests for First-Year Students

The following two recommendations hopefully help the test writers to visualize how to write a speaking test for TNU language students in general and for first-year students in particular.

5.1.2.1 Suggested Tasks in the TLU Domain for Inclusion in Speaking Tests for First-Year Students

One of the important steps in test development, as described in 2.3.1 (Chapter 2), is identification of tasks in the TLU domain for selection of actual test tasks included in a test. TLU tasks must be selected based on the teaching content, yet at TNU each teacher in charge of speaking skill training has used their own teaching materials. No speaking skill syllabus or teaching material has been officially approved; a syllabus for this skill is now under development and revision. Therefore, TLU tasks can not be typically chosen from all the materials unofficially in use, and all the following suggested TLU tasks, adapted from a course book named ‘English for International Communication’ by Richards (2002), are intended to partially help the author to design two sample speaking tests for first-year students or for elementary and post-elementary levels in the level scale suggested in 5.1.1.1. TLU tasks are suggested as follows:
  • Introducing oneself or someone
  • Exchanging personal information
  • Describing school and house
  • Talking about families and family members
  • Describing family life
  • Talking about daily activities
  • Talking about likes and dislikes
  • Buying and selling things, Talking about prices, ordering a meal
  • Asking about and describing locations of places
  • Asking about and describing people’s appearance
  • Asking about and describing objects
  • Making invitations and excuses, accepting and refusing invitations
  • Talking abilities
  • Talking about past experiences and events
  • Making comparisons
  • Asking for and giving advice
  • Asking for and giving suggestions
  • Taking and leaving messages
  • Describing changes
  • Talking about plans for the future

5.1.2.2 Two Sample Achievement Speaking Tests for First-Year Students

The two achievement speaking tests designed below are based on the blueprint suggested in 5.1.1.2
Sample achievement speaking test for first-year students – Term 1
Task 1: Conversation between the assessor and each student about personal information.
Purpose of the task:
to assess the students’ ability to interact in typical daily situations
Specified components of speaking ability to be tested:
The ability to talk about themselves and use social language in common interactions
The place where the task occurs:
In the classroom
Expected duration of task performance:
About 2 minutes
Specific and understandable instructions:
Each student starts the dialogue with the assessors. The assessors ask each student questions about himself/herself and about his/her family.
Areas of linguistic, pragmatic and topical knowledge adequate
Simple and common vocabulary and grammar
Simple functions such as greeting, agreeing or disagreeing, and easy description.
About oneself and his/her family
Marking key
Students are assessed on their own performance according to the criteria in the marking scales in the table 5.1
MARKING SCALES – TASK 1 (6 marks out of 10)
Marks Fluency Accuracy and Appropriacy of Language Pronunciation Task Achievement
6 Occasional hesitations, but not such as to impede communication Meaning is conveyed despite noticeable structural inaccuracies, lack of vocabulary Generally easy to understand despite L1 accent Tasks dealt with adequately
5 Hesitation often demands unreasonable patience of listener. Meaning occasionally obscured by structural inaccuracies and limited vocabulary L1 interference occasionally causes difficulty in understanding Limited ability to deal with tasks
4-3 Speech very disconnected and difficult to follow Frequently incomprehensible because of limited vocabulary and numerous structural errors Frequently impossible to understand Ineffective handling of tasks
2-1 No connected speech. Incomprehensible because of insufficient vocabulary and gross structural errors Impossible to understand Unable to deal with tasks
Table 5.1: The Marking Scales for Task 1 of the Sample Term 1 Achievement Speaking Test
Task 2: Describe your normal day
Purpose of the task:
to assess the students’ ability to use English to take a bit transactional long turns, ie. communicate some information.
Specified components of speaking ability to be tested:
The ability to make a description through a short oral presentation.
The place where the task occurs:
In the classroom
Expected duration of task performance:
About 3 minutes
Specific and understandable instructions:
The student tells the assessors about the main activities you normally do during the day. Your talk is about 100 words or less.
Areas of linguistic, pragmatic and topical knowledge adequate
Simple and common vocabulary and grammatical structures
Functions such as starting and closing a presentation
About oneself and common daily activities
Marking key
Students are assessed on their own performance according to the criteria in the marking scales in the table 5.2 on page 68
MARKING SCALES – TASK 2 (4 marks out of 10)
Marks Fluency Accuracy and Appropriacy of Language Pronunciation Task Achievement
4 Occasional hesitations, but not such as to impede communication Meaning is conveyed despite noticeable structural inaccuracies, lack of vocabulary Generally easy to understand despite L1 accent Tasks dealt with adequately
3 Hesitation often demands unreasonable patience of listener. Meaning occasionally obscured by structural inaccuracies and limited vocabulary L1 interference occasionally causes difficulty in understanding Limited ability to deal with tasks
2 Speech very disconnected and difficult to follow Frequently incomprehensible because of limited vocabulary and numerous structural errors Frequently impossible to understand Ineffective handling of tasks
1 No connected speech. Incomprehensible because of insufficient vocabulary and gross structural errors Impossible to understand Unable to deal with tasks
Table 5.2: The Marking Scales for Task 2 of Sample Term 1 Achievement Speaking Test
Sample achievement speaking test for first-year students – Term 2
Task 1: The student starts the conversation to make the acquaintance of the assessor(s) at a party and then makes an invitation or an offer.
Purpose of the task:
to assess the students’ ability to interact in usual daily situations
Specified components of speaking ability to be tested:
The ability to introduce themselves and to use social language in common interactions
The place where the task occurs:
At a party
Expected duration of task performance:
About 2 minutes
Specific and understandable instructions:
Start the dialogue and make the acquaintance of the assessor(s). Then invite the assessor(s) to have something or to do something.
Areas of linguistic, pragmatic and topical knowledge adequate
Simple and common vocabulary and grammar
Functions such as greeting, addressing, making someone’s acquaintance and making invitations.
About oneself and normal daily meetings
Marking key
Students are assessed on their own performance according to the criteria in the marking scales in the table 5.3 below
MARKING SCALES – TASK 1 (5 marks out of 10)
Marks Fluency Accuracy and Appropriacy of Language Pronunciation Task Achievement
5 Occasional hesitations, but not such as to impede communication Meaning is conveyed despite noticeable structural inaccuracies, lack of vocabulary Generally easy to understand despite L1 accent Tasks dealt with adequately
4 Hesitation often demands unreasonable patience of listener. Meaning occasionally obscured by structural inaccuracies and limited vocabulary L1 interference occasionally causes difficulty in understanding Limited ability to deal with tasks
3 Speech very disconnected and difficult to follow Frequently incomprehensible because of limited vocabulary and numerous structural errors Frequently impossible to understand Ineffective handling of tasks
2-1 No connected speech. Incomprehensible because of insufficient vocabulary and gross structural errors Impossible to understand Unable to deal with tasks
Table 5.3: The Marking Scales for Task 1 of Sample Term 2 Achievement Speaking Test
Task 2: Talking about your next summer vacation
Purpose of the task:
to assess the students’ ability to use English to take a bit transactional long turns, ie. to communicate some information.
Specified components of speaking ability to be tested:
The ability to make an oral presentation on their future intentions
The place where the task occurs:
In the classroom
Expected duration of task performance:
About 4 minutes
Specific and understandable instructions:
Tell the assessors about your next summer vacation such as where to go, what to do, why you want to do so and how long to stay there. Your talk is about 150 words or less.
Areas of linguistic, pragmatic and topical knowledge adequate
Simple and common vocabulary and grammatical structures
Functions such as starting and closing a presentation
About oneself, future plans and hobbies
Marking key
Students are assessed on their own performance according to the criteria in the marking scales in the table 5.4 on page 71
MARKING SCALES – TASK 2 (5 marks out of 10)
Marks Fluency Accuracy and Appropriacy of Language Pronunciation Task Achievement
5 Occasional hesitations, but not such as to impede communication Meaning is conveyed despite noticeable structural inaccuracies, lack of vocabulary Generally easy to understand despite L1 accent Tasks dealt with adequately
4 Hesitation often demands unreasonable patience of listener. Meaning occasionally obscured by structural inaccuracies and limited vocabulary L1 interference occasionally causes difficulty in understanding Limited ability to deal with tasks
3 Speech very disconnected and difficult to follow Frequently incomprehensible because of limited vocabulary and numerous structural errors Frequently impossible to understand Ineffective handling of tasks
2-1 No connected speech. Incomprehensible because of insufficient vocabulary and gross structural errors Impossible to understand Unable to deal with tasks
Table 5.4: The Marking Scales for Task 2 of Sample Term 2 Achievement Speaking Test

5.2 Conclusion

Current oral language testing practices at TNU have been claimed to be very problematic, so this study is intentionally carried out to investigate the present practices. In reality, oral testing at this institution is far from being consistent with the theory in language testing. This finding is based on the result of the study which is aimed at evaluating the practices and the staff’s perceptions of oral testing.
The detailed review of the existing practices provides the basis for the analytical evaluation in order to identify the strengths and weaknesses. Also, a questionnaire survey helps to elicit the staff’s understanding of oral testing – the cause of the current practices.
The analysis of the study results shows that the present problem stems from the staff’s insufficient knowledge of oral testing. All the tests of speaking ability were inappropriately constructed and inadequately administered.
As a result, the findings of the study help to justify the claim and several practical recommendations are proposed in terms of the procedures and activities involved in oral test development.
Obviously, the study as a whole can be considered significant as it provides two main practical contributions towards TNU context of language testing as follows.
Firstly, the study has applied the theory of language testing, a science, to TNU language teaching with the purpose of ensuring and increasing the training effectiveness, namely evaluating and promoting professionalism in language training at TNU. In particular, the theory reviewed in the study surely provides a reasonable foundation on which TNU staff’s testing of oral proficiency can be based; thus, it is hoped that the review will assist the staff in better understanding the testing field.
Secondly, the study has made seven practical recommendations for TNU staff’s development process of speaking tests. Five of the recommendations are concerned with relevant applications in relation to the theoretical considerations in test development process as a whole, which is aimed at providing the concerned staff the guidelines for developing oral tests. The other two recommendations as a case study are related to the operationalization of a particular speaking test.
The findings of the study also help to make suggestions for further research. The fact is that no specified syllabus is designed for speaking ability teaching at TNU and the same circumstance for the other three subjects – listening, reading and writing. Therefore, testing of the other abilities is really problematic, and this kind of research is necessary to increase the quality of the training program of the English Section as well as of the institution as a whole. Areas of research should be concerned with development of tests of the other abilities or skills, and development of syllabuses for teaching of all the four skills or abilities.
In conclusion, the study will hopefully be taken in account during the process of oral test development of TNU staff – test designers or teachers, assessors or examiners as well as test administrators. However, one limitation of this thesis is that the researcher has been unable to find out the method of quantifying the results gathered from the survey of the staff’s perceptions of oral language testing.

REFERENCES

Alderson, J. C., Clapham, C. & Wall, C. (1995). Language test construction and evaluation. Cambridge: Cambridge University Press.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
Bachman, L. F & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.
BEC – Business English Certificates Handbook (Revised syllabus). University of Cambridge Local Examinations Syndicate.
Brown, H. D. (1994). Principles of language learning and teaching. Englewood Cliffs, NJ: Prentice Hall Regents.
Brown, H. D. & Gonzo, S. (1995). Readings on second language acquisition. Englewood Cliffs, NJ: Prentice Hall.
Brown, G. & Yule, G. (1983). Teaching the spoken language. Cambridge: Cambridge University Press.
Butler, F. A. et al (2000). TOEFL 2000 speaking framework: A working paper. (TOEFL Monograph Series Report No. 20). Princeton, NJ: Educational Testing Service.
Catt, C. (2003). IELTS Speaking: Preparation and practice. London: Longman.
Celce-Murcia, M. & Olshtain, E. (2000). Discourse and context in language teaching. Cambridge: Cambridge University Press.
Cohen, A. D. (1996). Assessing language ability in the classroom. Massachusetts: Heinle & Heinle Publishers.
Davies, P. (2000). Success in English teaching. Oxford: Oxford University Press.
Gillham, B. (2000). Developing a questionnaire. London and New York: Continuum.
Heaton, J. B. (1988). Writing English language tests. New York: Longman.
Heaton, J. B. (1990). Classroom testing. New York: Longman.
Hughes, A. (1989). Testing for language teachers. Cambridge: Cambridge University Press.
Madsen, H. S. (1983). Techniques in testing. Oxford: Oxford University Press.
Metre, D. V. (2003). Let’s talk – Testing packet. Cambridge: Cambridge University Press.
McNamara, T. (2000). Language testing. Oxford: Oxford University Press.
Nachmias, C. F. & Nachmias, D. (1996). Research methods in social sciences. New York: Oxford University Press.
Nunan, D. (1991). Language teaching methodology: A textbook for teachers. Englewood Cliffs, NJ: Prentice Hall.
O’Malley, J. M. & Pierce, L. V. (1996). Authentic assessment for English language teachers. Addison-Wesley.
PET Speaking Tests by University of Cambridge Local Examinations Syndicate.
Richards, J. C. (2002). English for international communication. Cambridge: Cambridge University Press.
Underhill, N. (1987). Testing spoken language: A handbook of oral testing techniques. Cambridge: Cambridge University Press.
Ur, P. (1996). A course in language teaching. Cambridge: Cambridge University Press.
Vu Thi Phuong Anh & Nguyen Thi Kim Thu (2003). He thong dinh chuan trinh do ngoai ngu cua Hoi dong Chau Au. In Hochiminh City University of Social Sciences and Humanities Workshop Record: Khoa hoc Xa hoi va Nhan van trong boi canh hoi nhap Quoc te (p. 270-277). Hochiminh City: Hochiminh City University of Social Sciences and Humanities.

APPENDIces

Appendix 1: Three Achievement Speaking Tests Used at TNU

TEST 1: ENGLISH SPEAKING TEST
Class: English K2000 – Term 1
Students are supposed to give a presentation about one of the following topics, then answer examiners’ two or three additional questions.
  1. Please introduce yourself. We would like to know about your birthplace, your age, and your family. How does your family help you in your study?
  2. Could you tell us about your daily activities and then your freetime activities? What are your likes and dislikes?
  3. Could you tell us about your new class and your new school? How did you feel when you first arrive here?
  4. Would you please describe the appearance of a special friend in your class? Why is he/she special to you?
  5. Who is the monitor of your class? What’s he like? What do you think of him as a monitor?
  6. Please describe your mother. Do you look like her? What kind of clothes does she often wear? Do you have the same way of dressing?
  7. Please describe your father’s character. What is his job? Do you often talk with him about your study and your friends?
  8. Where are you from? Could you tell us about some of the interesting places in your hometown?
  9. Please talk about your hometown. We would like to know about its location, the weather and the environment there.
  10. Could you tell us about life of people in your hometown? (population, their main jobs, their standard of living, their character...)
  11. What do you remember most about your childhood? Did you have a happy child hood?
  12. You are now a first-year student. Is there any difference between your life now and life when you were a highschool pupil?
  13. When do you think Christmas season begins? Could you describe the atmosphere around when Christmas is coming?
  14. Who is the most famous person at Christmas? Can you describe him? What kind of gift would you like him to give you?
  15. What did you do last Christmas? (Before, on Christmas Eve, on Christmas Day.) How did you feel?
  16. Lunar New Year Holiday is coming. How do you feel? What do you like most about it?
  17. What do you often do before the Lunar New Year Holiday? How do you feel at the very moment when the old year is out and the new year is in? what do you often do then?
  18. Could you tell us about some of the Vietnamese customs concerning the Tet Holiday?
  19. Are you superstitious? Could you give us some examples of your own superstition or your family superstition or the superstition of the Vietnamese people during the Tet Holiday?
  20. What do you often do on the three days of Tet? How do you feel on these days?
TEST 2: ENGLISH SPEAKING TEST
Class: English K2000 – Term 2
  1. Do you often cook the meals for your family? Why? Why not? Do you think that knowledge of cooking is essential? Why so? Why not?
  2. Some people say that cooking is the duty of women and girls only, men and boys don’t need to learn how to cook. Do you agree with them? Why? Why not?
  3. ‘I am the only child of my parents so though I’m a girl, I don’t have to do any housework. I just have to study. In my freetime, I can go out with my friends, listen to music or watch TV. Everything in the house is done by my mother because she doesn’t go to work,’ said Mai, a 19-year-old student.
What do you think about Mai’s behaviour at home and her duty to her family?
  1. Some people say that cooking is a waste of time because fast food is available now, and that we should save time to do other necessary things. What do you think about this idea? Completely agree? Partially agree or disagree? Why?
  2. Read the dialogue below and answer the questions given.
A: I’ve been looking at your brooch. It’s very unusual. Where did you get it?
B: I got it in Malaysia.
A: Oh, did you? How long were you there? By the way, I’m John Gooch,...
B: I’m Sylvia Martin. I was there for three years actually.
A: Really? That must have been a wonderful experience. Wht did you enjoy most?
Questions:
1. Do you think the two people in the conversation have known each other before? How do you know that?
2. How does John Gooch start the conversation? What is the common way to start a conversation?
  1. Many students have meals in the school canteen or in a café near their school or where they are living, but there are some students who cook for themselves. Which do you think is better?
  2. Read the dialogue below and answer the questions given.
A: May I introduce myself? I’m Robert Munns.
B: How do you do? I’m Tina Morley.
A: How do you do?
Questions:
1. What is the relation between Robert Munns and Tina Morley? How do you know that?
2. How do Robert and Tina introduce themselves to each other? What other information should people give when introducing each other?
  1. In the exam room, what do you do if you do not know or are not sure of the answer? If your friends have different choice or answer to a question from yours, will you change your mind?
  2. As a student of English, what English subjects or skills are you learning? Which do you find the most difficult? Do you know the reason why? What do you think you have to do to learn that subject or skill better?
  3. Read the dialogue below and answer the questions given.
A: Mr. Granger, I’d like you to meet Nick Thomas, from our Boston office.
B: How do you do, Nick.
C: Please to meet you, Mr. Granger.
B: Please call me Philip.
Questions:
1. What information should we mention when introducing someone to another?
2. what is the correct order when introducingsomeone to another? (In terms of age, sex, and position in society)
  1. What is the role of grammar in learning English? What is its relation to the other skills?
  2. Can you explain why a student cannot express himself or herself or be understood by others although his/her knowledge of grammar is good? He/She can do grammar exercises well and his/her writing skill is rather good.
  3. Are you worried or nervous when examinations are coming? What are you worried about? In order not to be worried about taking exams, what do you think you have to do during the semester?
  4. Should you help your friends in the exam room? Why? Why not? How should you help your friend(s)?
  5. What time do you go to bed every day? Do you go to bed later than usaul when examinations are coming? Some students stay up too late or even the whole night before the exam. Do you think that is a good way of learning? Why? Why not?
  6. Some students often feel sleepy when they are learning hard for the exams. How about you? What do you do if you feel sleepy when learning, going to bed or trying to do something to be awake? Why do you do so?
  7. Do you think that examinations are important in the process of teaching and learning? Why? Why not?
  8. Have you ever failed in your exams? If yes, how did you feel and what did you do after that to improve the situation? If not, how do you think you will feel? What will you do after that?
  9. If your friends or younger brothers or sisters ask you to give them some advice to prepare well for examinations, what is your advice?
  10. Talk about your summer holiday this year. What are you going to do after taking the exams?
TEST 3: ENGLISH SPEAKING TEST
Class: English K2000 – Term 3
  1. What job would you like to have after you graduate from university? State the reasons why you like that job.
  2. Between the job of a teacher of English and a tourist guide, which one would you prefer? State the reason for your choice.
  3. Do you want to work as a secretary in an office? Why? Why not?
  4. Between the job of a secretary in a big company and an air-hostess, which one would you prefer? State the reasons for your choice.
  5. What kindds of job do you think a student of English can have? State what they will use English for.
  6. Many people say reading is the best way of widening our knowledge; do you agree with them? Why? Why not?
  7. What do you think a student of English should read? Let us know your habit of reading (what you read? When? How often?, what you think, ect.)
  8. Do you agree that the pleasures of reading are varied according to age, personality, jobs, ect. Give examples to illustrate your idea.
  9. Have you ever found some money in a library book? If yes, what did you do with that sum of money? If no, what would you do if you found 100,000 VND in a library book?
  10. What would you do and how would you feel if you were invited to the rector’s party?
  11. If you could make three wishes, what would they be?
  12. Read the dialogue below and state if you agree with Peter’s idea: ‘digging garden is not a jod for a University graduate,’
Peter: Why don’t you get a decent job for a change?
Dick: But I like my job.
Peter: Look, digging garden is not a job for a University graduate.
Dick:But the money’s not bad and there’s plenty of fresh air.
Peter:If I were you, I’d go on some kind of course – teaching, accountancy.
Dick:Accountancy? Anything but that. It’s too boring.
Peter:Come on, you really must think of the future. Why don’t you just write a few application froms?
Dick:I’ll tell you what. I’d really like to be a doctor.
Peter:Well, you should think very seriously about that. It means a lot of study, and then working all sorts of hours.
Dick:Yes, may be. But the idea appeals to me.
Peter:Well then, you ought to get more information about it as soon as possible.
  1. Read the dialogue below and then paraphrase it in your own words, mentioning the relationship between them, where they are and what they are talking about, and so on.
Paola:You must take some rest. You’ve been working too much hard.
Mary:But how can I? The deadline is Friday.
Paola:Come on, couldn’t you take the afternoon off?
Mary: Well, if you really think so.
Paola:I really think you should. We can manage without you.
  1. Your friend tells you he/she met a wonderful woman/man yesterday and is getting married next week. What do you think and what will you say about his/her sudden decision?
  2. You have got tickets for a film. At the last moment your girlfriend/boyfriend rings up and says she/he has a headache and can’t come. What will you do and say to him/her?
  3. In your daily life, from whom do you often have to ask for permission? What permission you may want to ask for from your teachers? If you have a dental appointment and you need tomorrow off, what do you say to your teacher?
  4. In what situations do people often make comlaints? Have you ever made a complaint? What was it about?
  5. What sentences can you say when you want to ask someone for a lift to the station?
  6. Read the dialogue below and state if you agree with Peter’s idea about the work of a doctor.
Peter: Why don’t you get a decent job for a change?
Dick: But I like my job.
Peter: Look, digging garden is not a job for a University graduate.
Dick:But the money’s not bad and there’s plenty of fresh air.
Peter:If I were you, I’d go on some kind of course – teaching, accountancy.
Dick:Accountancy? Anything but that. It’s too boring.
Peter:Come on, you really must think of the future. Why don’t you just write a few application froms?
Dick:I’ll tell you what. I’d really like to be a doctor.
Peter:Well, you should think very seriously about that. It means a lot of study, and then working all sorts of hours.
Dick:Yes, may be. But the idea appeals to me.
Peter:Well then, you ought to get more information about it as soon as possible.
  1. Have you ever got a letter from a person you don’t know? If yes, what did you do with it? If no, imagine what you would do if you got a love letter from a person you didn’t know?

Appendix 2:

Achievement Speaking Test for the Second-Year Students

(Term 2 – School Year 2002-2003)

ĐỀ THI NÓI ANH VĂN 4 - LỚP ANH VĂN K2001
Thí sinh bốc thăm và thuyết trình về 1 trong các đề tài sau: (trong vòng 5 phút)
  1. What do you think of the importance of money? Is it always wonderful to have a lot of money?
  2. Your opinions about the size of a family? How many children would you like to have? Why?
  3. Are there any advantages/disadvantages of living in a multi- generation family? Justify your ideas.
  4. If you were born again would you like to have your sex changed? Why/ Why not?
  5. What do you think of the teaching career? Why did you choose teaching as your future career?
  6. What is your favourite subject? Why do you like it?
  7. In your opinion, is examination necessary or should it be given up? Why?
  8. In your opinion, what are the roles of a woman in the modern society?

Appendix 3:The Tapescript of the Test Recorded

Oral Test Performance of 10 Second-Year students
Student 1:
Good morning everyone. Today I’m in front of you to tell you something about roles of a woman. People said woman are...is heart of the world, so there’s no woman the world will not exist. In my opinion, woman is always play an important role in our life, especially in modern life. First, woman help and teach the children. Although the scientists can make a child from test-tubes, but I think the role of a woman can be replaced. Woman... A child is bad or good depend much on the way the mother teach you. If you are a bad mother, your children is not good. And if you are a good mother, your children may be good at other people. .... He will do... He will do his ... well. And the second, woman is always good at doing the housework. It’s difficult for men to do all things in his house. But I think it’s very easy for a woman to cook some meals, to sweep her house and to take ... of people in her family. ... She not only know how to do it but also know how to do it well. She makes her house more ... and neatly. She takes care of whole her family with good meals. And maybe there is a woman in the house, the house.... the family may be better. Third, in our society I think woman has the same position as men. She can do anything her husband can do. For example, go to university and her profession or and become independent. She shares money to improve her... her family life. She helps her husband with his work. She share all people in work and in .... with her husband. She ... she.... People often say that behind a successful man is a good woman. And I think it’s difficult for men to successful without having the help of his wife. These are the reasons why I said that women are always play an important role in our life, especially in modern life. Thank you for your listening.
Student 2:
Today I would like to present my topic – the size of a family. If anyone ask me how many children I would like to have when I ... I will say just one or two children. Why I say so, for I see that small... small family has many advantage than disadvantage. Firstly, having many children I can support ... support all needs of my children. Although in our society to earn much money isn’t easy. If I have... having many children I must buy food, clothes ... for them, epecially when they grow up, I must send them to school. School fee for the learning... the learning is... isn’t easy. Why I have... if I have one or two children, it is not problem. Secondly, if you are parents in the future you should understand their... the feelings of your children. If you have many children you must do hard to earn money. You have no time to... have no time to share the feeling. It is difficult for you to understand... understand them and... and you don’t know the way to teach them become... become good preson. Finally, I think... I think in family have many children it is also having many... having many noise. And children are always playing together. For example, your family has just one TV... Girls want to watch music programme while boys want to watch football programme. They may quarrel together to watch. So... although advantage I think ... I want to... I just want to have one or two children when I married. That’s all.
Student 3:
Now I want to tell you about my thinking of money. Nothing is more powerful than money, so money plays an important part in our life. First, if we have money we can buy everything such as we can buy clothes to make more... to make us more beautiful, more confident when going out or standing before a crowd. We can buy nutritious food to improve... to improve our life. When we are ill we can buy medicine, go to hospital. Secondly, with money we can improve our spirital activities easily. For example, we can buy television... to relax a hard day. With money we can travel... in our... in our summer holiday. Further, we use money not only to meet the basic needs but also... but also to pay... to pay our... investment for education which helps us to... how... to know culturally better. Nowadays many schools, many hospitals being built, but thousands of people are not able to bo to school and... and... and quiting... because their parents have no money. So if we have money we can go to any school, any university we like or we may be sent to abroad to study to further our knowledge. If we... if we are rich we can help the poor and the old people without... without children. However, lack of money our life have many... many difficulties. In fact thousands of people are being dying every day because they have no money to pay their... or they can’t buy nutritious food to improve their health. To sum up, money is... indispensible in our life. Thanks to money, it maybe gives us a comfortable life and a cheerful heart. That’s all.
The assessor:If you have a lot of money what will you do?
Student 3:If I have a lot of money there are many things I want to do. But the first thing I do... I buy book, I buy English book for me... to... to study better.
The assessor:OK. Thank you.
Student 4:
Hello everybody. My topic is roles of women in the modern society. Women in the modern society play an important role. They not only have to be good at housework but also have to complete the work outside home. It’s said that men build the house women make the home. Women have to make their family happy. They always have to do housework completely. The house is always clean when husband and children come home... They have a good meal together. Women also have to take care of her husband and children. After a hard day... after a hard day taking care of wife will take the time with the husband. Women have to teach the children to be good children. Everything will be difficult when we in charge of women’s hands. In the old time, women just did only housework and... were not to charge such work, but now they also have to .... with what in society. They have to earn money... Women and men have to take care of their family together. Women can work than men and can get higher position in a company... offices. Being a women we have to ... being a women in the modern society I have to ... complete not only at home but also in society. It’s said that women is the heart of the world, so some day there’s no woman in the world, I think, the heart of the sky will be .... the heart of the world will be ruined. That’s all.
Student 5:
Good morning teachers. My topic is the subject I like best. As you know, up to now I learn.. have learned many subjects, for example mathematics, history. The subject I like best is English. Yes. There are many, I think, there are many reasons to make me to like English, but there are 3 main reasons. The first, with English I can be ability to learn it. That means I can learn it well. Yes. It is very easy for me to understand my teachers said, my teachers teach. And I can... I feel confident when I do exercise... So second, I think, in my opinion, I think with English subject... with English I can read many books... I read and understand many books... example Sunflower... Sunflower magazine. And with English I can talk and understand... tell and understand with foreigners. Maybe I can help them when they lose way or they want to buy something in the market, but they don’t know Vietmanese. The third, I... I learn... if I learn English well, I can choose many careers. Yes. As you know, nowadays English in many careers... example in school I can become a teacher, English teacher. In companies I can become a translator, translator, so maybe... To sum up, I choose English... I choose English is my best favourite subject because... I... I feel comfortable when I learn it and with English I can choose many careers. Thank you.
Student 6:
Good morning everybody. Today I’d like to present my topic. The topic is my opinion about the size of family... and... how many children I’d like when I get married. First, my opinion about the size of family. I think small family is always better than big family because when living in small family parents, husband and wife, cam support the needs of family easily. When... only when living in small family we can send our children to good school where... which have the best education and the best training. And... when I married I... two children is enough for me because you and I are parents to be and we have to responsible for the children ... for our children. For example, we have to taking care of our children’s health, and we have to send our children good school. First, we have... we want to send our children to good schools. When we live in small family we have a lot... we go out to earn money and we save a lot of money to send our children to good school. And when we were sent... we are sent to good school we have money to support them to buy some good books, some equipment for learning ... for studying... And about the health, when we live in small family, I think, we have a good taking care of the health, so I’d like to have only two children, one boy or one girl is better, but both of them are the boys or girls is no problem for me. That’s all.
Student 7:
Good morning everybody. How do you feel now? I’m a little nervous, but be confident. Yeah, my topic is opinion about the multi family.
The assessor:Multi-generation family.
Student 7:Yes. Living in a multi-generation family, there is plenty of fun, but there are some problems. Nowadays parents have to work all day, so they don’t have much time to care for their children. In multi-generation families, grandparents will have done to care for their children... Children can share their troubles, they can share everything that happen in their daily life to their grandparents. And when they meet problems they will have... they will get valuable advice from their grandparents. Grandparents will tell stories, will... they will sing many folksongs and... they will care for them all day when their parents are in work. I think there is plenty of fun, but there are some problems. We can see that a multi-generation family is the one that consists... that consists of three or more generations. And members in such kind of family are not in the same generation, so there are many differences in their opinion and their lifestyle. When old people prefer a quiet life, the young like to live an active and noisy life, together for pop music that too loud for old people. And their grandparents will complain and that makes them not pleased. Nowadays young people want to dress sexually when their grandparents don’t want their grandparents... their grandparents to do so. So i’m sorry... when their grandparents don’t want their granddaughter or grandson to do so. So they will complain and that makes them not pleased. It’s not eassy to compromise their lifestyle and their opinion. So to sum up, I think that everyone has two sides and a multi-generation family is the same. It has plenty of fun, but it also has problems. I think that is... that’s all. Thank you for your listening.
Student 8:
Hello everyone. Today... my topic today is teaching career. In society there are many careers, but teaching is the job I like best. I’m going to tell you the reason why I like this job. First, teaching career get high respect from society. It’s... it’s not only gets respect from people but also gets respect from their... their parents or people around... around them. Second, teaching career... teaching career is the career not ... it less competition and... you just go to class and teach... teach the pupils. You... you must have to worry about whether you get higher position than others. This makes you always feel happy. Besides it, I think, teaching career is the career you... which you... have to learn all life time. It seems to be not good, but all of us have the need to learn. Nowadays the world have many chances, but we always have to learn to improve our knowledge, to teach students. To sum up, teaching career is not only get high position but also it takes less competition than other careers.
Student 9:
Good morning teachers and everybody. Today I would like to tell you the importance of money. We can’t live without money. We need money to satisfy... our life. We need money to ... save for expenses, for example in family, in business and other problems. So, in some extent, money play important and necessary role in our life in our life because of these following reasons. The first, in family why we have to... when they don’t have enough money to expense, sorry, to pay for expense in our life... in their life. Without money they don’t... they don’t... they can’t buy what they need. For example, they don’t... they don’t afford all and all other what they need to live comfortable life. Without money their children don’t have good conditions in learning, for example good school, good teacher and good book for example. Without money, when they are ill... they... they don’t have good doctor, good hospital and can’t buy medicine. To do all of these, they need money, have a lot of money. Having money they can buy what they need and all other... they live... they need to live comfortable life. Having money they can have good healthy care... good learning in good condition, and... so they must work hard to earn money. Secondly, in business, if you want to open a shop, found a company,... you build factory, you need money, a lot of money. Having money you can pay salary for employees or doing business. Third, having money you can do charity... You need money to support poor students... scholarship with.. with the money... with the money you help them... contribute learning... and other. In conclusion, money play... important and necessary role in our life. In our life everybody try to work hard to earn money as much as possible. That’s all. Thank you for your listening.
The assessor:Can you buy happiness with money?
Student 9:I think money is... money play role important role in our life, but some... in some extent, money can all what they need, but we can’t buy ... spiritual... love and happiness with money.
The assessor:How can you buy it? How can you buy love? You say you can buy happiness with money. OK? But how?
Student 9:I wish I will go abroad to study, in Australia for example. It’s my happiness. With money I can do it.
Student 10:
My topic is number 4. If you were born again, would you like to change your sex? If one day a fairy appears... suddenly appears and asks me, ‘would you like to change your sex?’ I couldn’t hesitate to answer, ‘no, I wouldn’t’... I’d like to be a woman because I find that being a woman... has some advantages... Today I’d like to tell you the reason why I don’t want to change my sex. First, being a woman I consider as a fair sex. I can make me more beautiful, more... by making up. You can choose any clothes... fitting my body to make me more beautiful... If I... if I could... all my thought people... people who... around me said that being a woman I’m pleased... If a manusually helps me if I have a problem. Supposed that I’m the last person to go the bus. The bus at that time very crowded and no seat left to be sit on... to be sit on. Yes. Suddenly a man is ready ... to give his seat to me to sit on. If...if...if I’m not a woman, whether the man could... sit her... his to leave or that. Another example, I have just left the supermarket with many things heavy, a man in the street is ready... ready help me by carrying this for me. He want to prove that he is a polite person. But if I were a man at that time the man could carry... I could help him because I’m a girl. To sum up, being a woman I find it very interesting and I don’t want to change my sex. That’s all. Thank you for your listening.

Appendix 4: PHIẾU KHẢO SÁT

Thông tin trong phiếu khảo sát này chỉ dùng cho mục đích nghiên cứu. Mong quý thầy cô cho biết một số ý kiến về công tác kiểm tra đánh giá kỹ năng Nói.
  1. Khi đánh giá khả năng giao tiếp (taking interactional and transactional short turn) của sinh viên, thầy cô ưu tiên đánh giá những yếu tố nào dưới đây? (đánh số theo mức độ ưu tiên: 1 ưu tiên nhất .....8 ít ưu tiên nhất)
  1. Ngữ pháp đúng.
  2. Phát âm chấp nhận được.
  3. Từ vựng phù hợp.
  4. Giao tiếp được ý tưởng.
  5. Lưu loát.
  6. Làm tốt cả hai vai: người trả lời và người hỏi.
  7. Thường xuyên nói câu dài khi trả lời.
  8. Câu trả lời phù hợp với tình huống liên quan.
  9. Khi đánh giá khả năng trình bày (taking transactional long turns, e.g. oral report) của sinh viên, thầy cô ưu tiên đánh giá những yếu tố nào dưới đây? (đánh số theo mức độ ưu tiên: 1 ưu tiên nhất ..... 9 ít ưu tiên nhất)
  1. Ngữ pháp đúng.
  2. Phát âm chấp nhận được.
  3. Từ vựng phù hợp.
  4. Giao tiếp được ý tưởng.
  5. Lưu loát.
  6. Làm tốt cả hai vai: người trả lời và người hỏi.
  7. Thường xuyên nói câu dài khi trả lời.
  8. Câu trả lời phù hợp với tình huống liên quan.
  9. Nội dung đúng yêu cầu và phù hợp với đề tài.
  10. Theo thầy cô để kiểm tra được khả năng nói toàn diện của sinh viên, đề thi nên có số lượng bài tập (task) là…. (có thể có hơn 1 lựa chọn)
  1. 1
  2. 2
  3. 3
  4. 4
  5. Đánh dấu () vào ô trống cho sẵn loại hình bài tập thi thầy cô cho là phù hợp với năm 1, năm 2 và năm 3.
Năm 1 Năm 2 Năm 3
  1. Thảo luận/trao đổi ý kiến giữa 2 sinh viên.
  2. Trình bày chủ đề.
  3. Phỏng vấn (chỉ trả lời câu hỏi của giám khảo).
  4. Đưa ra hướng dẫn để người khác thực hiện công
việc (chỉ đường, vẽ sơ đồ,v.v...).
  1. Hoàn thành bài hội thoại bị giấu đi một số lời thoại.
  2. Kể chuyện lại sau khi được đọc.
  3. Kể chuyện theo tranh.
  4. Đóng vai
  5. Đọc bài khóa hay một đoạn bài khóa.
  6. Những câu hỏi nào dưới đây thầy cô chọn để hỏi sinh viên năm 1, 2 và 3? (Đánh dấu vào ô được chọn)
Năm 1 Năm 2 Năm 3
  1. Describe the most important sports event in your
city/country. 
  1. What do you do to keep fit?
  2. What sports do you think are dangerous? Why?
  3. What are your suggestions for reducing traffic jams?
  4. What can you do to help reduce pollution in the city?
  5. What can you think are global problems?
  6. What are your suggestions for being a fluent speaker
in a foreign language?
  1. What makes a good language learner?
  2. How do you learn new words, pronunciation and
grammar?
  1. Theo thầy cô phải thực hiện những việc gì dưới đây để tổ chức một kỳ thi Nói?
  1. Xác định mục đích của bài thi.
  2. Lựa chọn các dạng bài tập/nội dung thi.
  3. Tham khảo nội dung chương trình sinh viên đã học.
  4. Xác định tiêu chí về lĩnh vực kiến thức, kỹ năng và nội dung cần kiểm tra.
  5. Xác định cấu trúc bài thi (gồm mấy bài tập – task).
  6. Xác định đặc điểm cụ thể của từng bài tập (task), như mục đích, nội dung kiến thức và kỹ năng, thời gian thực hiện và cách chấm điểm từng bài tập.
  7. Theo thầy cô kết quả thầy cô đánh giá khả năng nói của sinh viên có phản ánh đúng năng lực của sinh viên?
  1. Chắc chắn đúng
  2. Không chắc lắm
  3. Không chắc
  1. Lý do vì sao thầy cô chọn câu b và c? (có thể có hơn 1 lựa chọn)
  1. Sinh viên được chuẩn bị trước các đề tài/câu hỏi thi.
  2. Sinh viên dự đoán và chuẩn bị trước các câu hỏi của giám khảo và câu trả lời.
  3. Hình thức thi/bài tập (task) không đảm bảo đánh giá khả năng sử dụng tiếng Anh để giao tiếp của sinh viên.
  4. Không có tiêu chí, yêu cầu, thang điểm đánh giá cụ thể cho từng loại hình bài tập (task).
  5. Xin thầy cô vui lòng cho biết số năm giảng dạy môn tiếng Anh.
  6. Thầy cô đã từng tham gia khóa học hay hội thảo nào về kiểm tra đánh giá (testing)?
Rất cám ơn quý thầy cô đã bớt chút thời gian.
QUESTIONNAIRE
Give your opinion on oral language testing.
1. When assessing students’ performance on interactional and transactional short turns, which level of priority do you give to the following abilities? (Number your level of priority, e.g. the highest priority is 1 and the lowest 8)
a. Ability to use grammar accurately
b. Ability to pronounce acceptably
c. Ability to use vocabulary appropriately
d. Ability to convey their intended meaning(s)
e. Ability to speak fluently
f. Ability to interact effectively
g. Ability to produce extended speech
h. Ability to make responses appropriate to the situation
2. When assessing students’ performance on transactional long turns, which level of priority do you give to the following abilities? (Number your level of priority, e.g. the highest priority is 1 and the lowest 9)
a. Ability to use grammar accurately
b. Ability to pronounce acceptably
c. Ability to use vocabulary appropriately
d. Ability to convey their intended meaning(s)
e. Ability to speak fluently
f. Ability to interact effectively
g. Ability to produce extended speech
h. Ability to make responses appropriate to the situation
i. Ability to make a presentation with an adequate content
3. A test of overall speaking ability should make use of test tasks. (More than one choice is possible.)
a. 1
b. 2
c. 3
d. 4
4. Tick () the elicitation techniques that you think are suitable for students of Year 1, Year 2 and Year 3.
Year 1Year 2Year 3
a. Discussion/conversation
b. Oral report
c. Interview or Question & answer
d. Learner-learner description and re-creation
e. Reading blank dialogue
f. Retelling a story
g. Picture story
h. Role-play
i. Reading aloud
5. Tick () the particular test tasks / questions that you use for students of Year 1, Year 2 and Year 3.
Year 1Year 2Year 3
a. Describe the most important sports event in your
city/country. 
b. What do you do to keep fit?
c. What sports do you think are dangerous? Why?
d. What are your suggestions for reducing traffic jams?
e. What can you do to help reduce pollution in the city?
f. What can you think are global problems?
g. What are your suggestions for being a fluent speaker
in a foreign language?
h. What makes a good language learner?
i. How do you learn new words, pronunciation and
grammar?
6. In order to develop an achievement speaking test, which following things must we do? (More than one choice is possible)
a. Identify the test purpose(s)
b. Choose test tasks in the target language use domain.
c. Determine students’ topical knowledge and profile of language ability
d. Determine the construct/ability to be measured
e. Determine the structure of the test (the number of test tasks)
f. Identify the specifications of each test task such as purposes of the test task, specified components oral ability to be tested, expected duration of task performance and scoring method
7. Are you sure that the marks you have given on your students’ oral test performance can reflect their actual speaking ability?
a. Very sure
b. Not very sure
c. Not sure
8. The reason(s) for your choice of (b) or (c) for Question 7 is/are that. (More than one choice is possible)
a. The students were prepared for the topics or test questions in advance.
b. The students might guess the assessors’ questions and prepare in advance the answers to these questions.
c. The test tasks used can’t get the students to show their actual ability to communicate in English.
d. There were no criteria and instructions for marking each test task.
9. How long have you taught English? – For ………….. years.
10. Have you ever attended any courses in or workshops on language testing?

Comments, questions, feedback, criticisms?

Send feedback