|
Working Group Report
The BILC Chair tasked a Working Panel on Test Administration to provide guidelines related to STANAG test scores and qualifications for raters. Members of the panel are:
Canada Julie Dubeau
Germany Herbert Walinsky
Netherlands Gerard Seinhorst
United Kingdom Hugh Morgan
United States Martha Herzog
During the panel discussion on 8 June 2004 in Strasbourg, it was determined that, among the nations represented, STANAG test scores remain valid for 1 to 3 years. The panel unanimously agreed that this should be a national policy. Each nation has laws, procedures, or precedents for managing and reviewing test scores. However, all panel members concur that there should be a limit. Otherwise, examinees may be credited with a score that does not represent their true ability to perform tasks in the language. The 1-3 year range seems reasonable to the panel.
There was agreement that a testing system should have a method for examinees to appeal a score; for example, this would be appropriate when testing conditions did not meet acceptable standards. The United States has a fairly elaborate system of appeals; however, not all nations have found a need for this. Any appeals system should be in line with national expectations and needs.
Test security was considered important by the panel. The STANAG testing system should be independent from the instructional system. All test materials should be kept secure and under the control of a trained Test Control Officer.
On the other hand, examinees and language instructors should be familiar with the format of the test, the types of items, range of levels, and instructions. To ensure familiarity, it would be useful for nations to develop a method of providing orientation through sample items or even through familiarization guides that describe all facets of the test and provide samples that do not compromise actual test material.
It was agreed that personnel who rate speaking and writing tests must be well-trained and have regular opportunities for refresher training. These tests require considerable judgment, so raters need to be normed and re-normed periodically to ensure they adhere to standards.
The nations represented on the panel control the training of all test raters, including government employees and contract personnel. All are required to meet national standards for awarding a STANAG level. These nations do not use test results from commercial firms for official assignment of proficiency levels.
|