By David Oglesby
Partner Language Training Center Europe

TOURS, France (Sept. 9, 2019) – Forty-Seven language testers from 27 NATO and partner nations met here for the 10^th annual Bureau for International Language Coordination STAGNAG 6001 Testing Workshop from Sept. 3 to 5.  

In NATO, a Standardization Agreement (STANAG) defines processes, procedures, terms, and conditions for common military or technical procedures or equipment between the member countries of the alliance. Each NATO state ratifies a STANAG and implements it within their own military. The Standardization Agreement (STANAG 6001) describes language proficiency levels associated with listening, speaking, reading, and writing.

 “The annual BILC STANAG 6001 Testing Workshop promotes standardization of NATO and partner nation language proficiency testers by providing a forum for the exchange of information, best practices and new approaches in the field of language testing,” said Branka Petek from Slovenia, who is the BILC chair. “It provides a unique opportunity for the national testing teams to align their testing with STANAG 6001 language proficiency descriptors.”

Since the BILC motto is “Progress made by one is shared by all,” BILC leaders decided that the theme of this 2019 STANAG 6001 Testing Workshop was NATO Benchmark Advisory Test 2 (BAT2) lessons learned.

History of Improving Language Testing Standardization

For the past 20 years, members of the NATO’s Bureau for International Language Coordination (BILC) have worked to improve standardization efforts of STANAG 6001 language testing community. 

The launch of a Language Testing Seminar at the Partner Language Training Center Europe in 2000 began this process in earnest, and the 2009 Benchmark Advisory Test (BAT), an external measure of the consistency of national STANAG 6001 test scoring, proved an initial measure of standardization success for the community. 

Ten years later, following the implementation of other standardization efforts – an Advanced LTS, an annual STANAG 6001 Testing Workshop, a collection of language-testing best practices, and various other initiatives – BILC leaders decided to run a second BAT (BAT2) to determine what effect such actions had had on the community. 

Language Proficiency Levels Vital to Interoperability

As with weapons and information systems, a Standardization Agreement for Language Proficiency Levels (STANAG 6001) is vital to interoperability within the alliance. 

“NATO's overarching goal of interoperability among NATO and Partner countries is about the ability to communicate and work together,” said Petek. “When representatives of nations work together on a tactical, operational or strategic level, they need to share a language.”

She added, “Ensuring that national representatives share a common language is a national responsibility but the interpretation of standards and interpretation of test results have to be normed across nations. The BILC's mission to promote and foster interoperability among NATO and partner nations by furthering standardization of language training and testing plays a crucial role in this process.”

She explained, “Language-proficiency test results that are aligned to a common standard promote personnel mobility and enable employers to rely on national nominations.”  At the same time, they enable the personnel to perform their military duties without struggling with linguistic barriers.”

American Council on the Teaching of Foreign Languages

In December 2017, the George C. Marshall European Center for Security Studies awarded a contract to the American Council on the Teaching of Foreign Languages (ACTFL) to develop and administer BAT2. 

Additionally, ACTFL conducted two one-week norming forums for 20 BAT2 Speaking and Writing tester and rater candidates.  ACTFL trainers selected for certification only candidates, which was a total of 13, who completed the norming forum and submitted rated Speaking and/or Writing samples demonstrating a high degree of interrater reliability. 

The BAT 2 Project

Between November 2018 and June 2019, 18 nations participated in the BAT2 project, in which National STANAG 6001 language testers administered their own Listening, Speaking, Reading, and Writing assessments and, then, a battery of BAT2 assessments covering the same skills.

Upon conclusion of the project, subject matter experts from PLTCE, Brigham Young University, Yale University, and ACTFL analyzed the results of BAT2 and compared national STANAG 6001 test scores with BAT2 scores. 

Post-facto reviews with subject matter experts are a necessary step in recognizing, revealing, studying, documenting, and implementing lessons learned from any major project.  A total of 196 test takers from the 18 participating nations successfully completed both batteries of STANAG 6001-based assessments. 

The BAT2 test takers, proctors, and testers/raters submitted post-test questionnaires about the quality of the tests and the testing experience. 

Result of the BAT2 Project, the Future

The subject matter exerts agreed that, although quantitative and qualitative analyses indicate that, overall, the BILC language-testing community is standardized and normed, standardization is an ongoing process, not an end product. 

The BAT2 organizers, proctors, testers and raters, analysts and stakeholders, in general, concurred that the project had produced useful statistical information, valuable lessons learned, and a much-needed supply of fresh sample test items to update language-testing seminars and online demos. 

Perhaps, in another 10 years, BILC leaders will consider organizing BAT3.