More details on test redesign

Yesterday USCIS conducted a public engagement to explain the test redesign project and take comments/questions about it. If you missed it, you can see the presentation slides here: https://www.uscis.gov/sites/default/files/document/outreach-engagements/USCIS_Stakeholder_Engagement_2022_Naturalization_Test_Redesign_Initiative.pdf

They reiterated some of the information I posted a few weeks ago, and added a few more details:

  • The purpose of the redesign is to eliminate barriers in the naturalization process, and create a consistent, fair, and transparent testing process. They aim to ensure that applicants have the information they need to prepare for all parts of the test. (Note: currently there is almost nothing available to students to help them prepare for the speaking test.)
  • The redesign is driven by an analysis they did on the fairness and efficiency of the current testing process as well as feedback from stakeholders (us!). Another factor is advances in technology for administering the test. (Note: It looks like they plan to administer the entire test on ipads/tablets.)
  • As announced earlier, only the civics and speaking test are being revised for this project. This is because the reading and writing tests are already standardized and aligned to the high-beginning NRS level.
  • The new speaking test will use picture prompts and and require applicants to describe the picture using simple words and phrases. Here's a sample of what they have in mind:

  • Scoring will be based on whether the applicant responds with relevant content vocabulary and uses simple words and phrases. The proposed passing score would be the same as the current reading and writing test that requires one correct answer out of a possible three attempts (I'm assuming this means with three different picture prompts).
  • The civics test will be updated with some new items, but will use much the same content as the current 100 questions. The big difference is that it will be a multiple choice test. (They didn't say whether applicants will be able to access audio, and this is an important point. If the questions and answer options are only in the form of text, the civics test become a de facto reading test, and if this is the case, it doesn't make sense to me that a separate reading test would be needed.)
  • Applicants will continue to have a list of 100 study questions and answers, and will need to answer six out of ten questions correctly to pass the test.
  • If I understood correctly, they said that the vocabulary used in the distractors (incorrect answer options) would likely be familiar to test-takers because they would be words taken from other questions and answers in the 100 study questions.
  • USCIS will seek recommendations from outside experts in language acquisition and test developpment to ensure that the civics and speaking tests are aligned to NRS Level 3 and that it follows best practices in test design. These experts will review draft test items before and after the trial test and provide recommendations.
  • No information was offered in regard to who would be doing the scoring (adjudication officers? someone with training/expertise in language assessment?) and at what point in the application process this would occur (at the interview, earlier as part of the application submission process?)

My takeaway so far is that this is a genuine attempt to make the test fairer and the testing process more efficient. But there are some pitfalls to multiple choice tests (and using tablets to administer them). This will add additional skill sets that some students will need to be familiarized with. I am particularly anxious to learn whether audio accompanyment will be provided. I also want to know more about the scoring protocals for the proposed speaking test before I can predict whether this will be an improvement over the current speaking test. I'll keep you posted as soon as I get more information!