Intelligence tests have a long tradition across different fields, yet little attention has been paid to howtheir measurement properties may change over time and impact selection outcomes and intelligence trends.This study illustrates these gaps and consequences, using a synonym test from the Swedish Enlistment Battery(SEB) as an example. The test aims to measure lexical ability, a narrow verbal ability. First, the measurementproperties of the synonym test across years (2014, 2018, and 2022) and between subgroups were evaluatedusing Rasch analysis. Secondly, the implications for changes in the synonym test's measurement propertieswere evaluated for basic military training in Sweden. Misfitting items and invariance issues were identified.With the removal of misfitting items, improved measurement properties were achieved, but this did notsignificantly impact the possibilities of identifying group differences. However, the lexical ability of some testtakers at the lower end of the scale was likely overestimated, whereas it was likely underestimated at the upperend of the scale if all original items were used. These findings underscore the importance of ensuring unbiasedmeasurement properties and invariance over time, highlighting the need for regular evaluations andrefinements of intelligence tests to maintain fairness and accuracy.