Paired Learning Gains and Assessment Ceilings in Automotive Electrical Vocational Education

Abstract

Automotive electrical-system education requires an understanding of circuit principles and component functions, yet evaluations based on class averages can obscure individual differences and assessment limitations. This study examined changes in test performance before and after instruction recorded as problem-based learning in a headlamp and starter-system unit for Grade XI vocational automotive students. A one-group design was used to analyse 34 matched pre-test and post-test scores on a 0-to-100 scale. Descriptive statistics, paired t tests, Wilcoxon signed-rank tests, an exact sign test, standardized mean change, normalized gains, bootstrap confidence intervals, and exploratory analyses of baseline scores and gains were conducted. Mean scores increased from 71.91 to 95.88, representing a gain of 23.97 points. Thirty-two students improved, while two recorded lower post-test scores. All paired-change tests indicated statistically significant differences, with p values below 0.001, and the standardized within-student effect size was 1.09. An inverse association between baseline performance and arithmetic gain requires caution because the measures are mathematically linked. Seventeen students attained the maximum post-test score, indicating substantial ceiling concentration that may limit the assessment's discrimination at higher achievement levels. The results document a large, widespread improvement in recorded scores following the instructional unit. However, the absence of a comparison group prevents causal attribution of the improvement to problem-based learning. Student-level analysis and scrutiny of assessment properties offer a more informative basis for evaluating learning gains in vocational education.