This case is a statistical analysis of how video teaching materials simulating events that actually occur in the medical setting affect trainees’ “learning effect” and “mental load” in training for healthcare workers.
Reconciling improved quality of medical education with learners’ well-being is an important challenge, and evaluating which training modules are most effective while not imposing unnecessary mental burden is essential to designing more practical and humane education programs.
Dr.DataScience quantitatively evaluated this complex relationship and made data-based proposals for improving the educational content.
Background and objective
In introducing new training modules for healthcare workers, the client wished to objectively evaluate how each module affects trainees’ “learning effect” (knowledge acquisition and understanding of techniques) and “mental load” (stress and sense of burden).
The aim was to provide balanced training in which trainees acquire practical skills while not being exposed to excessive mental stress—not merely cramming in knowledge. Through this analysis, what was required was to clarify the degree of “learning effect” and “mental load” of each training module, what relationship exists between the two, whether particular training modules have a statistically significant difference in “learning effect” or “mental load” compared with other modules, and whether trainees’ “clinical experience” introduces bias affecting these evaluation items.
Data and variables
The data used in the analysis of this case were evaluation responses (2,000 responses in total) from healthcare workers or healthcare students who viewed multiple medical-training modules.
- Learning effect: a score rating, on a 5-point scale, the degree of knowledge acquisition and understanding of techniques obtained from each training module.
- Mental load: a score rating, on a 5-point scale, the mental stress and sense of burden felt after viewing each training module.
- Clinical experience: a categorical variable indicating whether the respondent has clinical experience (“no experience,” “has experience,” “no response/unknown”).
Analytical methods
In this case, I performed the following analyses regarding the evaluation of the medical-training modules.
- Preliminary analysis
- I tabulated the distribution of respondents’ “clinical experience” attribute.
- To statistically check whether there was bias in “clinical experience” across the training-module IDs, I performed a chi-squared test (significance level 0.05).
- To check whether the score distributions of the main evaluation items, “learning effect” and “mental load,” followed a normal distribution, I performed the Shapiro–Wilk test (significance level 0.05) and judged the appropriateness of the analytical methods.
- Analysis not accounting for “clinical experience”
- To evaluate whether there was a statistically significant relationship between “learning effect” and “mental load,” I performed a correlation analysis using Spearman’s rank correlation coefficient. I selected a nonparametric method because both variables were non-normally distributed.
- To determine whether there was a statistically significant difference in “learning effect” scores among the training-module IDs, I performed the Kruskal–Wallis test (nonparametric analysis of variance).
- Where the Kruskal–Wallis test found a significant difference, I performed multiple-comparison analysis using the Steel–Dwass test to identify specifically which combinations of training-module IDs differed significantly.
Overview of the main results and clinical considerations
As a result of the analysis, the important findings were as follows: first, no statistical bias was found in respondents’ “clinical experience” across the training-module IDs, and because neither the score distribution of “learning effect” nor that of “mental load” showed normality, nonparametric methods were judged appropriate.
Next, a statistically significant positive correlation was found between “learning effect” and “mental load,” suggesting that the higher the learning effect of a training module, the higher the mental load felt by trainees tends to be. The clinical consideration here is that videos dealing with more advanced techniques or ethically complex cases may have a high learning effect but, because they require concentration and emotional engagement, may also impose a greater mental burden.
Furthermore, it was found that there were statistically significant differences in “learning effect” scores among the training-module IDs, indicating that particular modules have a high learning effect. Through multiple comparisons (the Steel–Dwass test), several specific combinations of module IDs with significant differences in learning effect were identified.
These results suggest the importance of identifying modules with a high learning effect and using them preferentially when designing an education program, and show that it is possible to select and combine teaching materials that are effective and considerate of trainees, taking the balance with mental load into account.
Dr.DataScience’s contribution
In this case, Dr.DataScience provided data analysis to evaluate the effectiveness of medical training from multiple angles. As its main contribution, based on the preliminary-analysis finding that “learning effect” and “mental load” were non-normally distributed, I appropriately selected and applied nonparametric tests—the chi-squared test, Spearman’s rank correlation coefficient, the Kruskal–Wallis test, and the Steel–Dwass test—and derived reliable results matching the characteristics of the data.
I also quantitatively showed the correlation between the two important evaluation items, “learning effect” and “mental load,” clarifying the trade-off whereby a higher learning effect leads to an increase in mental load. This insight provides important implications for medical educators to consider the balance between effect and burden when designing training content.
Furthermore, by identifying the significant differences in “learning effect” among the training modules and concretely showing which modules are highly effective, I supported the client’s data-based decision-making for building more practical and effective education programs, contributing to improving the quality of medical education and optimizing the learning experience of trainees.