Factor Analysis | Narrowing Down Items in a Chemotherapy Questionnaire

In studies that develop a new questionnaire, the prepared items include some for which almost everyone chooses the same answer and some that ask the same thing as another item. This case study uses 26 items asking about difficulties experienced by patients receiving outpatient chemotherapy. I narrowed down the items with item analysis, ran a factor analysis, and judged the number of factors and the issues that remained.

The study setting, disease, variables, and numbers in this article were newly created by me to explain how the analysis was carried out. The numbers were calculated from a randomly generated dataset and do not reflect the content or circumstances of any actual client project.

26 items, a 5-point scale, and responses from 542 patients

The study was a cross-sectional survey of 542 patients with cancer who were receiving outpatient chemotherapy. They answered 26 items about difficulties in daily life during treatment on a 5-point scale (1 to 5). The research was at the stage of developing the questionnaire, and the items had been prepared with 3 domains in mind: physical symptoms, feelings, and daily life and work.

I fixed the criteria for narrowing down the items before tabulating the responses. There were 3: remove items for which 60% or more of the responses were at an end of the scale (1 or 5), remove items whose correlation with the total score excluding that item was below 0.3, and remove one item from any pair with an inter-item correlation of 0.8 or more.

As a result, fever and distress from waiting times had nearly 70% of responses at an end of the scale, inconvenient parking and preference for hospital food had correlations with the total score of 0.07 and 0.08, and worry about the future had a correlation of 0.94 with anxiety about what lies ahead. These 5 items were removed, leaving 21 items (Table 1).

Table 1. Results of the item analysis (26 items, n=542)
Criterion fixed in advance No. of items Items meeting the criterion
60% or more of responses at an end of the scale (1 or 5) 2 Fever, distress from waiting times
Correlation with the total score excluding the item below 0.3 2 Preference for hospital food, inconvenient parking
Inter-item correlation of 0.8 or more (one item removed) 1 Worry about the future (correlation of 0.94 with anxiety about what lies ahead)
Total items removed 5
Items remaining 21
Correlations are polychoric correlations (polyserial correlations for the correlation with the total score). The criteria were fixed before seeing the results, and items meeting any one of them were removed.

Deciding how to calculate correlations and how to choose the number of factors before the factor analysis

Responses on a 5-point scale are ordered categories, and the intervals between the numbers are not necessarily equal. With the ordinary correlation coefficient, items with more skewed responses give smaller correlations. From the item analysis through the factor analysis, I therefore calculated correlations as polychoric correlations, which assume an underlying continuous quantity.

If the number of factors is decided only by the criterion of eigenvalues of 1 or more, more factors tend to be extracted as the number of items increases. I therefore used parallel analysis: I created 100 random datasets by shuffling the responses within each item to remove the associations between items, and took as the number of factors the number of actual eigenvalues that exceeded the 95th percentile of the eigenvalues from the random data.

Furthermore, it is natural to expect that patients with stronger physical symptoms also have more difficulties with feelings and daily life. An orthogonal rotation, which sets the factors to be uncorrelated, would express this relationship in distorted loadings. I therefore used an oblique rotation (oblimin), which allows the factors to correlate.

If confirmatory factor analysis were also run on the same 542 patients, the fit would be checked in the data used to search for the structure, and it would look too good. Confirmation belongs in a separate sample, so the scope here was the exploratory factor analysis and the reliability coefficient (omega) for each factor.

Three factors emerged, and 1 item loaded on 2 factors

Line plot overlaying the first 10 eigenvalues of the actual data for the 21 items and the 95th percentile of the eigenvalues from random data
Figure 1. Results of the parallel analysis used to decide the number of factors in the factor analysis. The solid line shows the eigenvalues of the actual data, and the dashed line the 95th percentile of the eigenvalues from 100 random datasets in which the responses were shuffled within each item.

The sampling adequacy (KMO) of the remaining 21 items was 0.93, a level suitable for proceeding to factor analysis. In the parallel analysis, the actual eigenvalues exceeded those of the random data up to the third (1.78 vs. 1.32 for the third, and 0.88 vs. 1.26 for the fourth), so I set the number of factors at 3 (Figure 1).

Table 2. Factor loadings, interfactor correlations, and reliability coefficients from the exploratory factor analysis (21 items)
Item Physical symptoms* Feelings* Daily life and work* Communality
Fatigue 0.77 0.56
Loss of appetite 0.72 0.55
Pain 0.67 0.46
Nausea 0.67 0.44
Change in taste 0.65 0.43
Numbness 0.53 0.29
Shortness of breath 0.50 0.27
Low mood 0.79 0.58
Anxiety about what lies ahead 0.78 0.62
Difficulty concentrating 0.63 0.37
Irritability 0.62 0.41
Tearfulness 0.53 0.32
Loneliness 0.52 0.37
Light sleep 0.41 0.47 0.62
Giving up hobbies 0.72 0.49
Adjusting work 0.70 0.50
Going out less 0.65 0.42
Burden of housework 0.60 0.40
Worry about treatment costs 0.59 0.32
Less social contact 0.56 0.34
Burden of hospital visits 0.55 0.34
Interfactor correlation with physical symptoms — 0.48 0.43
Interfactor correlation with feelings 0.48 — 0.54
Omega coefficient† 0.84 0.85 0.82
Results of the minimum residual method based on polychoric correlations, with oblimin rotation. The factors are named from the content of the items with high loadings.
* Factor loadings. Absolute values below 0.3 are left blank.
† Reliability coefficient calculated from the items that loaded most highly on each factor.

From the content of the items with high loadings, the 3 factors could be read as physical symptoms, feelings, and daily life and work (Table 2). The interfactor correlations were moderate, at 0.43 to 0.54, consistent with the premise for choosing an oblique rotation, and the omega coefficients were 0.82 to 0.85.

However, light sleep loaded to a similar degree on 2 factors, at 0.41 on physical symptoms and 0.47 on feelings. Whether to keep this item, and in which domain score to include it, should be decided from the content of the item. I therefore reported it without removing it in the analysis and left the decision to the researchers.

Taken together, the statistical conclusion is that the 21 items formed 3 factors with sufficient reliability for each factor, but that the handling of light sleep is undecided and the communalities of numbness and shortness of breath were low, at 0.29 and 0.27. Because this structure was found by searching the same data, confirmatory factor analysis in a separate sample is needed before the questionnaire is used.

Work performed and estimated fee

Item Details
Analysis method Exploratory factor analysis (polychoric correlations, number of factors decided by parallel analysis, oblique rotation (oblimin), and a reliability coefficient for each factor)
Additional analyses Item analysis (ceiling and floor effects, item-total correlations, and inter-item correlations)
Tables prepared 1 table of the item analysis, 1 table of factor loadings, interfactor correlations, and reliability coefficients; 2 tables in total
Figures prepared 1 figure of the parallel analysis
Deliverables Analysis report (reasons for the choice of methods and discussion of the results), tables of results (Excel), figures (PNG and TIFF), and a draft of the statistical analysis section for the paper. Analysis code is available on request for an additional fee (¥22,000, incl. tax)
Estimated time About 1 to 2 weeks from receipt of the data
Estimated fee ¥132,000 (incl. tax)

The table lists the main analyses only. I also carry out the finer analyses needed to follow a sound statistical process, such as checking distributions and cleaning the data; these, together with after-care with no time limit (answers to your questions after delivery), are included in the fee.

Data Scientist Kenichiro Suzuki

Data ScientistKenichiro Suzuki

Free consultationSee fee examples

Recent Posts

Linear Mixed Model | 6-Minute Walk Distance in COPD Rehabilitation

A linear mixed model comparing 6-minute walk distance at four time points with missing data. How to read a non-significant group-by-time interaction.

Cox Regression and Multiple Imputation | Mortality in Hemodialysis

Cox regression of 8 factors and all-cause mortality in new hemodialysis patients, with multiple imputation for missing grip strength and other values.

ROC Analysis | Comparing the AUCs of 4 Blood Markers for Bacteremia

ROC analysis of the AUCs of 4 markers for bacteremia: all pairs tested with the paired DeLong test and corrected with the Benjamini–Hochberg procedure.

Kruskal–Wallis Test | Fatigue and Pain in 5 Rheumatoid Arthritis Groups

Kruskal–Wallis test with corrected pairwise comparisons of fatigue and pain scores among 5 rheumatoid arthritis treatment groups, plus factor analysis.

RMST | Pancreatic Cancer Chemotherapy When Survival Curves Cross

RMST (restricted mean survival time) comparing 2 first-line chemotherapies for advanced pancreatic cancer when survival curves cross, with adjustment.

Multiple Regression | Factors Associated With Dementia Caregiver Burden

Multiple regression with 8 prespecified factors for burden and quality of life in family caregivers of people with dementia, with principal components.

Multiple Regression | Monthly Antimicrobial Use and Resistance Over Time

Multiple regression of monthly antimicrobial use and the proportion of resistant Pseudomonas aeruginosa, after unit root tests and differencing.

Cox Regression | Recurrence After 2 Methods of Atrial Fibrillation Ablation

Cox regression comparing recurrence after 2 methods of catheter ablation for atrial fibrillation, with 6 subgroups read from tests of interaction.