In studies that develop a new questionnaire, the prepared items include some for which almost everyone chooses the same answer and some that ask the same thing as another item. This case study uses 26 items asking about difficulties experienced by patients receiving outpatient chemotherapy. I narrowed down the items with item analysis, ran a factor analysis, and judged the number of factors and the issues that remained.
The study setting, disease, variables, and numbers in this article were newly created by me to explain how the analysis was carried out. The numbers were calculated from a randomly generated dataset and do not reflect the content or circumstances of any actual client project.
26 items, a 5-point scale, and responses from 542 patients
The study was a cross-sectional survey of 542 patients with cancer who were receiving outpatient chemotherapy. They answered 26 items about difficulties in daily life during treatment on a 5-point scale (1 to 5). The research was at the stage of developing the questionnaire, and the items had been prepared with 3 domains in mind: physical symptoms, feelings, and daily life and work.
I fixed the criteria for narrowing down the items before tabulating the responses. There were 3: remove items for which 60% or more of the responses were at an end of the scale (1 or 5), remove items whose correlation with the total score excluding that item was below 0.3, and remove one item from any pair with an inter-item correlation of 0.8 or more.
As a result, fever and distress from waiting times had nearly 70% of responses at an end of the scale, inconvenient parking and preference for hospital food had correlations with the total score of 0.07 and 0.08, and worry about the future had a correlation of 0.94 with anxiety about what lies ahead. These 5 items were removed, leaving 21 items (Table 1).
| Criterion fixed in advance | No. of items | Items meeting the criterion |
|---|---|---|
| 60% or more of responses at an end of the scale (1 or 5) | 2 | Fever, distress from waiting times |
| Correlation with the total score excluding the item below 0.3 | 2 | Preference for hospital food, inconvenient parking |
| Inter-item correlation of 0.8 or more (one item removed) | 1 | Worry about the future (correlation of 0.94 with anxiety about what lies ahead) |
| Total items removed | 5 | |
| Items remaining | 21 |
Deciding how to calculate correlations and how to choose the number of factors before the factor analysis
Responses on a 5-point scale are ordered categories, and the intervals between the numbers are not necessarily equal. With the ordinary correlation coefficient, items with more skewed responses give smaller correlations. From the item analysis through the factor analysis, I therefore calculated correlations as polychoric correlations, which assume an underlying continuous quantity.
If the number of factors is decided only by the criterion of eigenvalues of 1 or more, more factors tend to be extracted as the number of items increases. I therefore used parallel analysis: I created 100 random datasets by shuffling the responses within each item to remove the associations between items, and took as the number of factors the number of actual eigenvalues that exceeded the 95th percentile of the eigenvalues from the random data.
Furthermore, it is natural to expect that patients with stronger physical symptoms also have more difficulties with feelings and daily life. An orthogonal rotation, which sets the factors to be uncorrelated, would express this relationship in distorted loadings. I therefore used an oblique rotation (oblimin), which allows the factors to correlate.
If confirmatory factor analysis were also run on the same 542 patients, the fit would be checked in the data used to search for the structure, and it would look too good. Confirmation belongs in a separate sample, so the scope here was the exploratory factor analysis and the reliability coefficient (omega) for each factor.
Three factors emerged, and 1 item loaded on 2 factors

The sampling adequacy (KMO) of the remaining 21 items was 0.93, a level suitable for proceeding to factor analysis. In the parallel analysis, the actual eigenvalues exceeded those of the random data up to the third (1.78 vs. 1.32 for the third, and 0.88 vs. 1.26 for the fourth), so I set the number of factors at 3 (Figure 1).
| Item | Physical symptoms* | Feelings* | Daily life and work* | Communality |
|---|---|---|---|---|
| Fatigue | 0.77 | 0.56 | ||
| Loss of appetite | 0.72 | 0.55 | ||
| Pain | 0.67 | 0.46 | ||
| Nausea | 0.67 | 0.44 | ||
| Change in taste | 0.65 | 0.43 | ||
| Numbness | 0.53 | 0.29 | ||
| Shortness of breath | 0.50 | 0.27 | ||
| Low mood | 0.79 | 0.58 | ||
| Anxiety about what lies ahead | 0.78 | 0.62 | ||
| Difficulty concentrating | 0.63 | 0.37 | ||
| Irritability | 0.62 | 0.41 | ||
| Tearfulness | 0.53 | 0.32 | ||
| Loneliness | 0.52 | 0.37 | ||
| Light sleep | 0.41 | 0.47 | 0.62 | |
| Giving up hobbies | 0.72 | 0.49 | ||
| Adjusting work | 0.70 | 0.50 | ||
| Going out less | 0.65 | 0.42 | ||
| Burden of housework | 0.60 | 0.40 | ||
| Worry about treatment costs | 0.59 | 0.32 | ||
| Less social contact | 0.56 | 0.34 | ||
| Burden of hospital visits | 0.55 | 0.34 | ||
| Interfactor correlation with physical symptoms | — | 0.48 | 0.43 | |
| Interfactor correlation with feelings | 0.48 | — | 0.54 | |
| Omega coefficient† | 0.84 | 0.85 | 0.82 |
* Factor loadings. Absolute values below 0.3 are left blank.
† Reliability coefficient calculated from the items that loaded most highly on each factor.
From the content of the items with high loadings, the 3 factors could be read as physical symptoms, feelings, and daily life and work (Table 2). The interfactor correlations were moderate, at 0.43 to 0.54, consistent with the premise for choosing an oblique rotation, and the omega coefficients were 0.82 to 0.85.
However, light sleep loaded to a similar degree on 2 factors, at 0.41 on physical symptoms and 0.47 on feelings. Whether to keep this item, and in which domain score to include it, should be decided from the content of the item. I therefore reported it without removing it in the analysis and left the decision to the researchers.
Taken together, the statistical conclusion is that the 21 items formed 3 factors with sufficient reliability for each factor, but that the handling of light sleep is undecided and the communalities of numbness and shortness of breath were low, at 0.29 and 0.27. Because this structure was found by searching the same data, confirmatory factor analysis in a separate sample is needed before the questionnaire is used.
Work performed and estimated fee
| Item | Details |
|---|---|
| Analysis method | Exploratory factor analysis (polychoric correlations, number of factors decided by parallel analysis, oblique rotation (oblimin), and a reliability coefficient for each factor) |
| Additional analyses | Item analysis (ceiling and floor effects, item-total correlations, and inter-item correlations) |
| Tables prepared | 1 table of the item analysis, 1 table of factor loadings, interfactor correlations, and reliability coefficients; 2 tables in total |
| Figures prepared | 1 figure of the parallel analysis |
| Deliverables | Analysis report (reasons for the choice of methods and discussion of the results), tables of results (Excel), figures (PNG and TIFF), and a draft of the statistical analysis section for the paper. Analysis code is available on request for an additional fee (¥22,000, incl. tax) |
| Estimated time | About 1 to 2 weeks from receipt of the data |
| Estimated fee | ¥132,000 (incl. tax) |
The table lists the main analyses only. I also carry out the finer analyses needed to follow a sound statistical process, such as checking distributions and cleaning the data; these, together with after-care with no time limit (answers to your questions after delivery), are included in the fee.

