| Gulliksen's pool |
|
Exploratory factor analysis is widely used for item analysis in the earlier stages of test development, usually with large pools of items. In this scenario, the presence of inappropriate or ineffective items can hamper the process of analysis, making it very difficult to correctly assess dimensionality and structure. To avoid, or greatly minimize, this (quite frequent) problem, we implement the procedure Gulliksen’s-Pool (G-Pool) designed to flag potentially problematic items before we specify any particular factorial solution. The procedure defines regions of item appropriateness and efficiency based on the combined impact of two prior item features: extremeness and consistency. The basic G-Pool proposal consists of (a) defining a safety central region of item appropriateness, and (b) flagging those items that fall outside the boundaries of this region as inappropriate for being included in further factor item analysis solutions based on this item set. In more detail, the items outside this region are those considered to be (a) ineffective, (b) prone to leading to problems, or both. The boundaries of the appropriateness region will be determined by using extensive simulation.
The R script GulliksenPool.r It is a script that uses only native functions in R, so no packages need to be downloaded to use it. In order to use it, the researcher has to store participants' responses in a text file and to execute the script. Different parameters can be set:
Click here to download the R script
The SPSS script GulliksenPool.sps It is a script that uses MATRIX language in SPSS. In order to use it, the researcher has to store participants' responses in a SPSS data file. Different parameters can be set:
Click here to download the SPSS script
Gulliksen's pool implemented in FACTOR software The analysis has also been implemented in FACTOR. You can run into using the button:
in the configuration menu:
Illustrative OUTPUT The output indicates the items that are identified as the most convenient ones to remain in an exploratory factor analysis, and the ones proposed to be revised. This is an example of the output provided when analyzing a dataset using the following configuration in the R script.
CONFIGURATION OF PARAMETERS############################################################## # UPDATE HERE THE NAME OF YOUR INPUT DATA FILE # A TEXT FILE WITH NO LABELS IS EXPECTED # filein <- "VandoSample1.dat" # # UPDATE HERE THE CODE VALUE FOR MISSING VALUES # MV <- 9 # # UPDATE HERE THE NUMBER OF BOOTSTRAP SAMPLES (AT LEAST 500) # K <- 3000 # # UPDATE HERE THE CONFIDENCE INTERVAL # C <- 0.95 # # # UPDATE HERE THE MSA CRITERION TO ELIMINATE ITEMS # CRIT_MSA <- 0.50 # # # UPDATE HERE THE RDI CRITERIA TO INCLUDE ITEMS IN THE POOL (ALWAYS IN LINEAR PARAMETRIZATION) # spool_rdi_inf <- 0.10 spool_rdi_sup <- 0.90 # # # UPDATE HERE THE ICI CRITERIA TO INCLUDE ITEMS IN THE POOL (ALWAYS IN LINEAR PARAMETRIZATION) # spool_ici_inf <- 0.20 spool_ici_sup <- 0.75 # # # UPDATE HERE THE PARAMETRIZATION FOR THE OUTPUT # # VALUE 1 LINEAR (INTERCEPT/CORRELATION) # VALUE 2 NON LINEAR - ITEM RESPONSE THEORY (THRESHOLD/SLOPE) # parametrization <- 2 # # ############################################################## OUTPUT PRODUCED########################################################################################### # # # GULLIKSEN'S POOL R C O D E # # # # AUTHORS: URBANO LORENZO-SEVA & PERE J. FERRANDO # # URV, TARRAGONA (SPAIN) # # # # DATE: 21/09/2022 # # # ########################################################################################### # # Filein : VandoSample1.dat # Cases : 578 # Cases with missings : 1 # Variables : 61 # Boostrap samples : 3000 # Percentile : 0.95 # Parametrization : NON LINEAR - ITEM RESPONSE THEORY (THRESHOLD/SLOPE) # Items to revise : 21 # #GULLIKSEN'S POOL: A QUICK TOOL FOR PRELIMINARY DETECTION OF INAPPROPRIATE ITEMS # #STEP 1: The pool based on Overall Item Threshold (OIT) and Overall Item Slope (OIS) # # #Items OIT 95% Confidence OIS 95% Confidence The Pool # interval interval #----- ---------------------------- ------------------------- -------- # 1 -0.665 (-0.560 -- -0.778) 0.515 (0.553 -- 0.738) Admit # 2 -0.289 (-0.186 -- -0.390) 0.664 (0.684 -- 0.887) Admit # 3 -0.302 (-0.203 -- -0.404) 0.914 (0.905 -- 1.158) Admit # 4 -0.067 ( 0.037 -- -0.168) 0.518 (0.556 -- 0.744) Admit # 5 0.495 ( 0.606 -- 0.385) 0.622 (0.649 -- 0.844) Admit # 6 0.366 ( 0.471 -- 0.266) 0.360 (0.429 -- 0.598) Admit # 7 0.115 ( 0.221 -- 0.011) 0.546 (0.584 -- 0.770) Admit # 8 0.622 ( 0.737 -- 0.515) 0.367 (0.437 -- 0.601) Admit # 9 -0.085 ( 0.011 -- -0.186) 0.801 (0.812 -- 1.034) Admit # 10 -0.772 (-0.665 -- -0.888) 0.619 (0.635 -- 0.851) Admit # 11 -0.432 (-0.325 -- -0.540) 0.390 (0.453 -- 0.626) Admit # 12 -0.385 (-0.284 -- -0.495) 0.542 (0.577 -- 0.769) Admit # 13 0.505 ( 0.612 -- 0.404) 0.378 (0.441 -- 0.616) Admit # 14 0.715 ( 0.832 -- 0.601) 0.928 (0.912 -- 1.193) Admit # 15 0.129 ( 0.230 -- 0.028) 0.878 (0.883 -- 1.106) Admit # 16 0.089 ( 0.195 -- -0.015) 0.475 (0.520 -- 0.704) Admit # 17 -0.535 (-0.428 -- -0.643) 0.348 (0.422 -- 0.589) Admit # 18 -0.649 (-0.535 -- -0.760) 0.514 (0.549 -- 0.739) Admit # 19 0.302 ( 0.409 -- 0.199) 0.779 (0.785 -- 1.008) Admit # 20 -0.418 (-0.311 -- -0.525) 1.073 (1.072 -- 1.321) Admit # 21 0.002 ( 0.107 -- -0.098) 0.398 (0.460 -- 0.635) Admit # 22 0.080 ( 0.181 -- -0.020) 0.835 (0.838 -- 1.079) Admit # 23 -0.270 (-0.168 -- -0.376) 0.806 (0.813 -- 1.032) Admit # 24 0.307 ( 0.409 -- 0.203) 0.554 (0.586 -- 0.783) Admit # 25 0.059 ( 0.159 -- -0.041) 0.682 (0.695 -- 0.910) Admit # 26 0.150 ( 0.252 -- 0.046) 0.364 (0.436 -- 0.605) Admit # 27 -0.137 (-0.028 -- -0.239) 0.763 (0.773 -- 0.998) Admit # 28 -0.362 (-0.257 -- -0.471) 0.998 (0.996 -- 1.235) Admit # 29 -0.235 (-0.133 -- -0.339) 1.065 (1.067 -- 1.311) Admit # 30 0.732 ( 0.850 -- 0.617) 0.476 (0.517 -- 0.703) Admit # 31 0.054 ( 0.159 -- -0.046) 1.256 (1.230 -- 1.553) Might not work - Revise # 32 -0.028 ( 0.072 -- -0.129) 0.525 (0.561 -- 0.751) Admit # 33 0.606 ( 0.715 -- 0.500) 0.868 (0.870 -- 1.106) Admit # 34 0.456 ( 0.565 -- 0.353) 0.530 (0.564 -- 0.761) Admit # 35 0.423 ( 0.530 -- 0.320) 0.657 (0.673 -- 0.889) Admit # 36 -0.428 (-0.325 -- -0.530) 0.393 (0.448 -- 0.631) Admit # 37 0.172 ( 0.279 -- 0.072) 0.957 (0.966 -- 1.190) Admit # 38 -0.033 ( 0.072 -- -0.137) 0.492 (0.532 -- 0.714) Admit # 39 0.418 ( 0.525 -- 0.320) 0.716 (0.735 -- 0.938) Admit # 40 -0.235 (-0.133 -- -0.334) 0.666 (0.683 -- 0.895) Admit # 41 -0.481 (-0.376 -- -0.591) 0.489 (0.531 -- 0.717) Admit # 42 -0.115 (-0.015 -- -0.221) 0.503 (0.545 -- 0.728) Admit # 43 0.028 ( 0.133 -- -0.076) 0.740 (0.762 -- 0.964) Admit # 44 0.638 ( 0.749 -- 0.530) 0.540 (0.578 -- 0.767) Admit # 45 0.275 ( 0.380 -- 0.172) 0.629 (0.655 -- 0.854) Admit # 46 0.692 ( 0.807 -- 0.586) 0.437 (0.489 -- 0.665) Admit # 47 0.660 ( 0.778 -- 0.550) 0.690 (0.704 -- 0.915) Admit # 48 -0.591 (-0.485 -- -0.698) 0.400 (0.459 -- 0.633) Admit # 49 -0.155 (-0.054 -- -0.261) 0.462 (0.509 -- 0.692) Admit # 50 0.266 ( 0.376 -- 0.164) 0.979 (0.971 -- 1.239) Admit # 51 -0.339 (-0.235 -- -0.442) 0.532 (0.570 -- 0.759) Admit # 52 0.622 ( 0.732 -- 0.515) 0.549 (0.579 -- 0.780) Admit # 53 -0.357 (-0.257 -- -0.466) 0.486 (0.530 -- 0.711) Admit # 54 0.390 ( 0.495 -- 0.289) 0.456 (0.502 -- 0.692) Admit # 55 0.376 ( 0.485 -- 0.275) 0.970 (0.958 -- 1.234) Admit # 56 -0.261 (-0.159 -- -0.366) 0.429 (0.480 -- 0.655) Admit # 57 -1.063 (-0.934 -- -1.204) 0.661 (0.670 -- 0.902) Admit # 58 0.015 ( 0.115 -- -0.085) 1.388 (1.370 -- 1.691) Might not work - Revise # 59 0.418 ( 0.525 -- 0.316) 0.749 (0.764 -- 0.975) Admit # 60 0.124 ( 0.230 -- 0.024) 0.879 (0.882 -- 1.120) Admit # 61 -0.275 (-0.172 -- -0.380) 0.436 (0.491 -- 0.666) Admit #----- ---------------------------- ------------------------- -------- # #STEP 2: Assessment of the measure of sampling adequacy (MSA) at the item level # # Item MSA 95% Confidence The Pool # interval # ----- ----- ------------------ --------- # 1 0.651 (0.480 -- 0.699) Might not work - Revise # 2 0.699 (0.555 -- 0.734) Admit # 3 0.788 (0.677 -- 0.810) Admit # 4 0.604 (0.456 -- 0.670) Might not work - Revise # 5 0.908 (0.783 -- 0.893) Admit # 6 0.669 (0.446 -- 0.710) Might not work - Revise # 7 0.824 (0.653 -- 0.827) Admit # 8 0.564 (0.412 -- 0.623) Might not work - Revise # 9 0.919 (0.826 -- 0.911) Admit # 10 0.864 (0.724 -- 0.860) Admit # 11 0.581 (0.408 -- 0.662) Might not work - Revise # 12 0.825 (0.654 -- 0.831) Admit # 13 0.620 (0.429 -- 0.671) Might not work - Revise # 14 0.664 (0.559 -- 0.695) Admit # 15 0.840 (0.736 -- 0.851) Admit # 16 0.558 (0.423 -- 0.629) Might not work - Revise # 17 0.464 (0.356 -- 0.582) Might not work - Revise # 18 0.779 (0.594 -- 0.784) Admit # 19 0.666 (0.559 -- 0.700) Admit # 20 0.890 (0.817 -- 0.892) Admit # 21 0.610 (0.419 -- 0.685) Might not work - Revise # 22 0.861 (0.751 -- 0.862) Admit # 23 0.866 (0.760 -- 0.871) Admit # 24 0.753 (0.584 -- 0.755) Admit # 25 0.846 (0.709 -- 0.849) Admit # 26 0.731 (0.476 -- 0.764) Might not work - Revise # 27 0.874 (0.767 -- 0.876) Admit # 28 0.856 (0.772 -- 0.865) Admit # 29 0.879 (0.799 -- 0.881) Admit # 30 0.704 (0.502 -- 0.743) Admit # 32 0.600 (0.469 -- 0.648) Might not work - Revise # 33 0.901 (0.812 -- 0.898) Admit # 34 0.870 (0.705 -- 0.864) Admit # 35 0.585 (0.475 -- 0.631) Might not work - Revise # 36 0.773 (0.534 -- 0.791) Admit # 37 0.918 (0.845 -- 0.915) Admit # 38 0.670 (0.488 -- 0.717) Might not work - Revise # 39 0.871 (0.752 -- 0.874) Admit # 40 0.875 (0.753 -- 0.879) Admit # 41 0.730 (0.535 -- 0.762) Admit # 42 0.680 (0.494 -- 0.727) Might not work - Revise # 43 0.856 (0.737 -- 0.862) Admit # 44 0.847 (0.676 -- 0.851) Admit # 45 0.868 (0.732 -- 0.870) Admit # 46 0.565 (0.405 -- 0.658) Might not work - Revise # 47 0.800 (0.656 -- 0.809) Admit # 48 0.716 (0.491 -- 0.746) Might not work - Revise # 49 0.644 (0.470 -- 0.696) Might not work - Revise # 50 0.911 (0.834 -- 0.908) Admit # 51 0.837 (0.664 -- 0.840) Admit # 52 0.871 (0.716 -- 0.866) Admit # 53 0.801 (0.602 -- 0.814) Admit # 54 0.669 (0.470 -- 0.722) Might not work - Revise # 55 0.658 (0.567 -- 0.685) Admit # 56 0.537 (0.389 -- 0.637) Might not work - Revise # 57 0.745 (0.586 -- 0.773) Admit # 59 0.759 (0.625 -- 0.791) Admit # 60 0.840 (0.739 -- 0.852) Admit # 61 0.788 (0.571 -- 0.796) Admit # ----- ----- ------------------ --------- # #STEP 3: Inspection of the Prior Relative Efficiency for the items admitted in the pool # # Item Relative 95% Confidence # Efficience interval # ----- ------------------------------ # 2 0.515 (0.531 -- 0.745) # 3 0.763 (0.745 -- 0.973) # 5 0.425 (0.436 -- 0.650) # 7 0.405 (0.447 -- 0.657) # 9 0.692 (0.700 -- 0.915) # 10 0.338 (0.329 -- 0.533) # 12 0.367 (0.397 -- 0.605) # 14 0.596 (0.542 -- 0.799) # 15 0.765 (0.766 -- 0.968) # 18 0.285 (0.304 -- 0.497) # 19 0.634 (0.630 -- 0.851) # 20 0.852 (0.818 -- 1.000) # 22 0.727 (0.729 -- 0.953) # 23 0.668 (0.667 -- 0.878) # 24 0.393 (0.424 -- 0.639) # 25 0.563 (0.577 -- 0.802) # 27 0.647 (0.653 -- 0.878) # 28 0.817 (0.793 -- 1.000) # 29 0.912 (0.902 -- 1.000) # 30 0.234 (0.255 -- 0.435) # 33 0.605 (0.574 -- 0.811) # 34 0.341 (0.369 -- 0.578) # 36 0.211 (0.262 -- 0.454) # 37 0.834 (0.836 -- 1.000) # 39 0.539 (0.544 -- 0.758) # 40 0.527 (0.542 -- 0.768) # 41 0.296 (0.330 -- 0.531) # 43 0.629 (0.652 -- 0.855) # 44 0.311 (0.331 -- 0.524) # 45 0.480 (0.501 -- 0.719) # 47 0.435 (0.423 -- 0.640) # 50 0.832 (0.810 -- 1.000) # 51 0.365 (0.402 -- 0.611) # 52 0.322 (0.340 -- 0.539) # 53 0.313 (0.353 -- 0.553) # 55 0.788 (0.763 -- 0.991) # 57 0.266 (0.239 -- 0.439) # 59 0.572 (0.570 -- 0.791) # 60 0.768 (0.768 -- 0.980) # 61 0.271 (0.327 -- 0.523) # ----- ------------------------------ # ########################################################################################### |
