The following are details specific to Analysis Type: Item Analysis - Differential Item Functioning.
When you are creating, editing, or duplicating an analysis template for Item Analysis - Differential Item Functioning, the following sub-menus and options are available:
Processing Options
- Combine Item Responses
- single select option to configure if responses should be combined by accession # across forms/AUs or analyze the response within the specific form/AU
- Edit Equivalent Values
- single select option to edit equivalent values by type or value, if required
- Max Popular Wrong Responses
- single selection option to configure the maximum number of wrong responses for calculation and graphing purposes
- Automated Constructed Response Scoring Contributory Score
- single select option for treatment of automated constructed response scoring contributory score by category weight or precise item score.
- Refine Criterion Score
- toggle Yes/No
- if "No", no additional options are displayed
- if "Yes"
- Statistic to use for external services (for example, statistics to use in IBIS)
- single selection option 'Before Refinement' or 'After Refinement'
- Statistic to use for external services (for example, statistics to use in IBIS)
Note : Attention Message indicating that Differential Item Functioning Analysis does not traditionally include Omitted and Not Reached responses and to confirm with your program before selecting to include them.
- Treatment of Omitted Selected Responses
- toggle to exclude or include omitted Selected Responses.
- Treatment of Omitted Constructed Responses
- toggle to exclude or include omitted Constructed Responses.
- Treatment of Not Reached Selected Responses
- toggle to exclude or include not reached Selected Responses.
- Treatment of Not Reached Constructed Responses
- toggle to exclude or include not reached Constructed Responses
The image below shows all default selections for Processing Options:
Comparisons
- Group Requirements
- Smaller of Reference or Focal
- manual entry box to indicate minimum sample/volume size of Reference or Focal group to run analysis.
- Reference + Focal
- manual entry box to indicate minimum sample/volume size of the combination of Reference and Focal group to run analysis.
- Note: must be a number greater than 'Smaller of Reference or Focal' entry.
- determined by
- single select option for method of determining the minimum sample/volume size selections.
- Smaller of Reference or Focal
- Comparisons Table
- Reference and Focal Group Selections:
- Group - single select drop down (with search feature) that includes listing of all group IDs.
- Values displayed in alphabetical order by Group ID
- Reference and Focal Test Taker Selection Queries - single-select drop down (with search feature) that includes all Test Taker Selection queries configured for the program, with the exception of any queries with mathematical expressions.
- Query type must be Test Taker Selection
- Available values displayed in alphabetical order.
- See Queries for additional details.
- Group - single select drop down (with search feature) that includes listing of all group IDs.
- Additional Focal groups - when multiple focal groups are to be compared with the same reference group, select '+' in the Focal column to add row(s), and select Group and Test Taker Selection Query for each focal group.
- If a Focal group is added, save will be disabled until all required selections of Group and Test Taker Selection Query have been made.
- Once a Focal group row is added, there is a Delete icon '-' displayed next to each row in the Focal column. Any row can be deleted. The Delete icon disappears if only one row remains.
- A minimum of one comparison (one Reference group and one Focal group) is required for an Item Analysis - Differential Item Functioning template to be saved.
- Additional Comparisons - when multiple Reference groups are needed, select '+' in the Add/Delete Comparisons column to add another comparison, and select Group and Test Taker Selection Query for each Reference and Focal group
- If a Comparison is added, save will be disabled until all required selections of Group and Test Taker Selection Query have been made
- Once a comparison row is added, there is a Delete icon (trashcan) displayed in the Add/Delete Comparisons column next to each comparison. Any comparison can be deleted. The Delete icon disappears if only one comparison remains.
- Validations
- Reference and Focal Group ID cannot be the same within a comparison.
- If adding additional Focal Groups, duplicate use of Focal Group within a comparison is not allowed.
- Reference and Focal Group Selections:
Validation messages for Comparisons:
Statistics Tags
Once all required selections in the Analysis Template tab have been entered, the Statistics Tags tab becomes available.
- Item Analysis - Differential Item Functioning templates have item-level tags and test-level tags available to be configured.
- Tags selected in the analysis template are the default tags used in an analysis request using that template. Tag overrides can be performed for the following:
- By analysis or LIC in Test Settings (BEFORE a request is generated). See Test Settings for Item Analysis - Differential Item Functioning for more information.
- By item (selecting by item, LIC, or AU) in Request Details (AFTER a request is run). See Tag Statistics for more information.
Viewing an Item Analysis - Differential Item Functioning analysis template displays the options as seen below:
Processing Options
Comparisons
In the Statistics Tags tab:
- Check marks show default item-level and test-level tag selections configured during template creation.
For Test Settings details see Test Settings for Item Analysis - Differential Item Functioning
For Request Details information see Request Details
For Statistics and Visualizations available for Item Analysis see Visualize Results
For Statistics Tags definitions and uses see Tagging Statistics