From the PARcore Data Visualization Studio Landing Page, navigate to this report by selecting the Summary Across Comparisons report icon.   This report is only applicable for requests that include the Analysis Type of Item Analysis - Differential Item Functioning and at least one analysis had sufficient volume to run DIF.  If no analyses in the request had sufficient volume, this report will not appear on the Landing Page.


The bento box menu icon at the top left of the report can be used to navigate back to the Landing Page.
 




When first launching the report it will be empty.  Make initial filter selections in order to visualize the report.  Any filter with a red asterisk indicates it is required to have a selection made before the report can be displayed.


Filters
Refer to Visualize Results for general information about filter navigation for any report.  The filters for this report are:


 

• Analysis - select one analysis sequence number to view at a time.  The analysis sequence number will reflect the sequencing from Request Details for this request.
  • Upon first initializing this dashboard, the filter will display as "(Choose your analysis)" and a selection must be made. 
  • Select the search icon (magnifying glass) and enter search text to see Analysis filter options for the request you are viewing.  Using an asterisk in the search text box will display all possible analyses to choose from.  
  • Once the current analysis results have been reviewed, continue to use the search feature or utilize the slider arrows (< and >) to view each analysis one at a time.  If you only have one analysis to view, the slider arrows will be disabled.
  • The filter description will include the criterion score being analyzed as well as the reference group.  Only one reference group will be displayed at a time.
  • If an analysis sequence in your request had insufficient DIF volume for all items, that Analysis will not appear in the filter for selection.  You can only select analyses where at least some items had sufficient volume to run DIF.

• Refinement Type - select one refinement to view at a time.  
  • Upon first initializing this dashboard, the filter will display as "(Choose refinement type.)" and a selection must be made before any data is visualized.  Use the dropdown filter to display the possible refinements stored for this comparison group.  
  • The available refinement type(s) to choose from depend upon the user selected options for this analysis as well as the analysis results.  The available options are:
    • No Refinement:  This will appear if the user selected "No" to the analysis template option to Refine Criterion Score.  This will be the only refinement option available.
    • None Required:  This will appear if the user selected "Yes" to the analysis template option to Refine Criterion Score but the comparison analysis resulted in no C items and therefore no refinement was performed.  This will be the only refinement option available.
    • Before Refinement:  This will be available if the user selected "Yes" to the analysis template option to Refine Criterion Score and the comparison analysis resulted in at least one C item.  The dropdown menu should show both "Before Refinement" and "After Refinement" as options.  Selecting Before Refinement will display the frequency distributions and summary statistics before refinement was performed. 
    • After Refinement:  This will be available if the user selected "Yes" to the analysis template option to Refine Criterion Score and the comparison analysis resulted in at least one C item.  The dropdown menu should show both "Before Refinement" and "After Refinement" as options.  Selecting After Refinement will display the frequency distributions and summary statistics after criterion refinement was performed.

• Item Scoring:  Users can select which set of items to view based on the item scoring (dichotomous and/or polytomous). 
  • Since dichotomous items and polytomous items display a different set of statistics, users will need to view one set of statistics at a time if  both dichotomous and polytomous items are included in the analysis.  
  • The default selection is "Dichotomous".  If the comparison being viewed contains only polytomous items the filter selection will need to be updated before the table can be rendered.

• Items to Display:  Users can select to which set of items are displayed in the summary table.  
  • There are two options for display:
    • C Flags (selected as default)
    • All Items
  • If the selected comparison did not have any C flags, a message will appear to alert the user that there were no C flags to review.  If you desire to review all items instead, change the radio button to select 'All Items'.

• Display Total Group Statistics:  Users can select whether to display the Total Group statistics (average item score, delta (if applicable), and correlation(s)) if desired.  
  • There are two options for display:
    • No (selected as default)
    • Yes

• A list of items, based on the Items to Display selection, will display in the visualized table.   The focal and reference group labels and counts are shown for every comparison within this analysis sequence number.  If this sequence has more than one focal group, they will all be displayed in this table. 
  • If one or more comparisons did not have sufficient volume to run the DIF analysis, that group will not be displayed.

• The Greatest Flag Across Focal Group(s) displays the highest flag value for this item across all of the comparisons within the selected analysis sequence number.
  • If any item is in more than one assembly unit and the processing option for combined item responses is "Combine Responses by Accession # Across Forms/AUs" each sequence and assembly unit is shown, but the DIF statistics will only display once because they are combined across all the assembly units.    For filtering purposes only the greatest flag across focal groups value will repeat for every assembly unit.   

• If the "Display Total Group Statistics" was selected as Yes, a set of Total Group statistics is displayed after the Greatest Flag Across Focal Groups column.  The column header will detail the counts for the total group.  Separate columns display the average item score, delta (for dichotomous items), and correlation(s) for each item.  If the "Display Total Group Statistics" selection is No, these group of columns will be hidden from the view.
  • If the selected analysis contains some items that did not meet the DIF volume requirements specified in the analysis template, those items will not be displayed unless the Total Group Statistics option is Yes.  The total group statistics for those items is calculated and displayed, but the DIF Category Flag and MH D-DIF or SMD statistic will be blank because it could not be calculated for that item.

• For each comparison listed, the DIF Flag Category and MH D-DIF (if dichotomous) or SMD (if polytomous) statistics for each item is displayed.  These statistics will always display, regardless of the "Display Total Group Statistics" selection.

• If the selected analysis contains both dichotomous and polytomous items use the Item Scoring filter to review one set of results before changing the filter to view the remaining items.

• When viewing an analysis that had refinement, a footnote at the bottom of the report will be included when viewing the refinement (Before or After) that is being stored to external services based on the "Statistics to use for external services" selection made by the user in the analysis template used for this request.