From the PARcore Data Visualization Studio Landing Page, navigate to this report by selecting the Calibration Parameter Estimates report icon.   This report is only applicable for requests that include the Analysis Type of Item Response Theory and include the option to include calibration analyses.


The bento box menu icon at the top left of the report can be used to navigate back to the Landing Page.
 
Go to landing page



Information about the request that is being viewed are displayed at the top, including the request number, admin, test, form ID or package ID, and user form code. 


When first launching the report it will be empty.  Make initial filter selections in order to visualize the report.  Any filter with a red asterisk indicates it is required to have a selection made before the report can be displayed.


Choose Filters


Screenshot of Analysis Filter


Refer to Visualize Results for general information about filter navigation for any report.  The filter for this report is:
     

  • Analysis - select one analysis sequence number to view at a time.  The analysis sequence number will reflect the sequencing from Request Details for this request.
    • Upon first initializing this dashboard, the filter will display as (Choose your analysis) and a selection must be made. 
      • Select the search icon (magnifying glass) and enter search text to see Analysis filter options for the request you are viewing. 
      • Using an asterisk in the search text box will display all possible analyses to choose from.  
    • Once the current analysis results have been reviewed, continue to use the search feature or utilize the slider arrows (< and >) to view each analysis one at a time.  If there is only one analysis for this report the navigation buttons will be disabled.
    • The analysis template name used for this request analysis will also be included in the filter description.


Navigating Tabs in Tableau:

  • There will be individual tabs to navigate to different reports (Bivariate Scatterplot, Item Summary, Problem Item Report, and Profile Report).  The tab being viewed is displayed in white, with blue-font tab label.  Tabs that are available but not being viewed are pale blue with blue-font tab label.  Click into a tab label to view that report.  You can only view one tab at a time.       
           Tabs that are included

        


Bivariate Scatterplot Tab

In this tab you are able to view scatterplots of a-parameters versus b-parameters for all of the items in the selected analysis.  Parameters are rounded to four decimals before plotting.  There will be separate plots for dichotomous items and polytomous items (when applicable).


There is a Highlight Accession Number filter that allows the user to find and highlight a specific accession number in the plot. 
  •  Accession numbers are listed alphabetically but users can enter into the search text box (with the gray font text "Highlight Accession Number) to narrow the selection as needed to find the accession number they want to view.
  • When selecting an accession number, it will automatically highlight that item in the plot(s) below.  
  • To close out of a selected accession number, select the "X" icon in the filter.

       Images for how to navigate and use highlight accession number


A tooltip is available when hovering over a data point to identify the statistic values.  The tooltip also includes information to identify which item(s) are represented by the data point. 
  • There is a potential for item overlap if more than one item has exactly the same a-parameter and b-parameter values (when both are rounded to 4 decimals).   As a result, a frequency count of how many items are represented by the data point and the list of those items is included in the tooltip. 
    • Data points that represent more than one item (have a frequency count > 1) will be slightly brighter/darker as an additional indication to the user.
  • There is a limit to how many items can be displayed within one tooltip (approximately 20).  If the number of items overlapping at one data point reaches the tooltip space limit, a message will appear in the tooltip that "View is too large to show" and the user will not be able to see all the items displayed.  
  • If an item has more than one assembly unit, all assembly unit names and their respective Sequence in AU values will be displayed in separate rows for that item.  The frequency count is not impacted by the multiple assembly units since it is a count of the number of accession numbers represented by that data point.
          Example of the tooltip display:
             Example Tooltip                                             


          Example of a tooltip with more than one assembly units
              Tooltip with Multiple AUs



If desired, utilize the zoom feature to expand the plot to get a closer view of a set of data points.  Refer to Visualize Results for general information about zoom functionality.  



Item Summary Tab

In this tab you are able to see summary statistics for the calibration parameters (Intercept, Intercept SE, Slope, Slope SE, a, b). 
  • By default the "All Items" group of summary statistics will display.  The summaries are separated by Dichotomous and Polytomous when applicable.
 
  • If the Analysis Template option of "Additional Item Summaries" was marked as Yes, the additional summaries by Form, AU, and/or LIC will be available to view by using the "Display by Item Group" radio buttons on the right of the screen.  Only the options chosen in Analysis Templates will be available, so if the "Additional Item Summaries" was No in your template, only the Display By Item Group of "All Items" will be available.  
    • Change the radio button selections to review each Item Group and see a list of all summary statistics for the item group selected.
  
             
             Example Analysis Template Selection                                                         Display by Item Group options 
               Analysis Template - no additional summaries selected                                         Display by Item Group is only All




             Example Analysis Template Selection                                                           Display by Item Group options 
               Analysis Template - additional summaries selected          Display By Item Group is All, AU, and LIC


 
  • The Statistic Rows to Display filter allows the user to modify which summary statistics are displayed.  By default the Number of Items, Mean, SD (N), Maximum, and Minimum are displayed.  Use this filter to display additional summaries (Median, 75th Percentile, 25th Percentile) or remove summaries from the display if they do not need to be reviewed.  Select APPLY to apply any changes desired.
                             Statistic Rows to Display option


Problem Item Report Tab

In this tab you are able to see a summary of how many items in the selected analysis were flagged.   The table displays the following information for the flags:
  • A description of the flag, including the flag letter designation.
  • The user-selected criterion for the flag.   
    • In cases where a given flag has more than one flagging criterion defined in the analysis template (such as separate criterion for Dichotomous and Polytomous) only the flagging criterion that has been met will be displayed. 
    • We cannot display the criterion when no items in the data met the flagging criterion.  In those cases the Criterion cell will be "Flagging criterion not available" and the Number of Flagged Items will be 0.  A footnote will also display to clarify that the flagging criterion is not available for display when either (1) no items met the flagging requirement or (2) the flag was not requested as part of this analysis.
  • The total number of items analyzed will be a total of all items regardless of the scoring type (dichotomous or polytomous). 
  • The number of flagged items and percent of total items that triggered each flag for the selected analysis will also be displayed. 

If the request options indicate that no flagging is necessary, this report is still produced and displays the Flag column (for both a and b) and the Total Number, but criterion will not be available and the number flagged will always be 0.  




Profile Report Tab

In this tab you are able to see a table of all items that were flagged.  Only flagged items are included in this report.  The table displays the following information for the flags:
  • Item identifier information, such as the Accession Number, Scoring Version, Sequence in AU, and Assembly Unit Name.
    • Items are sorted and displayed by ascending location number.
    • If an item has more than one assembly unit associated with it, each Assembly Unit and each Sequence in AU for that item will display in the cell for their respective columns.  If the list is so long that all values cannot display in one cell, you can hover over the cell and a tooltip will appear that shows the full list without truncation. The tooltip only appears when the cell values are truncating due to space limitation.
  • Which flag(s) were triggered for the item.  
    • Currently for calibration users have the option to indicate flagging criterion for only two types of flags:  a-parameter flag, b-parameter flag.   
    • The letter designation for the flag that was triggered will display.  This will be one of the following:
      • a = only the a-parameter flagging criterion was met
      • b = only the b-parameter flagging criterion was met
      • a b = both the a-parameter and b-parameter flagging criterion were met.
  • The a value and b value for the displayed item.

If no flagging was request or if no items met the flagging criterion, a message will display that no items were flagged in this analysis.  

 

Additional statistics are produced and available to users beyond what is presented in the Data Visualization Studio.  Refer to the RTSSTAT API and Download Statistics help documentation for more information regarding accessing all of the available statistics produced.