From the PARcore Data Visualization Studio Landing Page, navigate to this report by selecting the Parameters and Statistics report icon.   This report is only applicable for requests that include the Analysis Type of Item Response Theory and include the option to Produce TBLT.


The bento box menu icon at the top left of the report can be used to navigate back to the Landing Page.
 
Go to landing page



Information about the request that is being viewed are displayed at the top, including the request number, admin, test, form ID or package ID, user form code, and system form code. 


When first launching the report it will be empty.  Make initial filter selections in order to visualize the report.  Any filter with a red asterisk indicates it is required to have a selection made before the report can be displayed.


Choose Filters


Screenshot of Analysis Filter


Refer to Visualize Results for general information about filter navigation for any report.  The filter for this report is:
     

  • Analysis - select one analysis sequence number to view at a time.  The analysis sequence number will reflect the sequencing from Request Details for this request.
    • Upon first initializing this dashboard, the filter will display as (Choose your analysis) and a selection must be made. 
      • Select the search icon (magnifying glass) and enter search text to see Analysis filter options for the request you are viewing. 
      • Using an asterisk in the search text box will display all possible analyses to choose from.  
    • Once the current analysis results have been reviewed, continue to use the search feature or utilize the slider arrows (< and >) to view each analysis one at a time.  If there is only one analysis for this report the navigation buttons will be disabled.
    • The analysis template name used for this request analysis will also be included in the filter description.


Navigating Tabs in Tableau:

  • There will be individual tabs to navigate to different reports (Quasi-Newton, Parameter Comparison, and ICC Comparison).  The tab currently being viewed is displayed in white, with blue-font tab label.  Tabs that are available but not being viewed are pale blue with blue-font tab label.  Click into a tab label to view that report.  You can only view one tab at a time.       
           Image of tab structure

        


Quasi-Newton Tab

In this tab you are able to view the transformation parameters for the Quasi-Newton Minimization that was performed.  The information provided includes:
  • Number of iterations taken to converge
  • Function value (mean square difference between TCCs)
  • Slope
  • DFDR
  • Intercept
  • DFDM
  • Final line formulas with values for both reference and new. 




Parameter Comparison Tab

In this tab you are able to view the parameter comparison statistics between the new and reference common items. 


"Item Scoring" filter:  For this tab only, there is an additional filter that allows users to see the list of common items separately for dichotomous and polytomous items, if applicable. 
  • Only one item scoring set of common items can be viewed at a time.  By default the dichotomous items are displayed.   
  • If the analysis contains no dichotomous items, users will need to select the radio button for "Polytomous" in order to render the table. 

                 Item Scoring:  both dichotomous and polytomous        Item Scoring filter with both   
      
                 Item Scoring: only dichotomous items                          Item Scoring filter - dichotomous   
          
                 Item Scoring: only polytomous items                           Item Scoring filter - polytomous



The table displays the following information:
  • New form context information for each common item:
    • Item Number - a sequential numbering of common items, ordered based on the order they were input into the PARSTAT system.
    • Location Number - the sparse matrix number for the item, which is derived during calibration by sorting by form and then accession number.  
    • Accession Number - the scoring version is also appended to the end of the accession number
    • Sequence and Assembly Unit Name - this is a concatenation of the Sequence in AU, which is presented in parentheses, and the Assembly Unit Name. 
      • Hover over the cell to see the full assembly unit name.  If the item is present in more than one assembly unit, each sequence and assembly unit will be displayed in the tooltip.  When there is more than one assembly unit, they are displayed in order by the item position in form value and then alphabetical AU name.  
  • Reference form context information for each common item:
    • Item Number
    • Accession Number - the scoring version is also appended to the end of the accession number
    • The assembly unit information (Assembly Unit Name and Sequence in AU) is not available for the reference form.
  • Comparison Flags 
    • This column only appears in the display if the user selected to include "Comparison of Parameter Estimates" in the request details for Flag Options and one or more of the common items had triggered a flag.
    • Common items that had a comparison flag triggered are indicated with a "Y".
    • All flagging is done by comparing the user-entered criterion against the relevant unrounded statistic value(s).
    • The statistic that triggered the flag (a-parameter difference, b-parameter difference, transformed a-parameter, reference a-parameter) will be colored with a red font.
    • If the analysis had no Comparison of ICC flags triggered this column does not appear and no statistics are marked with red font.  
    • Only the Comparison of Parameter Estimates flags are indicated in this report.  Items that triggered other types of flags (a-parameter, b parameter, Comparison of ICC) are displayed in other reports (Problem Items, ICC Comparison).
  • a-parameter statistics for each common item:
    • Transformed New
    • Reference
    • Difference
  • b-parameter statistics for each common item:
    • Transformed New
    • Reference
    • Difference
  • (b-d)-parameter statistics for each common item:  Only relevant for polytomous items.  
    • Transformed New
    • Reference
      • If the reference statistics are coming from a legacy request (either from the database or from a previous request directly) some item types, such as Speech items, did not have (b-d) statistics stored and therefore those fields will be blank.  The difference values are stored and presented for all item types.    
    • Difference

At the bottom of the report to the left we display the following summary statistics.  The statistics will be calculated separately for dichotomous items and polytomous items, when relevant:
  • Root Mean Square Deviations between item parameter estimates
  • Mean Absolute Deviations between item parameter estimates
  • Mean Deviations between item parameter estimates

At the bottom of the report to the right of the summary statistics is a footnote with information about the criterion set for Comparison flagging.
  • For each flag type (a-parameter difference, b-parameter difference, transformed a-parameter, reference a-parameter):
    • If at least one item is flagged, the footnote details will display what the flagging criterion was.
    • If no items were flagged or if the Comparison of Parameter Estimates flagging was not requested in the request details, the footnote will indicate that the flag was not triggered.  If needed, refer back to the request details to determine what, if any, flag criterion was used.  
  • Please be aware that the a-parameter difference and b-parameter difference flags are triggered based on the absolute value of the difference, but the actual difference values (not absolute) are displayed in the "Difference" columns.






ICC Comparison Tab

In this tab you are able to view the ICC comparison statistics between the new and reference common items. 
The table displays the following information:
  • New form context information for each common item:
    • Item Number - a sequential numbering of common items, ordered based on the order they were input into the PARSTAT system.
    • Location Number - the sparse matrix number for the item, which is derived during calibration by sorting by form and then accession number.  
    • Accession Number - the scoring version is also appended to the end of the accession number
    • Sequence and Assembly Unit Name - this is a concatenation of the Sequence in AU, which is presented in parentheses, and the Assembly Unit Name. 
      • Hover over the cell to see the full assembly unit name.  If the item is present in more than one assembly unit, each sequence and assembly unit will be displayed in the tooltip.  When there is more than one assembly unit, they are displayed in order by the item position in form value and then alphabetical AU name.   
 
  • Reference form context information for each common item:
    • Item Number
    • Accession Number - the scoring version is also appended to the end of the accession number
    • The assembly unit information (Assembly Unit Name and Sequence in AU) is not available for the reference form.
 
  • Comparison Flags 
    • This column only appears in the display if the user selected to include "Comparison of ICC" in the request details for Flag Options and one or more of the common items had triggered a flag.
    • Common items that had a comparison flag triggered are indicated with a "Y".
    • All flagging is done by comparing the user-entered criterion against the relevant unrounded statistic value(s).
    • The statistic that triggered the flag (Unweighted Maximum Difference and/or Weighted Root MSE) will be colored with a red font.
    • If the analysis had no Comparison of ICC flags triggered this column does not appear and no statistics are marked with red font.  
    • Only the Comparison of ICC flags are indicated in this report.  Items that triggered other types of flags (a-parameter, b parameter, Comparison of Parameter Estimates) are displayed in other reports (Problem Items, Parameter Comparison).
  • Unweighted statistics for each common item:
    • Root MSE
    • MAD
    • Maximum Difference
    • Mean
    • S.D.
  • Weighted statistics for each common item:
    • Root MSE
    • MAD
    • Mean
    • S.D.

At the bottom of the report to the left we display the following unweighted and weighted summary statistics.  The statistics will be calculated separately for dichotomous items and polytomous items, when relevant:
  • Mean Absolute Difference between ICCs averaged over items
  • Root Mean Square Error between ICCs averaged over items
  • Mean Deviations between ICCs averaged over items.

At the bottom of the report to the right of the summary statistics is a footnote with information about the criterion set for ICC Comparison flagging.
  • For each flag type (Unweighted Maximum Difference and Weighted Root MSE):
    • If at least one item is flagged, the footnote details will display what the flagging criterion was.
    • If no items were flagged or if the Comparison of ICC flagging was not requested in the request details, the footnote will indicate that the flag was not triggered.  If needed, refer back to the request details to determine what, if any, flag criterion was used.  
 

Additional statistics are produced and available to users beyond what is presented in the Data Visualization Studio.  Refer to the RTSSTAT API and Download Statistics help documentation for more information regarding accessing all of the available statistics produced.