A Statistical Retrieval Rule (SRR) indicates which available statistics record and which statistics fields within that record should be retrieved for items with predefined filter criteria used for resolving and returning statistics. The SRR also determines the order in which the statistics are presented. The SRR Service houses all statistics for both PARcore and GENASYS. Statistics can be retrieved in PARcore by running an SRR or IBIS® Application users can access the statistics via an SRR.
SRRs are created by PAR staff with input from ALTRD staff at the program level and apply to all tests within the program. The SRRs available for use are configured according to the Program and Assessments requirements for which records and which fields should be referenced for specific purposes, such as Post-Administration reviews or Assembly of new forms.
A Statistical Retrieval Request contains the following components:
• The list of statistics to be displayed
• The tags assigned to each statistics record (e.g., Test Assembly, PIA, FIA)
• The rules used to acquire the statistics (e.g., retrieve statistics by a certain run date)
Table of Contents
Tagging Statistics
Create and Manage SRRs
Run SRRs
Statistical Retrieval Rules in the IBIS Application
The IBIS Application will query eSTOR to retrieve statistics. The primary functions which use SRRs to retrieve statistics are:
1. Test Assembly: Build a new section or form from a collection of items, item meta data, and statistics. When adding items to an item collection or AU, the AD staff selects the appropriate SRR to retrieve statistics for the items selected. They may also optionally supply additional selection criteria (testing program, test, administration, form) when the SRR is invoked. Therefore, the same test assembly SRR can be used to get statistics for both equating items and non-equating items.
2. Item Review: For the items from a selected administration, view item statistics and flag items. Currently, item review uses a default SRR. The default SRR returns a standard set of Item Analysis (IA), Differential Item Functioning (DIF), and IRT statistics. In the unusual case that a test requires additional statistics to perform item review, then a user can create additional PIA and/or FIA item review SRRs. These statistics can be requested during item review by selecting an additional SRR and viewing those statistics. Most tests will not require any additional item review SRRs beyond the default.
3. View item statistics history: For an individual item, see other instances of statistics from different administrations. When viewing statistics history, the AD staff selects the appropriate SRR to see all instances of statistics which match the tags used in that SRR. If AD selects the test assembly SRR from the statistics history tab, they will be able to see all instances of statistics which meet the selection criteria of the test assembly SRR. Some tests may load and tag statistics which are not suitable for test assembly, but which may be of interest for other purposes. For example, they may load statistics which are based on a subscore criterion as well as statistics based on a total criterion. Or they may load statistics which are suitable for PIA but not for test assembly. In order to see both test assembly and informational statistics in the statistics history tab, it will be necessary to create an SRR which does not specify any tags.
4. Inventory: Create an Excel worksheet of item meta data and statistics. This function is very similar to creating a test assembly worksheet; however, the user may select a different SRR (depending on the intended use of the exported spreadsheet). A typical export may be for the purpose of creating a spreadsheet showing the inventory of items available for test assembly. This worksheet will contain both item meta data and statistics and can be analyzed
to direct future item development. A test may create an inventory SRR if the list of statistic names needed for inventory is larger than or different from the list of statistics used for test assembly. Otherwise, there is no need to create an additional inventory SRR.
5. Item Lifecycle Evaluation: If your program/test is configured for Item Lifecycle in the IBIS application, the Item Lifecycle SRR is used to retrieve the statistics that are tagged as Item Lifecycle for use in the Item Lifecycle rule evaluation which determines the proposed next lifecycle state of an item.
Minimally, when a test is configured in the IBIS Application, the ALTRD and PAR staff will define an appropriate test assembly SRR which will work in conjunction with an extended tagging system either in PARcore or GENASYS to display the relevant statistics in the IBIS Application. Most tests will also set up a statistics history SRR with no tags in order to view both test assembly and informational statistics together. If additional SRRs are needed for the other two purposes above, they should also be defined. For more information, refer to the IBIS® Reference Library Working with Item Statistics.
SRRs are created by PAR staff with input from ALTRD staff at the program level and apply to all tests within the program. The SRRs available for use are configured according to the Program and Assessments requirements for which records and which fields should be referenced for specific purposes, such as Post-Administration reviews or Assembly of new forms.
A Statistical Retrieval Request contains the following components:
• The list of statistics to be displayed
• The tags assigned to each statistics record (e.g., Test Assembly, PIA, FIA)
• The rules used to acquire the statistics (e.g., retrieve statistics by a certain run date)
Table of Contents
Tagging Statistics
Create and Manage SRRs
Run SRRs
Statistical Retrieval Rules in the IBIS Application
The IBIS Application will query eSTOR to retrieve statistics. The primary functions which use SRRs to retrieve statistics are:
1. Test Assembly: Build a new section or form from a collection of items, item meta data, and statistics. When adding items to an item collection or AU, the AD staff selects the appropriate SRR to retrieve statistics for the items selected. They may also optionally supply additional selection criteria (testing program, test, administration, form) when the SRR is invoked. Therefore, the same test assembly SRR can be used to get statistics for both equating items and non-equating items.
2. Item Review: For the items from a selected administration, view item statistics and flag items. Currently, item review uses a default SRR. The default SRR returns a standard set of Item Analysis (IA), Differential Item Functioning (DIF), and IRT statistics. In the unusual case that a test requires additional statistics to perform item review, then a user can create additional PIA and/or FIA item review SRRs. These statistics can be requested during item review by selecting an additional SRR and viewing those statistics. Most tests will not require any additional item review SRRs beyond the default.
3. View item statistics history: For an individual item, see other instances of statistics from different administrations. When viewing statistics history, the AD staff selects the appropriate SRR to see all instances of statistics which match the tags used in that SRR. If AD selects the test assembly SRR from the statistics history tab, they will be able to see all instances of statistics which meet the selection criteria of the test assembly SRR. Some tests may load and tag statistics which are not suitable for test assembly, but which may be of interest for other purposes. For example, they may load statistics which are based on a subscore criterion as well as statistics based on a total criterion. Or they may load statistics which are suitable for PIA but not for test assembly. In order to see both test assembly and informational statistics in the statistics history tab, it will be necessary to create an SRR which does not specify any tags.
4. Inventory: Create an Excel worksheet of item meta data and statistics. This function is very similar to creating a test assembly worksheet; however, the user may select a different SRR (depending on the intended use of the exported spreadsheet). A typical export may be for the purpose of creating a spreadsheet showing the inventory of items available for test assembly. This worksheet will contain both item meta data and statistics and can be analyzed
to direct future item development. A test may create an inventory SRR if the list of statistic names needed for inventory is larger than or different from the list of statistics used for test assembly. Otherwise, there is no need to create an additional inventory SRR.
5. Item Lifecycle Evaluation: If your program/test is configured for Item Lifecycle in the IBIS application, the Item Lifecycle SRR is used to retrieve the statistics that are tagged as Item Lifecycle for use in the Item Lifecycle rule evaluation which determines the proposed next lifecycle state of an item.
Minimally, when a test is configured in the IBIS Application, the ALTRD and PAR staff will define an appropriate test assembly SRR which will work in conjunction with an extended tagging system either in PARcore or GENASYS to display the relevant statistics in the IBIS Application. Most tests will also set up a statistics history SRR with no tags in order to view both test assembly and informational statistics together. If additional SRRs are needed for the other two purposes above, they should also be defined. For more information, refer to the IBIS® Reference Library Working with Item Statistics.