Item Analysis - Classical

For general instructions, navigating, and actions related to Test Settings see Test Settings

The following are details specific to Analysis Type: Item Analysis - Classical.

Editing Test Settings for an Item Analysis - Classical Template
Users with Super Admin or PAR Admin role for the program selected in Global Navigation will be able to edit Test Settings for an Item Analysis - Classical template.

1. To open an analysis template in Edit mode do one of the following:

Select Edit under the Actions column when viewing the list of tests
- A list of all approved and enabled Item Analysis templates applied to the selected test will display in the Analysis Settings tab under "Item Analysis - Classical"
- Select a template
Select the Test Name hyperlink
- A list of all approved and enabled Item Analysis templates applied to the selected test will display in the Analysis Settings tab under "Item Analysis - Classical"
- Select a template
- The template will open in view-mode (if nothing has been configured yet for the template, "No Values Selected" will display in the table)
- Select the Edit button in the upper right hand corner

2. Criterion Score and LIC(s) (both are required fields)

Criterion Score - a single-select drop down that lists all scores that have been configured for the selected test.
- the list of scores is displayed in alphabetical order by Score Name
LIC(s) - a multi-select drop down that lists all LICs that have been configured for the selected test
- the list of LICs is displayed in alphabetical order.
- a Select All button is available to select all LICs

2. Tag Statistics (optional)

Once you have completed the selections for Criterion Score and LIC(s), Statistics Tags (tag icon) will be enabled.

This allows the user to override the Default Statistics Tags that have been previously selected at the Analysis Template level.
Overriding can be done at both the analysis level and/or the LIC level.

To override tags:

Select the Statistics Tag icon
The Statistics Tag modal displays
- The Criterion Score Analysis is the first row displayed followed by LIC(s) listed in alphabetical order
Clicking on the Analysis row will display any tag selections that are inherited from the tags selected in the analysis template (all checkboxes will be disabled)
- At the Analysis level, Item Level and Test Level tags can be overridden
- If you want to override tags at the Analysis level, move the Override toggle to ON and make your selections
- Any selections made will REPLACE the default tags selected in the Analysis Template
Clicking on any of the LIC rows will display a blank screen on the right
- At the LIC level, only Item Level tags are able to be overridden
- If you want to override tags at the LIC level, move the Override toggle to ON and make your selections
- Any selections made at the LIC level will REPLACE default tags selected in the analysis template and will OVERRIDE any tags selected at the Analysis level

NOTE: If items are shared between LICs, and more than one LIC has tag overrides applied, the last LIC in the list will take precedence and the selected tags for that LIC will be applied to the item.

Select Apply after all tag selections have been made
If Cancel is selected no selections will be saved

3. Use Equate Item Statistics (optional)

Within an Item Analysis - Classical template, you're able to use a reference univariate frequency distribution (CUSFDS) to be used for equating item statistics purposes.

NOTES:

This analysis should only be performed using a scaled score frequency distribution
If Compare Deltas has been selected in the analysis template, this option will not be displayed for the test in Test Settings.

Select the checkbox next to 'Use Equate Item Statistics' to open a modal where you'll select a request to use.

Enter a valid request number and select Search
A list of request/analysis sequence numbers will display including the following context information:
- Request ID is the combination of program, request number, and analysis sequence number
- Admin
- Test
- Form
- Description
- Statistical Model - Restricted to only yield results of CUSFDS (Univariate Frequency Distribution)
- Origination
- Run Date

Select the frequency distribution and click Apply
- You will be returned to Edit Test Settings, with your selection now being displayed.
- An Edit button is now enabled if you need to edit your selection in the future.
If Cancel is selected, you will not associate any frequency distribution and you will be returned to Edit Test Settings.

4. Additional Criterion Score(s)

When multiple Item Analysis - Classical analyses are desired within a test, ie. use of another criterion score, the user will need to select '+ Add Row' and make the desired selections.

Each row will represent a separate analysis sequence number within a request.
If a row is added, Save will be disabled until the user has completed all required selections of Criterion Score and LIC(s).
Once a row is added, there is a Delete icon enabled next to each row.
Note that a minimum of one row is required for an Item Analysis - Classical test settings to be saved and used in request generation.

5. ISD Reliability Score and LIC (optional)

If the Analysis Template selected is configured to produce reliabilities for an ISD Form, an additional table will display in Test Settings for the selections of a Score and LIC for the calculation of the reliability.

Score - a single-select drop down that lists all scores that have been configured for the selected test.
- the list of scores is displayed in ascending order by Score ID
LIC - a single-select drop down that lists all LICs configured for the selected test
- the list of LICs is displayed in alphabetical order
If more than one ISD Reliability Score-LIC combination needs to be produced, the '+Add Row' button can be used to include another row.
- Once a row is added, there is a Delete icon enabled next to each row.
- At least one row needs to be included to be able to save successfully

NOTE: ISD Reliability is added on to the analysis sequence number of the general Item Analysis - Classical. In the event you have multiple analysis sequences, ISD Reliability will be added to each of the analysis sequence numbers.

6. Apply Scores & LIC(s) to Test(s) and Include Tags (optional)

If the selected analysis template has been applied to other tests, you can apply Test Settings selections to any of the additional tests using the "Apply Score(s) and LIC(s) to Test(s)." (next to Cancel and Save buttons)

Select the checkbox for "Apply Score(s) & LIC(s) to Test(s)"
- Upon selecting user can "Include Tags", which will then include the Tags selections in the Save and apply process.
The "Save" button changes to "Save and Apply"

Select Save & Apply to display a modal
The list of tests applied to the analysis template selected will display in sequential order.
- Test Code is sortable and filterable
- Test Name is filterable
Select the test(s) in the modal that you wish to apply selections to and select Next.
- A count of tests selected is displayed at the bottom.

If no validation warnings or errors are found, a confirmation modal listing the current test and selected additional tests will display, along with options of 'Go Back', 'Cancel' and "Save & Apply'.

Selecting 'Go Back' will return you to the list where you can revise your test selections
Selecting 'Cancel' will return you to the Test Setting screen without saving any changes
Selecting 'Save & Apply' will save the Test Settings for all additional selected tests

Validation checks are run to ensure that existing values aren't overwritten, and the selected score(s) and LIC(s) exist for the selected test(s).

Possible Errors:

"None of the selected score(s)/LIC(s) exist for this test".

Possible Warnings:

"One or more of the selected score(s)/LIC(s) do not exist for this test. These will not be applied."
"Score(s)/LIC(s) have already been selected for this test. Existing values will not be overwritten. Selected score(s) /LIC(s) will be added."

In the event of any warnings or errors, the table of tests will display these messages inline.

Any tests with Errors will move to the top of the list.
- Tests with Errors must be deselected to proceed/enable the 'Next' button.
Warnings will be listed after any Errors.
- Warnings can be ignored/bypassed and user can select 'Next' to apply the Test Settings selections.
Selecting 'Next' will prompt a confirmation modal listing the current test and selected additional tests, along with options of 'Go Back', 'Cancel' and "Save & Apply'.

7. Save, Cancel, and Unsaved Changes

When you've finished entering all required information, and 'Save' is selected, a banner will display at the top of the screen:

Selecting 'Cancel' will result in the following:

If no selections were made, you'll be returned to the list of tests under Test Settings
If selections were made but not saved, selecting 'Cancel' or anywhere else on the screen will display a warning as seen below:

Selecting 'Continue' will discard all selections and return you back to the list of tests under Test Settings
Selecting 'Cancel' will return back to edit mode for the template where you can save or continue making selections.

While in Edit mode you can navigate to other templates while remaining in edit mode, or select the breadcrumb 'Test Settings' to return to the list of tests for the selected program.

Viewing Test Settings for an Item Analysis - Classical Template
All users with access to the program selected in Global Navigation will be able to view Test Settings for an Item Analysis - Classical template.

To open an analysis template in View mode:

Select the Test Name hyperlink
- A list of all approved and enabled Item Analysis templates applied to the selected test will display in the Analysis Settings tab under "Item Analysis - Classical"
- Select a template
- The template will open in view-mode (if nothing has been configured yet for the template, "No Values Selected" will display in the table)

One or two tables are displayed (depending on the options selected in the analysis template):

1. Score(s) & LIC(s) - "Template name"

Four columns display with selections in the same order as they were selected while editing test settings for the template.
- Criterion Score
- LIC(s)
- Equate Item Statistics Req ID (N/A if none was selected)
- Tags
  - Icon that upon selecting will display a read only version of tag modal.

2. ISD Reliability (if applicable)

If the Analysis Template selected is configured to produce reliabilities for an ISD Form, an additional table will display for the selections of a Score and LIC for the calculation of the reliability.
Two columns display with selections in the same order as they were selected while editing test settings for the template.
- Score
- LIC

** If you have a PAR Admin role for the selected program, or a Super Admin role, an Edit button will display in the upper right-hand corner.

While in View mode you can navigate to other templates, or select the breadcrumb 'Test Settings' to return to the list of tests for the selected program.