Performance Criteria
Performance criteria within ACMS are managed using 'Criteria Sets'.
New Criteria Sets can be created in the Criteria Admin of the Criteria Set option on the dropdown menu. After selecting 'Add' the following page appears.
Here, you can complete all the relevant information needed:
The Version and Status fields ensure that people are not assessed against out-of-date criteria. There are 4 possible statuses:
- Influx: criteria set can be edited
- Sealed: criteria set cannot be edited
- Deprecated: criteria set is no longer in use
- Deleted: criteria set has been deleted from the system
It is recommended that all Criteria Sets should be 'Sealed' once they are created, as this prevents sets from being changed accidentally. This could result in candidates being signed off on something they haven't been assessed against. If Criteria Sets do need to be edited, admins can right click on a set and select 'Edit' to then change the status of the set to 'Influx.'
The Level Set refers to the set of marks that can be given during an assessment. There is an AssessTech Level Set available by default, however, custom assessment levels can also be created/used.
Editing an existing Criteria Set
It is important that a Criteria Set that is currently in use by candidates is not edited. If a Criteria Set needs to be edited, right click the Criteria Set and select 'Copy'.
Once a new 'Version Number' is entered, a copy of the original Criteria Set will be created. It is then possible to right click the new version and select 'Edit' before making the necessary changes.
A new Schedule Template will also need to be created so that once candidates finish their cycle with the old Criteria Set, they can be assigned to a Schedule with the new Criteria Set.
Criteria Units
Once a new Criteria Set has been submitted, Criteria Units can be added.
Double click the Criteria Set and select 'Add' under the Criteria Unit Admin.
Here you can add the Code, Order, Risk Level and Description of the Unit:
The Code and Order control the display way in which Criteria Units are displayed on ACMS and aAssess. The Risk Level defines the level of risk associated with the Criteria Unit and the Description is a free text field in which you can outline the aim of this Criteria Unit.
Criteria Items
Once a new Criteria Unit has been submitted, Criteria Items can be added.
Double click the Criteria Unit and select 'Add' under the Criteria Item Admin.
This will open the following page, in which you can add information about the questions a candidate will be assessed against:
Code
Auto-populates to the next relevant code when adding multiple criteria items.
Type
Defines the action that the candidate must complete to demonstrate their competence e.g. demonstrate, knowledge.
Order
Simply defines the order in which the criteria items are displayed.
Risk level
Defines the the level of risk for each criteria item. This is useful when analysing candidates' performance against criteria.
Supplemental
Indicates whether the criteria item is supplemental, and therefore, not assessable.
Text tab
Used to define on which action/question the candidate will be assessed.
Default Assessment Comments tab
Comments can be added through the 'Comments Admin'.
Supplemental Criteria tab
Supplemental criteria can be added to the criteria item by dragging the criteria from 'Unassigned' to 'Assigned' or vice versa.
Schedules (Assessment Planning)
Schedules on ACMS consist of two elements:
- Schedule Periods;
- Schedule Templates.
These elements define when assessment windows occur and what needs to be assessed during the windows, respectively.
Schedule Type
The Schedule Type specifies the assessment periods and when they should occur.
This is done by entering:
- Offset – The number of days from the start of the schedule that this period begins
- Number of days – The number of days that this period lasts
This can be done by following this pathway:
Menu → Schedule → Schedule Type → Schedule Type Admin → 'Add' → Schedule Period Admin → Enter offset and number of days
Flex schedules
This are designed to ensure that the final period of the Schedule goes to the Schedule end date e.g. if the Schedule is 100 days long but the Schedule periods only equal 90 days in length, a Flex Schedule will extend the final period so that it finishes on the 100th day of the Schedule.
It is important that the Schedule is activated on the Dashboard and CMS which is completed using the drop-down boxes under the Schedule Type Property tab.
Schedule Template
The Schedule Template is then created to outline which criteria should be assessed during which period. For example, during an Unobtrusive Assessment, you will not be focusing on asking policy questions. This ensures that the Assessor is shown the questions that will be most pertinent.
This can be created via the Schedule Template admin under Menu → Schedule → Schedule Item
A Schedule Template Wizard is used to create a new schedule, following four steps:
- Assign a Schedule Type
- Select a Criteria Set
- Select the Criteria Units that will be assessed during the Schedule
- Assign the selected Criteria Units to the Schedule Periods
Adding a Schedule
Adding a Schedule to a candidate is done via the People Search. On the homepage, you can search for the person you are looking for and all relevant safety critical details will be displayed.
From here, you can click on the CMS tab to be shown Competence Management information such as Schedules and Assessments.
To add a Schedule, drop down the Schedule Admin and select 'Add' and then pick the relevant competence cycle for the user and their start date (this can be configured to start from their competence date, today or next month under Schedule Type → Schedule Type Property → Default Start):
Once the Schedule has been added, they are ready to be assessed either directly on ACMS or via aAssess.
Carrying out Assessments
Assessments on ACMS
Search for the candidate you would like to assess using People Search
Navigate to their Assessment sub-tab under CMS
Select 'Start Assessment' in Assessment Admin
Follow the steps in the Wizard
Select schedule you would like to assess
Select the period you would like to assess (it is possible to complete an assessment without selecting a period, if you wish to do an ad-hoc assessment)
Select start and end date/time (these fields are auto-populated)
Once an assessment has been created the Assessment Details page will immediately load
Assessment Details Page
The different tabs on this page enable you to enter all relevant data in order to complete an assessment:
Items
This tab enables Assessors to mark criteria and leave comments
Code: this displays the Criteria Unit being assessed. Criteria Items which have associated Supplemental Criteria are indicated with a + icon. This can be expanded to detail the Supplemental Criteria
Criteria column: this displays the tasks/questions the candidate is being assessed against
Level: this displays the level assigned by the Assessor
Comment: a tick mark will appear in this column once a comment has been left against the Criteria Item
Quick assess: this displays the Level Set for the Criteria Item
Criteria are assessed by selecting a level under the Quick Assess column.
Selecting a standard level (e.g. green) will simply result in the Level column changing to Green.
Selecting any other level will prompt the Assessor to leave a comment. Once a comment has been submitted a tick will appear in the Comment column.
It is possible to save a Criteria Item without a comment, however, you will be unable to complete the assessment if Criteria Items are found to be below or above standard level (e.g. Green) without comment.
Waypoints
Waypoints are optional.
Typically, Waypoints are used to track changes in location when completing Driver assessments, however, it is also possible to use this page to track other changes in location (e.g. classroom to office).
Notes
This is simply a free text box which can be used to add any additional or live notes during the assessment.
Candidate
This tab can be used to enter information about whether the candidate was present during the assessment, whether they have any objections, and provides a field for candidates to leave comments.
The Start and End PIN Validated fields are used when completing assessments via aAssess.
If an Assessor states that the candidate is present for the assessment, then the app will prompt the candidate to enter their ACMS password. Once a candidate has signed an assessment on ACMS the End Pin Validated field will auto-populate:
Summary
This tab is used to record:
Start/End Dates and Times
Whether a candidate is deemed Competent
The Assessor Status (Draft or Complete)
The Candidate Status (Draft, Sealed or Objected)
Assessor Comments (in order to complete an assessment, assessors must leave a comment)
There are three different states that assessments can have:
Draft: neither the assessor nor the candidate have signed the assessment, and therefore, changes can still be made
Complete: the assessor has signed the assessment and the candidate can sign it. No further changes can be made, however, the assigned assessor can unseal the assessment to return to Draft so that additional changes can be made
Sealed: the assessor and candidate have signed the assessment, and therefore, no changes can be made
Attachments
Attachments can be uploaded, previewed, emailed, edited and deleted through the Data Admin on this tab. A number of different file types can be uploaded as assessment attachments, for example, voice comms.
This feature can be used to upload attachments to assessments completed on aAssess.
Review
This page is much like the Review page on aAssess. Criteria Items which are assessed below or above the standard level (e.g. Green) and have not been commented upon will appear here.
Any Criteria Items below or above the standard level require that a comment is added before they can be finalised and the assessment completed.
CMS Events
The CMS Events page is likely to be one of the first things Assessors will see when logged into ACMS. CMS Events can be accessed via the widget on the homepage and also via the dropdown menu.
CMS Events displays all upcoming assessments, sorted on the basis of how imminently they must be completed, whether they are overdue, and also the depot of the candidates who must be assessed.
The assessments that an assessor can see on this page are determined by their Security Group.
The Export button in the top lefthand corner of the page can be used to detail the information displayed into a report which can be sent and received via email.
In CMS Events, it is possible to find a specific assessment, right click and access the following actions:
Request Authorisation
This is used to request authorisation for overdue assessments:
A Reason must be allocated from a set of configurable reasons.
The Notes field should be used to provide additional information.
The Proposed Date field can be optionally set to indicate when the assessment will be completed.
The Proposed Action field can be used to detail what action will be taken:
Authorise
This skips the Request Authorisation process in order to grant the authorisation.
View Authorisation History
This will open a page which displays information about any previous authorisation of overdue assessments.
CDP
This short guide explains the two possible Competence Development Plan (CDP) solutions on ACMS.
This can be done on the system by choosing whether to make these CDP schedules assessable:
For Users
Users are only required to add the specific Schedules onto their candidates.
If using the Assessable CDP model, this is done by going to:
People Search → (Select a Candidate) → CMS tab
Each intervention then needs to be added with a start date. Each of these can then be assessed.
If using the Non-Assessable CDP model, this is done by going to:
People Search → (Select a Candidate) → Schedules tab
The length of the CDP needs to be selected along with a start date.
For admins: Assessable CDP
An Assessable CDP is set up by adding many small schedules, which allow for the bespoke creation of CDP schedules for individuals. This is done by creating Schedule Types and Templates that open windows for 1 month for each type of assessment that may take place. For example: practical, unobtrusive, simulator, professional discussion, etc.
These can then be added with start dates on an individual to build a CDP schedule unique to them. These periods can be assessed as usual.
For admins: Non-assessable CDP
This schedule does not define the individual interventions that will take place but is a marker that “this candidate is on a CDP”. These can then last anywhere from 1 day to any amount of years.