Section 1.0 — Definitions

1.01  Accredited Contractor Examination

An examination, based on contractor classifications described in the NASCLA National Contractor Classification Scopes, that has been evaluated and Accredited by an Auditing Organization chosen by NASCLA.

1.02  Accredited Contractor Program:

Develops and administers examinations using documented policies and procedures that follow sound psychometric practices and are consistent with the NASCLA Examination Standards.

1.03  Auditing:

In this context, is the process in which an NASCLA selected Auditing Organization evaluates a Contractor Examination Program and examination to verify that it meets standards set by NASCLA.

1.04  Auditing Organization:

An independent organization selected by NASCLA to determine whether Contractor Examinations meet the standards set by NASCLA.

1.05  Competency:

In the NASCLA Examination Standards, a knowledge, cognitive (intellectual) skill, or a performance required for safe and effective performance as a construction contractor.

1.06  Examination Administrator:

The individual who has the ultimate responsibility for conducting an Accredited Contractor Examination.

1.07  Examination Classification:

The statistical data of a population, especially the data concerning age, gender, ethnic distribution, geographic distribution, education, or other information that will describe the characteristics of the referenced group.

1.08  Examination/Exam:

In the NASCLA Examination Standards, an instrument that assesses whether an individual has attained at least the minimum level of the competencies, as defined elsewhere in these Standards, that have been determined to be necessary to perform effectively and safely as a construction contractor.

1.09  Demographic data:

The statistical data of a population, especially the data concerning age, gender, ethnic distribution, geographic distribution, education, or other information that will describe the characteristics of the referenced group.

1.10  Entity:

In this document, the same as 'legal entity'.

1.11  Equivalency (in "equivalent examinations"):

There is specific psychometric evidence that various forms of an examination cover the same content and their respective passing scores represent the same degree of competence.

1.12  Examination forms:

Alternate sets of test questions to assess the same competencies, conforming to the same examination specifications.

1.13  Examination Provider:

An Entity that develops and administers examinations.

1.14  Examination specifications:

The description of the specific content areas of an examination, stipulating the number or proportion of items for each area of competency and the level of complexity of those items.

1.15  Examination version:

A test in which the exact set of items in an examination form is presented in another order, language, manner or medium.

1.16  Item bank:

All of the items that have been developed for the several forms of an examination. It includes all of the items available to create examination forms.

1.17  Knowledge, Skills and Abilities (KSAs):

The information and other attributes tha the contractor must possess in order to perform effectively and safely.

1.18  Legal entity:

Either an individual or an organization that is structured in a manner that allows it to function legally and be recognized as a responsible party within the legal system.

1.19  Legally defensible:

The ability to withstand a legal challenge to the appropriateness or relevance of the examination for the purpose for which it is used.

1.20  Minimal Competency:

The ability to accomplish job tasks effectively at a level that does not pose a threat to public safety or welfare.

1.21  NASCLA Accredited:

Designation applied to examinations that, upon being audited by NASCLA, have been determined to meet NASCLA Standards.

1.22  NASCLA Accredited Provider:

An entity that develops and administers examinations that has been determined to meet NASCLA's business, financial and security standards.

1.23  NASCLA National Contractor Classification Scopes:

An assimilation of the common elements of state scopes into one national scope reflecting the central theme of the combined state scope definitions.

1.24  Occupational Analysis:

T he description of functions or tasks required for an individual to perform to entry level standards in a specific job or occupation, including information about the attributes required for that performance. It defines the performance dimension of a job and includes knowledge, skills and abilities necessary to carry out the tasks.

1.25  Proctor:

Person assisting in test administration.

1.26  Psychometric:

Scientific measurement or quantification of human qualities, traits or behaviors.

1.27  Psychometrician:

A professional with specific competencies, as evidenced by education and training, in development and analysis of tests and other assessment techniques and in statistical methods.

1.28  Regulatory Agency:

A governmental or quasi-governmental body that has been duly formed under the laws of a jurisdiction to administer and enforce laws concerning services to the public.

1.29  Reliability:

The degree of consistency of test scores.

1.30  Tasks:

Tasks are the individual functions, whether mental or physical, necessary to carry out an aspect of a specific job.

1.31  Validity:

The extent to which the interpretation of a score from a test or other type of assessment represents the attributes it was designed to measure.

Section 2.0 — Responsibilities of Entities Providing Accredited Contractor Examinations

2.01

An entity providing examinations attesting to the competency of Construction Contractors is responsible to the individuals being tested and to the public.

2.02

Entities applying for Approval of a Contractor Examination are responsible for providing truthful, accurate, and complete information to the Auditing Organization. Falsification of any information constitutes grounds for denial or retraction of approval of the entity and its program.

Section 3.0 — Structure and Resources of Examination Providers

3.01

Entities providing Accredited Contractor Examinations shall conform to all NASCLA requirements for approval and shall demonstrate:>

  1. the availability of financial resources to effectively and thoroughly conduct regular and ongoing competency examination activities; and
  2. that staff possesses the knowledge and skills necessary to conduct the competency examination program or has available and makes use of non-staff consultants and professionals to sufficiently supplement staff knowledge and skills.
3.02

Entities providing Accredited Contractor Examinations shall maintain the financial and administrative independence between the examination program and any related association, organization or agency.

3.03 

In order to protect the reliability and validity of examination scores, it is of paramount importance that examination providers not generate materials that teach the content for the examination per se. Personnel developing the educational or training materials shall not have access to any aspect of the item and examination development process (other than publicly available specifications and materials) nor to the completed examination or item bank. Personnel in the latter process shall not have access to any aspect of the development of the educational material. Personnel involved in training for the test shall not be involved in any aspect of examination development.

3.04 

The Examination Provider and its staff shall not engage in any activities that may impair the validity of the examination results for their intended purpose.

Section 4.0 — Contractor Examination Development

4.01

The examination must be based on those tasks required to function competently at entry level in the construction contracting field or specialty for which the examination is developed, with emphasis on those tasks most closely linked to safe and effective practice as a Construction Contractor. Building Code knowledge as applied to actual job performance, professional ethics, and relevant aspects of federal law may be included among the examination topics, but cannot be the sole content.

4.02

Entities providing examinations shall review the Equal Employment Opportunity Commission (EEOC, et al) guidelines, the Uniform Guidelines on Employee Selection Procedures , and Standards for Educational and Psychological Testing produced by the American Educational Research Association, the American Psychological Association and the National Council on Measurement in Education, the Principles of the Society of Industrial and Organizational Psychologists, and the guidelines for computer based testing of the Association of Test Publishers to verify that the Contractor Examination(s) comply with all relevant provisions.

4.03

Each Examination Provider must submit evidence that it meets the following professional requirements:

  1. ability to conduct or otherwise use a legally efensible and psychometrically valid occupational analysis;
  2. experience in the development of psychometrically valid competency examinations;
  3. capability to develop and implement thorough procedures for security of the item bank, printed, taped or computerized examinations, and candidate scores;
  4. data handling capabilities commensurate with the requirements for effective processing, reporting and archiving of candidate information and scores; and;
  5. evidence of an understanding of and wilingness to abide by the principles of fairness and due process.
4.04

The examination provider must submit complete information for the Contractor Examination for each NASCLA classification. Information related to procedures and personnel involved in all aspects of the examination development and analysis may include :

  1. complete description of the scope and usage of the examination;
  2. job analysis task list, with knowledge, skills and abilities (KSAs);
  3. examination specifications as derived from the job analysis;
  4. the number of unduplicated items in the item bank;
  5. mapping of items to examination specifications;
  6. the distribution of items in the bank across the different dimensions of the table of specifications for the examination;
  7. statistical performance of each item in the bank;
  8. number of "paper and pencil" or linear computer delivery examination forms, with evidence of their equivalence to each other;
  9. algorithms for choice of items, point of ending examination, and other relevant aspects of examinations delivered by computer;
  10. description of method used to set passing score;
  11. data on decision consistency reliability and other reliability estimates;
  12. copies of all logs, diaries and personnel lists and descriptions kept as required in the development process;
  13. summary statistics for each examination;
  14. names, credentials and demographic information for all persons involved in development of the job analysis KSAs, item writing and review, examination specifications, and setting the passing score;
  15. demographic and statistical information about the group(s) on which items and examination forms were piloted;
  16. a sensitivity review and, to the extent possible, DIF analysis or diversity study, with information about actions taken if problems were found;
  17. description of method used to calculate each item statistic;
  18. description of equating methodology and the results of the equating of multiple forms;
  19. schedule for replenishment of item bank, including number of items to be added, retired, rested each quarter;
  20. procedures for verifying examinee identity;
  21. item exposure control policy and procedures;
  22. retake policy recommendations;
  23. methodology for conducting analysis to detect security breaches, frequency of conducting such analyses, and procedures for action if a breach is detected;
  24. for computer adaptive tests or other tests based on an IRT methodology, the test information function; and
  25. method used for establishing readability level or index;
  26. policy and procedures for changing items within examinations, especially when individual items are found to be flawed or exposed; and
  27. due process or appeals policies and procedures. for candidates.
4.05

he Accredited Contractor Examination must be based on a psychometrically valid national occupational analysis developed by psychometricians and a demographically and technically representative group of individuals with significant experience in the relevant field(s) of the construction industry. The representative group must include, but not necessarily be limited to, persons with experience in the specific construction area or field for which the examination is intended, persons with local, state or national regulatory experience in construction, and must be sufficiently diverse as to avoid cultural bias and ensure fairness in content according to all applicable federal requirements .

4.06

The occupational analysis must provide a complete description of the knowledge, skills and abilities (KSAs) required to function competently at entry level in the construction contracting field or specialty for which the examination is developed, with emphasis on those tasks most closely linked to safe and effective practice as a Construction Contractor. It must include, but is not necessarily limited to, the content of the appropriate classification of the NASCLA National Contractor Classification Scope.

4.07

Detailed Contractor Examination specifications must be derived from a valid study of the occupational analysis tasks and their accompanying knowledges, skills and abilities (KSAs) and must be appropriate to the NASCLA National Contractor Classification Scope for which the examination is intended. The examination specifications, consisting of percentage weights or number of items devoted to each content area, must be available to candidates and to the public.

4.08

The examination provider conducting an occupational analysis must maintain a log and diary of the procedures actually performed in the occupational analysis and a list of the qualifications, identities, and demographic data of the persons who participated in development of the KSAs and of the examination specifications. That information must be provided to the Auditing Organization on demand.

4.09

The occupational analysis must be reviewed and revalidated or redeveloped at least every five years. Examination providers are required to systematically evaluate practices in the construction industry to assure that the occupational analysis and KSAs on which an examination is based remains appropriate for the development of Accredited Construction Contractor Examinations.

4.10

Development of Accredited Contractor Examinations, including procedures for developing data used for setting a psychometrically sound passing score, shall be based on the most recent edition of Standards for Educational and Psychological Testing, developed jointly by the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, and on all relevant federal requirements. Accredited Construction Contractor Examinations must be revised as needed to be in compliance with changes in the Standards for Educational and Psychological Testing or in any of the applicable federal requirements.

4.11

The Contractor Examination must be based on psychometrically valid procedures to assure the relative equivalence of passing scores from various examination forms.

4.12

There shall be a sufficient number of forms of each classification of examination to assure that candidates cannot anticipate the particular set of items included in their examinations and to protect the items from overexposure. When forms are computer-generated, the algorithm for item selection and the number of items in the bank from which the test is generated shall assure that the items are protected from overexposure. Item usage statistics must be provided for all available items in the pool.

4.13

The Accredited Contractor Examination Program shall provide contractor examinations that :

  1. conform to all National Association of State Contractors Licensing Agencies Examination Standards;
  2. have been developed from an item bank that is: large enough to assure that the items are protected from overexposure; refreshed at least annually by enough items to generate the number of new items and forms needed to support the number of examinees and the schedule on which the tests are administered;
  3. are composed of psychometrically appropriate proportions of new items which are not clones or close variants of existing items, items not used within at least the past year, and recently used items as required for equating or other statistical processes;
  4. have not been nor are suspected to have been violated by a breach of security or item overexposure; and
  5. are written with readability level and language appropriate to the profession.
4.14

When any form of the Contractor Examination is translated to a language other than that in which it is originally developed and validated, the developer of the examination must provide evidence of accuracy of translation (content equivalency) of the translated version with the examination form from which the translation is derived. The developer must provide a detailed description of the translation method(s), including the rationale for selecting the translation method(s), and must demonstrate congruence of items and instructions with those examination form that was translated. To avoid potential problems in translation of terms specific or idiomatic to the construction industry, translation will be accomplished with the consultation of construction industry personnel competent in the languages of both the original and the translated version of the Construction Contractor Examination.

4.15

Examination Providers must maintain a log and diary of the procedures actually performed in item development, examination development, sensitivity review, translations, setting the passing score, and the statistical analyses of the test items and of the full examination. A full list of the identities, qualifications, and demographic data of the persons who participated in those procedures must be maintained. Documentation of the procedures performed in the specified activities and of the participants must be provided to the Auditing Organization on demand.

4.16

Examination Providers must demonstrate that procedures are developed and implemented to assure that individual items, item banks, Contractor Examinations presented in all media (printed, taped and computerized), and candidate scores are and remain secure. In addition to procedures to assure security, the system must include the protocol for handling any real or suspected breaches of security, the contingency plans for disposition of examination forms or items that are shown to be disclosed, and notification to the Auditing Organization and to NASCLA of the problem and its consequences. Policies and procedures for repairing forms in which individual items are flawed or have been compromised or exposed must be provided and must clearly show how the level of reliability and validity of the form is maintained.

4.17

At least semiannually, each Accredited Contractor Examination Program must report to the Auditing Organization, providing a review of its Accredited Contractor Examination(s). The report will include the following summary statistics for all examinations administered during the preceding six months, as well as other information that may be reasonably requested by the Auditing Organization:

  1. number of Contractor Examinations administered;
  2. mean;
  3. median;
  4. standard deviation;
  5. range;
  6. data on score reliability and decision consistency;
  7. number and percentage of candidates passing the examination;
  8. the statistics describing the performance of each item used on Contractor Examinations administered during the six-month period;
  9. number of candidates retaking the examination;
  10. item exposure data; and
  11. data from analyses to detect security breaches.

Section 5.0 Contractor Examination Administration / Delivery

5.01

All aspects of an Accredited Contractor Examination administration are to be conducted in a manner that provides for the security of the examinations. This must be accomplished in a manner that ensures fairness to all candidates.

5.02

Examination Providers must specify procedures for administering all Contractor Examinations in a standard manner in order to assure that all candidates are provided with the opportunity to perform according to their level of competency and to assure comparability of scores. Procedures must include, but not be limited to:

  1. qualifications of Test Administrators and Proctors and a suitable training program for each;
  2. a complete administration manual, describing each step of the test administration process and the rationale for each;
  3. clear instructions for candidates;
  4. specification of security procedures to assure lack of exposure of test items to unauthorized persons during testing and to prevent theft of examination items or booklets;
  5. clear criteria (with rationale) for physical facilities for examination administration; and
  6. clear criteria (with rationale) and procedures for adaptations necessary to accommodate qualified candidates with disabilities.
5.03

Construction Contractor Examinations must be presented in a manner that allows absolutely no one other than examinees to see the contents of the booklet or alternative medium. Examinees may see the examination contents only during the time and in the place allowed for the examination. Neither Test Administrators, Proctors, nor any other persons are authorized to see the examination contents except as is necessary in the performance of their duties.

5.04

There special accommodations must be made for otherwise qualified candidates under provisions of the Americans with Disabilities Act or in matters related to non English-speaking candidates, arrangements must be such that the Contractor Examination contents are not revealed to any test administration personnel with any conflict of interest. A written affirmation to that effect and a written nondisclosure statement from the individual who was chosen to assist the otherwise qualified candidate must be given to the examination provider and, on request, transmitted to the Auditing Organization.

5.05

The examination provider must present procedures to be followed in any instance where the security of a Contractor Examination is, or is suspected to be, breached. Included must be specific procedures for handling and reporting to the Auditing Organization and to NASCLA any and all suspected or alleged cheating incidents, lost or stolen booklets, breaching of electronic security for computer-administered examinations, intentional or unintentional divulging of test items by examinees or test administration personnel, or any other incidents perceived to have damaged the security of the examination or any of its individual items.

5.06

Security of the Contractor Examination materials must be maintained in transmittals to and from the examination administration site. Examination materials must be kept secure both before and after they are used.

5.07

Test Administrators are responsible for the organization and administration of all examination site activities and procedures. Test Administrators must have training, documented successful experience or a combination of experience and training in test administration and security procedures. They must provide written assurance of maintaining confidentiality of test contents and of adherence to standards and ethics of secure examination administration.

Test Administrator reponsibilities include, but are not limited to:

  1. assuring that the site conforms to requirements;
  2. training and supervising Proctors;
  3. assuring accurate identification of examinees;
  4. adherence to all procedures and instructions in the examination administration manual;
  5. maintaining security of test materials;
  6. assuring compliance with procedures for handling any breaches of security that may occur;
  7. proper handling of completed examinations;
  8. confidentiality of candidate information;
  9. such unspecified duties as may be required for safe and secure administration of the examination.
5.08

Proctors shall work under the direction of the Test Administrator, assuring that all aspects of an examination administration are being carried out with precision, with full attention to security and to the fair treatment of examinees. Proctors have the responsibility and must have the ability to observe examinee behaviors, accurately distribute and collect test materials, and assist the Test Administrator as assigned. They must have training or documented successful experience in monitoring practices and must affirm in writing that they will maintain test security and that they have no conflict of interest.

5.09

The number of Accredited Proctors assigned with a Test Administrator must be sufficient to allow each examinee to be observed and supervised to assure conformance with security requirements. In a 'pencil and paper test' setting, there shall be no less than one Test Administrator or Proctor per thirty examinees. When examinations are administered by computer or other electronic medium, the number of proctors will be that which is necessary to observe each examinee adequately, based on the configuration of the examinee work stations / seating arrangements.

5.10

Sites chosen for administering Contractor Examinations must conform to all legal requirements for safety, health, and accessibility for all qualified candidates. Additionally, the accommodations, lighting, space, comfort, and workspace for taking the examination must allow all candidates to perform at their level of competency.

5.11

Requirements at each site include, but are not limited to, the following criteria:

  1. accessibility in accordance with requirements of the Americans with Disabilities Act must be available for all qualified examinees, whether it be the main site for an administration, or in an alternative site meeting all other requirements of the main site;
  2. all sites must conform to all fire safety and occupancy codes of the jurisdiction in which they are located;
  3. in the area in which testing is conducted, there must be sufficient spacing between each examinee, or other appropriate and effective methods must be used to preclude any examinee from viewing another's test responses;
  4. acoustics must allow each examinee to hear instructions clearly, using an electronic audio system if necessary;
  5. lighting at each examinee's work space must be adequate and appropriate to allow clear viewing of the test content;
  6. ventilation and temperature must be appropriate for health and comfort of examinees;
  7. equipment used for electronic administration of the examination must be such that it is readily used by examinees at all levels of skill in the use of that testing equipment (i.e., computers), and, at a minimum, will conform to such visual and auditory criteria as is customary with non-electronic testing methods; and
  8. seating arrangements at work stations in sites for computer administration or other electronic administration must be such that proctors can observe examinees adequately to assure that the candidates do not have access to unauthorized sources of information or assistive devices.
5.12

Examination scoring procedures must be reliable, fair, and accurate. An alternative scoring method must be available in the event that the primary scoring method cannot be used (e.g., power failure). If scaled scores are required, the method used for score conversion must be provided to the Auditing Organization.

5.13

Contractor Examination scores will not be released as being official until verified.

5.14

Examinee test results will be confidential. They will be available only to the examinee, to NASCLA, to the appropriate Regulatory Agency, and to other persons or organizations Accredited in writing by the examinee.

5.15

Score reports will be delivered to examinees and other authorized recipients in a time frame specified in the application for the examination. If there is a delay on the part of the examination provider, authorized recipients will be informed and an approximate date for release of the scores will be announced. core reports will be delivered to examinees and other authorized recipients in a time frame specified in the application for the examination. If there is a delay on the part of the examination provider, authorized recipients will be informed and an approximate date for release of the scores will be announced.

Section 6.0 Examination Providers' Responsibilities to Users and the Public

6.01

A Contractor Examination Provider must:

  1. not discriminate among applicants as to age, sex, race, religion, ethnic origin, disabilities or marital status and shall include a statement of non-discrimination in announcement of the examination program;
  2. provide all applicants with copies of formalized procedures for application for the examination.
  3. review examination procedures to ensure that they are fair and equitable;
  4. prior to the examination, provide candidates with information concerning the examination specifications covered by the Contractor Examination;
  5. provide prompt reporting of the Contractor Examination results to examinees and other authorized recipients; and,
  6. have a formal policy on appeals procedures for applicants questioning any aspect of the examination, assuring that due process concerns are addressed.
6.02

Each Contractor Examination Provider must have a published protocol for systematically investigating problems presented by users of the program, including specific concerns about examination items, administration procedures, treatment of candidates, or other matters involving potential legal defensibility of the examination or program. The protocol will include a published time frame for reporting findings to the user.

6.03

Entities providing Contractor Examinations shall transmit a copy of all examinees' information to NASCLA in a timely manner. This shall contain all information specified by NASCLA in this document or in its future revisions. At least the following information must be included.

  1. Full legal name;
  2. Date of birth;
  3. Permanent address;
  4. Current address;
  5. NASCLA National Contractor Classification Scope;
  6. Form of examination;
  7. Date of examination;
  8. Test results;
  9. Examination provider (vendor);
  10. Examinee identification number; and
  11. Digital full-face photograph.
6.04

Only organizations whose Contractor Examinations are accredited as to conformance with all criteria contained within this Standard may make reference to providing an accredited examination. Any reference to accreditation will be limited to the specific Construction Contractor Programs that have been accredited by NASCLA.

Section 7. Length of Approval Period

7.01

Examination programs may be accredited for a maximum of five years. If there is a significant change in the development process or in industry practices, approval must be reviewed and may be continued, withdrawn or accredited for a shorter time period.

 

Top