(version of 26 September 2012)
Scope of this document
1.1. These guidelines have been developed by the partners of the project IMPROVE Improving Validation of Not-Formal Learning in European Career Guidance Practitioners 510640-LLP-1-2010-1-IT-GRUNDTVIG-GMP (2011-2012)[1]. More information can be found on the project website: www.improveguidance.eu/.
1.2. These guidelines are mostly focus on validation of the outcomes of not formal and informal learning of current workers (see point 3.2. C below).
Glossary
2.1. Assessment Center: a structured assessment process where a group of participants undertakes a standardized series of job-related exercises in controlled conditions under observation, so that their personal characteristics can be assessed. The exercises can include interviews, psychometric tests, individual and group role play (the former includes for example, leaderless discussion).
2.2. Behavioral Event Interview (BEI): a structured interview which elicits personal features. The interviewee describes, in his/her own words, what he/she said, thought, felt, and did in six episodes—three positive and three negative—at work. The interview is recorded, transcribed, and coded for various personal characteristics. Personal characteristics are coded both for frequency of occurrence in the interview and for the level of complexity or scope at which they are displayed (adapted by McClelland, 1998).
2.3. Competent worker: a person able to perform an occupation or a job task up to a predetermined standard.
2.4. Elements (for validation): personal features or key job activities that a worker has respectively to possess or master to successfully pass validation.
2.5. Evidence (for validation): a proof that demonstrate possession or mastering of the elements on which validation is based.
2.6. Job analysis: a systematic study of the tasks that are performed in a work role and of the personal features necessary to perform them. The job analysis allows identifying a hierarchy of tasks, from the most important and general (main tasks) to the minor ones (tasks and sub tasks). The tasks are identified drawing a flowchart describing how a job is carried out and this way main tasks, tasks and sub tasks are described as a tree root, where combination of simpler actions allow to carry out the more complicate.
2.7. Job tasks: the actions that can be identified in a flowchart describing how a job is carried out. See also (job) main task.
2.8. Learning: a process by which an individual assimilates information, ideas and values and thus acquires knowledge, know-how, skills and/or competences (source: quoted in CEDEFOP (2008) Terminology of European education and training policy. A selection of 100 key terms).
2.9. Learning outcomes: the set of knowledge, skills and/or competences an individual has acquired and/or is able to demonstrate after completion of a learning process, either formal, non-formal or informal (source: quoted in CEDEFOP (2008) Terminology of European education and training policy. A selection of 100 key terms).
2.10. Main (job) task (or key job activity): each of the main identifiable and self standing activities a person carrying out a job can accomplish.
2.11. Performance based approach (to validation): a framework for validation of the outcomes of not formal and informal learning where the elements to be assessed are the performance of job tasks.
2.12. Performance Focused Interview (PFI): a structured professional discussion (see entry below). In PFI the questions are focused on specific predetermined aspects of performance and all the Candidates are asked the same list of questions. However the Assessor may ask additional questions for clarification or a better understanding.
2.13. Personal features: personal attributes (for example skills, knowledge, attitudes) required to perform a certain work.
2.14. Personal features based approach (to validation): a framework for validation of the outcomes of not formal and informal learning where the elements to be assessed are knowledge and other personal features.
2.15. Professional Checkup: a methodology for validation of the outcomes of not formal and informal learning used in the pilot of the project IMPROVE and based on the Performance Focused Interview. The Professional Checkup can be carried out in small groups or at individual level. When carried out at individual level the Professional Checkup is made of three interviews carried out with an evaluator during a period of 1-3 weeks: the first interview is informative about the methodology, the second uses the Performance Focused Interview technique to assess performance in main tasks for which validation is sought; the third interview is used to help the practitioner to develop an improvement plan. The interviews can be carried out face to face in person or at distance using videoconference tools.
2.16. Professional discussion: An interview conducted between an Assessor and Candidate (assessed person), in which the Candidate describes his/her job tasks and how his/her performance achieves requirements set by standards.
2.17. Professional supervision: the process in which through focused observation and targeted questions an expert (the supervisor) considers the effectiveness of intervention, supports quality of work and practitioner’s ability of reflection of own work and of feelings during the process of working with clients.
2.18. Skill: a personal faculty required to do something or get something done. Skills are of a general nature. For example leadership is a skill, but to give instructions to subordinates is the result of applying leadership (together with other skills) to a job task and not a skill in itself.
2.19. Standard: something considered by an authority or by general consent as a basis of comparison (source: Dictionary.com)
2.20. Validation: a process where a competent body assesses and officially recognizes that a person, a person’s activity or one or more personal attributes comply with a predefined standard.
2.21. Validation framework: a set of rules for validating the outcomes of not formal and informal learning.
Preamble
3.1. As learning refers to the process of learning, the term validation must always be referred to the outcomes of learning.
3.2. Validation of the outcomes of not formal and informal learning can have three different goals:
- to allow to gain an educational title or a vocational qualification by shortening the length of the study path
- to hire people (recruitment) or to assign existing workers to higher or diverse positions and responsibilities (potential appraisal)
- to be sure current workers perform their tasks up to a set standard.
In cases B and C the outcomes of learning that are validated come from all forms of learning, including formal.
3.3. Every goal requires different directions of assessment. For example in case A. general or specialist knowledge and some cognitive skills (reading, writing, calculating) are assessed. In case B Personal features causing good performance on the job, such as knowledge, skills (usually interpersonal and cognitive), attitudes, motives, etc, depending on the target job. In case C. Performance on the job (observed and/or reconstructed).
3.4. All validation frameworks include four components (points 3.5., 3.6., 3.7, 3.8.):
3.5. What is recognized or granted at the end of the procedure of validation. It can be a title, (for example Education and Vocational Guidance Practitioner, as in the IAEVG’s framework), a qualification, an attribute to a qualification (as in Registered EuroPsy Psychologist), a certificate (such as MEVOC’s European Certificate for Career Guidance Counselors) or academic credits.
3.6. The elements to be assessed during validation . In a personal features based approach they can be for example knowledge, transversal skills, technical skills, attitudes, character traits, etc. In a performance based approach are job tasks. The elements are the learning outcomes we want to validate.
3.7. The evidences that demonstrate possession or mastering of the elements identified in the previous point 3.6. These evidences can be documents or results from tests, interviews, observation. For example a degree in psychology can be an evidence of some specific knowledge useful for working as a Career Guidance practitioner. Transferable skills are usually elicited and measured by the BEI Behavioral Event Interview and/or leaderless discussion, this last contained in the Assessment Centre. Job main tasks performance can for example be assessed by observation of the worker.
3.8. An assessment procedure to collect or elicit evidences.
3.9. More in detail, personal features are usually assessed using combination of some of the following assessment methods:
- Examination of Candidate’s CV or narrative of professional biography
- Control of possession of educational qualifications and other educational or training certificates
- Control of possession of proven experience
- Interview or written test about technical knowledge
- Tests (skills, personality, interests, etc.)
- Role playing focused on transversal skills (as in the Assessment Center)
- Interview focused on transversal skills (as in BEI Behavioral Event Interview).
3.10. More in detail, performance can be assessed using a combination of some of the following assessment methods:
- Direct observation of the person whilst carrying out his/her work
- Professional discussion
- PFI Performance Focused Interview
- Discussion of case studies
- Testimonies from colleagues and supervisors
- Testimonies from clients
- Examination of documentation produced by the person whilst carrying out his/her work
- Examination of portfolio of work
- Simulation of job tasks.
3.11. In all frameworks it is necessary to find a good compromise between efficacy and weight of the assessment procedure. A procedure may be very effective but if it requires significant dedication of time and economic resources it will have minimal possibility to become established and widely implemented (unless it is enforced by a public authority). On the other hand, a procedure which requires little time, but is less effective also presents the weakness of minimal utility.
The IMPROVE Guidelines
General principles
4.1. The validation process of the outcomes of not formal and informal learning of current workers must be performance based. Substantial focus on the assessment procedure must include the direct examination of the work performance of the Candidate and/or on the reconstruction of performance of Candidate at work such as in the Performance Focused Interview (PFI).
4.2. The elements (job main tasks and job tasks) the Candidates have to master must be previously defined through a job analysis, and examination of available documentation on occupations and a pilot study. The results of the investigation have to be discussed and agreed upon among practitioners and other sector stakeholders.
4.3. The evidences that demonstrate mastering of job main tasks should be based on the assessment methods taken from that listed at point 3.10.
4.4. Successful validation cannot be conditional on the possession of an educational qualification, proven experience or attendance of specific training courses, membership of association or group.
4.5. The validation framework, including its structure, assessment process, roles, scoring system, key terms must be clearly described and freely available.
4.6. The assessment process and assessment methodology used must be the same for all Candidates and applied in the same manner by all Assessors.
4.7. Validation can be initiated by an organization to check the outcomes of not formal and informal learning of its employees and collaborators or by the practitioner him/her self. In the second case the validation process, and specifically the Assessor , shall guarantee the confidentiality of the results towards third parties.
4.8. When validation is initiated by the practitioner him/her self, the validation process should also be a learning experience. The Candidates must receive a feedback which precisely describes how they scored on each task that was examined, identification of any areas of improvement and guidance on how to carry out such improvement.
The Assessors
4.9. The Assessors must be appropriately trained for the validation process and possess a thorough working experience of the main tasks they are assessing.
4.10. The quality assurance system of the validation procedure has to include professional supervision among the Assessors and the sharing of their experiences with other Assessors for learning purposes.
4.11. The Assessors must declare any possible conflict of interest and must withdraw themselves from any assessment in which impartiality and confidentiality cannot be assured.
The assessment process
4.12. A direct examination of the Candidate (direct contact or mediated contact through videoconference) is compulsory.
4.13. A copy of all the documents has to be conserved and the direct examination should be recorded according to the applicable data privacy regulations of the context of the assessment.
4.14. The evaluator must write a report on evaluation, which precisely describes the Candidates’ main answers, how they scored on each task that was examined, identification of any areas of improvement and guidance on how to carry out such improvement.
4.15. Feedback on the validation process has to be collected from every participant through a predetermined procedure. The feedback is also part of the documents that must be stored about the validation process on the given Candidate.
[1] In alphabetical order Leonardo Evangelista (ASEV Agenzia Sviluppo Empolese Valdelsa www.asev.it, IT), Zuzana Freibergova (Národní vzdělávací fond, o.p.s.www.nvf.cz/spps/, CZ), Rachel Nelson (DEP Institut www.dep.net, ES) Speranta Tibu (ODIP Asociaţia Observator pentru Dezvoltarea Învăţării Permanente www.odip.ro, RO), Mary Tountopoulou ISON Advanced Applications In Management Systems Psychometrica www.ison.gr, EL), Peter Weber (Heidelberg University www.ibw.uni-heidelberg.de/, DE). The project has been partly funded by the European Commission. This document reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.