Here is a summary of the very famour article Testing for competence rather than for “intelligence” by David McClelland.
The summary of Testing for competence rather than for “intelligence”.
Critics to intelligence and aptitude tests
McClelland starts writing: The testing movement in the United States has been a success if one judges success by the usual American criteria of size, influence and profitability. Intelligence and aptitude tests are used nearly everywhere by schools, colleges, and employers. (p.1). (…). Why should intelligence and aptitude tests (such as the Scholastic Aptitude Tests, administered by Educational Testing Service) have all this power? (1). My objectives are to review skeptically the main lines of evidence for the validity of intelligence and aptitude tests and to draw some inferences from this review as to new lines that testing might take in the future. (1). The Scholastic Aptitude Tests in fact predicts grades in school, but grades in school are only slightly related with ‘success in life. McClelland sometimes refers to success in life while some others to vocational success, superior on the job performance, job success (2).
The basic problem with many job proficiency measures for validating ability tests is that they depend heavily on the credentials the man brings to the job –the habits, values, accent, interests, etc.- that mean he is acceptable to management and to clients. Since we also know that social class background is related to getting higher ability test scores (…) as well as to having the right personal credential for success, the correlation between intelligence and test scores and job success often may be an artifact, the product of their joint association with class status. Employers may have the right to select bond salesman who have gone to the right schools because they do better, but psychologists do not have the right to argue that it is the intelligence that makes them more proficient in their job. We know that correlation does not equal causation (…) (3).
Furthermore, the intelligence tests are in reality simply a test of ability to do well in school. Here the test has become part of the criterion and has introduced the correlation artificially. (4) The tests are clearly discriminatory against those who have not been exposed to the culture, entrance to which is guarded by the tests. (7).
Brainstorming
But now I am on the spot. Having criticized what the testing movement has been doing, I feel some obligation to suggest alternatives.’ (…) My goal is to brainstorm a bit on how things might be different, not to present hard evidence that my proposal are better than what has been done to date. How would one test for competence, if I may use that word as a symbol for an alternative approach to traditional intelligence testing? (7).
1. The best testing is criteria sampling. (…) If you want to know how well a person can drive a car (the criterion) sample his ability to do so by giving him a driver’s test. Do not give him a paper-and-pencil test for following direction, a general intelligence test, etc. (7) Criterion sampling means that testers have got to get out of their offices where they play endless word and paper-and-pencil games in to the field where they actually analyze performance into its components. If you want to test who will be a good policeman, go find out what a policeman does. Follow him around, make a list of his activities and sample from that list in screening applicants. (7). (…) If someone wants to know who will make a good teacher, they will have to get videotapes of classrooms, as Kounin (1970) did, and find out how the behaviors of good and poor teacher differ. To pick future businessmen, research scientists, political leaders, prospects for happy marriage, they will have to make careful behavior analysis of these outcomes and then find ways of sampling the adaptive behavior in advance. (8)
2.test should be designed to reflect changes in what the individual has learned. It is difficult, if not impossible, to find a human characteristic that cannot be modified by training or experience (…). It seems wiser to abandon the search for pure ability factor and to select tests instead that are valid in the sense that scores on them change as the person grows in experience, wisdom and ability to perform effectively on various tasks that life presents him. (8)
3. How to improve the characteristic tested should be made public and explicit. (9)
Here come the competencies
4. Tests should assess competencies involved in clusters of life outcomes. (9) If we (…) move toward criterion sampling based on job analysis, there is the danger that the tests will become extremely specific to the criterion involved. For example the Project ABLE (Gagné 1965) has identified over 50 separate skills that can be assessed for the exit level of millman apprentice. (…) They include skills like ‘measures angles’, sharpen tools and planes’ (…) This way What one ends up is hundreds or even thousands of specific tests for dozens of different occupations. For some purposes it may be desirable to assess competencies that are more generally useful in clusters of life outcomes, including not only occupational outcomes but social ones as well, such as leadership, interpersonal skills, etc. Project ABLE has been excellent at identifying the manual skills involved in being a service station attendant, but so far it has been unable to get a simple index of whether or not the attendant is pleasant to the customer (9) Some of these competencies may be rather traditional cognitive ones involving reading, writing and calculating skills. Other should involve what traditionally have been called personality variables, although they might better be considered competencies. (10). The article than lists (10): Communication skills, Patience, Moderate goal setting (‘moderate’ means here realistic, neither too high or too low), Ego development (to be measured for example with Thematic Apperception Test, it measure taking initiative on behalf of others).
5. Tests should involve operant as well as respondent behavior. (…) McClelland refers here to tests where individual spontaneously makes a response in the absence of a very clearly defined stimulus (11) On the contrary in most existing tests respondents are obliged to chose an answer between a predetermined set. But life outside tests seldom presents the individual with (…) clearly defined alternatives. (11) Most (…) tests simply required the person to find the one correct answer the test maker has built into the item. What was needed were (instead) test items to which there were many correct answers, among which one was better than others in terms of some criteria of efficiency that the person would have to apply. This task seemed more life like (…) (11)
6. Tests should sample operant thought patterns to get maximum generalizability to various actions outcomes. As noted already the movement towards defining behavioral objectives in occupational testing can lead to greater specificity and huge inventories of small skills that have little general prediction power. One way to get around to this problem is to focus on defining thought codes because almost by definition they have a wider range of applicability to a variety of action possibilities. That is, they represent a higher order of behavioral abstraction than any given act itself which has not the capacity to stand for other acts the way a word does. (..) The n Achievement score –an operant thought measure- has many actions correlates from goal setting and occupational styles to color and time-span preferences (…) which individually have little power as ‘actones’ to predict each other.’ (…) The tester of the future is likely to get further in finding generalizable competencies of characteristics across life outcomes if he starts focusing on thought patterns rather than by trying to infer what thoughts must lie behind the clusters of action that come out in various factors in the traditional trait analysis. (12).
Suggestions to the Educational Testing Service
The article ends with some suggestions to the Educational Testing Service (13):
- To drop the term intelligence and adopt instead scholastic achievement test
- To support schools a profile of scholastic and non scholastic achievements in a number of different areas (also ego development and moral development should be measured), so the school can decide which areas wants to improve
- This profile should be upgraded at several points of the school career, so to become a device for helping students and teachers redesign the teaching-learning process to obtain mutually agreed-on objectives. Only then will educational testing turn from the sentencing procedure it now is into the genuine service it purports to be.
The article can be downloaded from here McClelland
Some hints on Testing for competence rather than for “intelligence
Dissatisfaction with IQ tests
McClelland attacks intelligence and aptitude tests (such as IQ and DAT) used to select future students and employees for two reasons:
- because they are class status biased (results are mostly related to elements such as propriety of speech depending by socioeconomic status, not to intelligence). Their use may be ethical for employers, but not in education
- because they predict well only success at school, but not superior job performance or more generic job success (the latter too in part related to socioeconomic status). Superior job performance is related not only to intelligence and traditional cognitive elements involving reading, writing and calculating skills, but also to personality variables, such as habits, values, leadership, interpersonal skills.
Recruitment
- in predicting future job performance McClelland dismisses IQ, and emphasizes ‘personality variables’ such as habits, values, leadership, interpersonal skills.
- Whatever they are, the elements to be assessed have to be find making use of a job analysis of each occupation,
- comparing excellent and ordinary performers, so to find the criterion, that is to say the kind of behavior that determines the superior performance (‘criterion’ means ‘a standard by which you judge, decide about or deal with something’ -Cambridge Advanced Learner’s Dictionary online, retrieved 8th September 2008).
- The behavior (the criterion) has to be looked after in potential employees .
A problem is in recruitment is in many cases not possible to sample the criterion, because applicants have never done the job they are selected for or is not possible to observe them at work before the selection procedure is ended. McClelland writes ‘ If you want to know how well a person can drive a car (the criterion) sample his ability to do so by giving him a driver’s test.’ But what if is not possible to test the criterion? For example if the person cannot drive, and selection is carried out to admit people to a school driving course? Then other ways have to be found. McClelland suggests to focus on ‘thought patterns’ such as n Achievement and ego development because they are well related with single clusters of action outcomes. (10). Later to identify ‘thought patterns’ McClelland will introduce the BEI Behavioral Event Interview. But this way McClelland makes a U-turn back. Let see his path: A. he starts criticizing an approach to recruitment based on some personal features (intelligence and aptitudes) B. then says recruitment should be based on direct observation of how people performs job tasks (criteria sampling) ‘If you want to know how well a person can drive a car (the criterion) sample his ability to do so by giving him a driver’s test.’ C. but as usually in recruitment criteria sampling is not possible he suggests to go back to personal features (others from intelligence and aptitudes). He moves from personal features supposed to be related to superior job performance to direct observation of how job tasks are carried out and then back again to personal features supposed to be related to superior job performance. A different solutions to the problem of effective criteria sampling has been to make the candidates to role play some work tasks, as in the assessment centre, or enrolling on trial the most promising candidate(s).
To better understand the two approaches, please see this scheme: when we examine a person who is working, we see personal attributes (knowledge, skills, attitudes, etc.) determine the performance on tasks to be carried out.
To find out if a person is competent, we can use two approaches to assessment (For a more detailed account see Evangelista L. (2006) Competencies and Career Guidance or, in Italian, Evangelista L. (2006) Le competenze. Cosa sono, come rilevarle, come si utilizzano nell’orientamento.
- 1. to examine the person to see if she/he has the personal attributes we consider causally related to effective or superior performance. Here the focus is on (good) personal attributes (note 10). This way we assess personal attributes. This can be defined as ‘The American approach to competence’ because has been developed in the US by McClelland, Boyatzis (note 11) and others. It is usually adopted when is not possible (for example in recruiting) to observe directly the person carrying out a job.
- 2. to observe the person carrying out a job, and to see if she/he performs well the tasks of the job. In practice, tasks are the main steps that can be identified in a flowchart describing a specific main task. Here the focus is on (good) performance. This way we assess performance in carrying out tasks and the overall performance result. This can be defined as ‘The UK approach to competence’, because it is used in the UK’s NVQ. This approach is usually adopted when we can observe directly (as when recognising a qualification to a worker) the person carrying out a job. Personal attributes can be assessed too (for example technical knowledge) but only related to a specific task (for example ‘How you choose your tools when cutting seasoned wood?’) and not explicitly listed as elements to be assessed.
Competence / competencies
McClelland uses the words competence and competencies but doesn’t define them. The word ‘competence’ first appears in the text is at page 6: ‘Yet, neither the tests nor school grades seem to have much power to predict real competence in many life outcomes (…)’. He is summing up what already said about tests in the previous pages. Previously he has been talking of ‘success in life’ ‘vocational success’, ‘superior on the job performance’, ‘job success’. At p. 7: How would one test for competence, if I may use that word for an alternative approach to traditional intelligence testing?’, hence the title of the article ‘Testing for competence rather than intelligence’. ‘Competence’ here means the ability to do something well, as in Cambridge Advanced Learner’s Dictionary online, (retrieved 8th September 2008).
The word ‘competencies’ first appears in the text is at page 9: ‘Tests should assess competencies involved in clusters of life outcomes.’ And ‘For some purposes it may be desirable to assess competencies that are more generally useful in clusters of life outcomes, including not only occupational outcomes but social ones as well, such as leadership, interpersonal skills, etc.’ It is evident here ‘competencies’ refers to something different. We can say ‘competencies’ (singular ‘competency’) is a new word created to indicated the personal features by which competence depends (see the Interview with McClelland quoted below).
In the article McClelland lists some ‘traditional competencies’: ‘reading, writing and calculating skills.’ Then continues: ‘Other (competencies) should involve what traditionally have been called personality variables (…) . Let me give some illustrations’ (10). McClelland lists here: ‘Communication skills’, ‘Patience’, ‘Moderate goal setting’, ‘Ego development’.
Is worth noting McClelland calls competencies some kind of skills (reading, writing, calculating, communication) but not the manual (technical) skills described when talking about the project ABLE, and in its example doesn’t introduce a more general technical competence that includes all the 50 ‘separate skills’ of the project ABLE. Instead, after having said the ABLE approach produces too many tests, he switches to clusters of social outcomes, criticizing the ABLE project also because ‘unable to get a simple index of whether or not the attendant is pleasant to the customer’ (9).
Some useful additional hints about the meaning of the word ‘competencies’ (and its singular ‘competency’) come from a late interview to McClelland (Adams, K. (1997) Interview with David McClelland). In the interview McClelland says: “’Competency’ was a term that was coined to replace the narrower term ‘skill’. In the USA, you can go to a technical high school and learn how to be a filling-station attendant. You practise unscrewing the cap, putting the petrol in, and so on. But, from my observation, that’s relatively unimportant in determining how often a customer stops at the station. More important is whether the attendant smiles or growls at you.”
So it comes out the term ‘competencies’ (singular ‘competency’) has been created first of all to stress the importance in recruitment and in job performance of ‘personality variables’ on respect of manual and technical skill.
The entire article may so be seen (and has been understood by many readers and researchers) under a different light. The critics of McClelland are addressed not only to the importance given in recruitment and education to IQ and aptitudes, but also to the importance given in recruitment to manual and technical skills.
Later the definition will be changed by Boyatzis, a collaborator of McClelland. According to Boyatzis a competency is ‘an underlying characteristics of an individual, which is causally related to effective or superior performance in a job’ such as ‘a motive, trait, skill, aspect of one’s self image or social role, or a body of knowledge which he or she uses’. Skills of all kinds (including technical and manual, big and small) are now included.
Autore Leonardo Evangelista. Leonardo Evangelista si occupa di orientamento dal 1993. Articolo contenuto sul sito www.orientamento.it. Leggi Informativa privacy, cookie policy e copyright.