The History & Citation Impact of Applied Psychological Measurement
Journal Interview, May 2011
In a recent analysis of Essential Science IndicatorsSM from Thomson Reuters data, the journal Applied Psychological Measurement (APM) entered the rankings of highly cited journals in the field of Psychiatry & Psychology. The journal's current record includes 320 papers cited a total of 1,542 times between January 1, 2000 and December 31, 2010.
APM was founded in 1976 by Dr. David J. Weiss. It is now published by SAGE Publications, Inc., and edited by Dr. Mark L. Davison.
Did you expect Applied Psychological Measurement (APM) to become highly cited, or is this surprising to you?
We were pleasantly surprised by this improvement in our citations. Like many journals, APM gives some consideration to the likelihood of citation in its editorial process, and we have been trying to improve our citation index. But our evaluations of articles do not consider the likelihood of citation explicitly. Our major criteria are "significance" (to the field of applied psychological measurement), "appropriateness" (fit to the journal's content mission), "clarity of expression," and" technical adequacy." Likelihood of influencing practice plays a bigger role in our decisions than does likelihood of citation.
This result is not completely surprising, however, as the measurement methodologies published in APM have been slowly spreading from applications in psychology and education to other fields, in particular medical behavior and medical outcomes research.
How would you account for the high citation rate of APM?
One reason for the increased citation rate may be the increased number of subscribers. From 2006 to 2009, the number of subscribers has more than tripled, rising from 1,369 to 5,128. The increased subscription base may mean an increase in the number of potential authors who may cite the journal.
"At the time the journal was created, there were journals dealing with statistical and technical issues in educational measurement but no journals devoted to similar issues in psychological measurement."
We try to publish articles that are readable and relevant to practice. Most of our articles introduce new methods or provide a comparative evaluation of several competing methods. These latter types of articles are designed to help test developers and researchers decide how best to address a problem in their work. We try to alert researchers and practicing professionals to new developments in relevant software. We also try to provide reviews of highly specialized books relevant to our field.
In a nutshell, our goal is to publish useful information.
Would you give us a brief history of the journal?
The journal was founded by Dr. David J. Weiss in the Department of Psychology at the University of Minnesota in 1976. He created a private, nonprofit corporation (Applied Psychological Measurement, Inc.) whose sole purpose was to publish the journal. He served as editor for 25 years. Dr. Mark Reckase of Michigan State University served as the second editor, and I succeeded him.
When the founder, Dr. Weiss, decided to step down, he decided to sell the journal to SAGE Publications, Inc. as a way to ensure the journal's continuity. Funds from the sale were used to create a foundation (the Applied Psychological Measurement Foundation) whose income has been used to fund graduate student research at universities around the world in the area of educational and psychological measurement methodologies.
At the time the journal was created, there were journals dealing with statistical and technical issues in educational measurement but no journals devoted to similar issues in psychological measurement. The journal focus is on methods applicable in psychological measurement or methods applicable in both educational and psychological measurement.
At the time the journal was founded, item response theory (a leading-edge technology for psychological measurement) and computerized testing were emerging as important new developments. Throughout its history, APM has had a special interest in computerized testing and the item response theories on which it is based.
What historical factors have contributed to the success of APM? Have there been specific developments in the fields served by APM that may have contributed?
One of the most important was is the emergence of both item response theory and computerized testing in higher education admissions, military testing, and professional licensing.
Another is the continuing societal concern about test fairness to women and minorities. The journal has published many articles on "differential item functioning," which involves methods of studying test fairness.
While this may not be an historical trend, of late we have been receiving and publishing articles on the detection and prevention of cheating.
The accountability movement in education has also been an influence.
What, in your view, is this journal's main significance or contribution in the field of Psychiatry & Psychology?
"We try to publish articles that are readable and relevant to practice."
In the past, APM's main contributions have been its role in the development of computerized testing and assessment. This has influenced all sorts of assessments in education (including medical education), industrial organizational psychology, counseling psychology, clinical psychology, psychiatry, and some areas of health care (e.g., mental functioning of the elderly in gerontology and the development of instruments for measuring patient-based outcomes of medical treatments).
How do you see your field(s) evolving in the next few years?
Computerized testing will continue to develop, but I think it will develop in new directions. Currently, computerized tests typically employ multiple-choice items, but in ways which allow examinations to (a) maintain accuracy but shorten test length or (b) increase accuracy while holding test length constant. It also allows us to report results far more quickly. Rather than employing multiple-choice response formats, future tests will use response formats that employ joysticks and touch screens to create objectively scorable test items that are not multiple choice.
Also, item presentation formats will change and become much more dynamic than is possible with a printed test booklet. Questions will become more interactive so that the questions posed at each stage will depend on the answers at earlier stages; in the future, the computer will "probe" examinee answers. These new testing methodologies will require new psychometric developments to handle the unique measurement problems that they pose.
In many current tests, the total score is very useful but subscores are less so. Future research will be designed to improve the information and interpretation of the subscores. For instance, an evaluation of employees may provide a useful "total" score evaluating the overall performance of the employee, but the subscores describing information in specific areas (leadership, team building, providing feedback) is often not nearly so useful.
Future work will be designed to improve the reliability and validity of subscores without markedly increasing the test length. By improving subscores, the tests will be providing more information and will become more useful for diagnosis, vocational placement, differentiation of instruction, and other applications.
Farther into the future, research on brain scanning techniques (e.g., fMRI) might lead to completely new methods for the measurement of such things as emotions and mental processing. However, it will take some time before these developments begin to affect applied psychological measurement.
What role do you see for your journal?
The journal's focus is on more technical aspects of the testing process. The journal will continue to publish new methods and evaluations of alternative methods for addressing problems in the field, including the problems described above. It will alert readers to software that can be used to address the problems and to highly specialized books describing possible solutions and research methods for evaluating those solutions. And it will continue to publish leading-edge methodological developments that address issues in all applications of psychological measurement techniques.
Applied Psychological Measurement
Mark L. Davison, Editor
David J. Weiss, Founder and Editor Emeritus
SAGE Publications, publishers
In February 2011, Applied Psychological Measurement was a named is a New Entrant in Psychiatry & Psychology.
KEYWORDS: APPLIED PSYCHOLOGICAL MEASUREMENT, MEASUREMENT METHODOLOGIES, PSYCHOLOGY, EDUCATION, MEDICAL BEHAVIOR, MEDICAL OUTCOMES, SUBSCRIPTION BASE, COMPARATIVE EVALUATION, COMPETING METHODS, TEST DEVELOPERS, RESEARCHERS, PRACTICING PROFESSIONALS, FOUNDATION, ITEM RESPONSE THEORY, COMPUTERIZED TESTING, HIGHER EDUCATION ADMISSIONS, MILITARY TESTING, PROFESSIONAL LICENSING, TEST FAIRNESS, WOMEN, MINORITIES, DIFFERENTIAL ITEM FUNCTIONING.