Methods for Assessing a Candidate’s File for Reappointment, Tenure, and Promotion | Department of Economics Handbook | Kent State University

Methods for Assessing a Candidate’s File for Reappointment, Tenure, and Promotion

This section outlines the methods for assessing a candidate’s file for reappointment, tenure, and promotion.  This includes how to evaluate the quantity and the quality of research, how to evaluate teaching as defined by course design and course delivery, and how to evaluate service.  This section only discusses the methods for assessment; the criteria for tenure, promotion, and reappointment are contained in sections B, C, and D.

1.  Method for Assessing Research

In evaluating a candidate’s research portfolio, both the quantity and quality of research is considered. 

a.  Assessing the Quality of Peer-Reviewed Journal Articles

Economics is a large, diverse discipline with hundreds of peer-reviewed journals.  In order to apply a consistent standard in evaluating the quality of journals, the department uses a journal quality index (JQI) for peer-reviewed journals in business and economics that has been constructed based on factors such as external international rankings of journals, journal influence scores, and peer institutions.    The JQI is fairly comprehensive, with almost 3,000 Business and Economics journals included.  The JQI is continuous on a scale from 0 to 10, but the distribution is skewed with more than forty percent of journals having a score of zero.  A JQI of 0 means the journal is peer-reviewed but is not ranked on international lists, is not targeted by peer institutions, and has a low journal influence score.  Given the skewed nature of the JQI score, reference is made to the percentile in the JQI distribution rather than the score.  The index will be periodically updated and candidates have the option of using the JQI percentile at the time the paper was submitted or the JQI percentile at the time the file is evaluated if the JQI percentile has changed.  If a faculty member publishes in a journal that is not on the JQI list, the faculty member should provide documentation on the journal quality.  The method used to calculate the JQI is described in Appendix B.

 In evaluating research, the department considers the four broad tiers of research outlined below.  These four tiers are meant as guides, with the recognition that journal quality is a continuous measure and there will be variation within tiers.  In addition, the external reviewer’s evaluation of the quality of the research and other indications of quality (such as particularly high citations for an article, best paper awards, etc.) are important factors in assessing the quality of published articles independently of the specific journal the article is published in.   

 Tier 1:  Highest Influence Journals

Tier 1 research is indicated by publication in journals that have the highest level of influence on the discipline.  This category includes the highest impact general interest journals and the top broad field journals.  Examples, with their JQI percentile at the time of the handbook revision in Fall, 2014, in parenthesis, include American Economic Review (100%), the American Economic Journal:  Applied Economics (98.3%), Economic Journal (97.9%), Journal of Monetary Economics (99.0%), Journal of Labor Economics (97.9%), Journal of Applied Econometrics (97.6%), Journal of Public Economics (97.2%), Journal of International Economics (98.6%), Rand Journal of Economics (99.0%), and Journal of Urban Economics (96.0%).  Journals in Tier 1 have a JQI in the top 5% of all journals on the JQI list.

 Tier 2:  High Influence Journals 

Tier 2 research is indicated by publication in journals that have a high level of influence on the discipline.  This category includes secondary general interest journals, well-read field journals, and top journals with a focus on more narrowly defined subfields.  Examples include Oxford Economic Papers (88.2%), Southern Economic Journal (83.2%), Macroeconomic Dynamics (78.1%), National Tax Journal (82.0%), Review of International Economics (82.8%), Journal of Regional Science (89.9%), Journal of Economic Behavior and Organization (90.3%), Economics of Education Review (88.2%), and Journal of Forecasting (81.4%).  Journals in Tier 2 have a JQI in the 75th to 95th percentile of all journals on the JQI list.

 Tier 3:  Journals with Influence

Tier 3 research is indicated by publication in externally validated journals that have an influence on the discipline but are more limited in readership.  This category includes lower-level general and field journals. Examples include Eastern Economic Journal (64.6%), Journal of Applied Economics (59%), Journal of Economics (71.4%), Quarterly Review of Economics and Finance (71.4%), Education Economics (69.4%), Review of Development Economics (62.8%), Public Finance Review (44.1%), and History of Economics Review (44.1%).  Journals in Tier 3 are externally validated by international journal rankings, influence factors, and peer institutions such that they have a positive JQI but are outside of the top quartile of all journals on the JQI list. 

 Tier 4:  Peer Reviewed Journals With Limited External Validation of Influence

Tier 4 research is indicated by publication in peer reviewed journals that have a limited external validation of their influence on the field.  Examples include Indian Journal of Economics, Global Economic Review, Northern Economic Review, Ethics and Economics, Business and Economics Research Journal, and Journal of Forensic Economics.  These journals have a JQI of zero indicating that they are not ranked on international lists, are not targeted by peer institutions, and have a low journal influence score.  More than 40 percent of all journals in business and economics fall into the category of Tier 4.  Research in Tier 4 will be considered for candidates at the regional campuses, but will receive no weight for candidates at the Kent campus. 

 b.  Assessing the Quantity of Peer-Reviewed Journal Articles

 The department generally expects a publication record at the Kent campus that averages about one publication per year of the probationary period and an average at the regional campuses of about two publications every three years.  However, given the length of time required to establish a research agenda and the long lag time in publication, it is not unusual in Economics to have most of the publications come near the end of the probationary period. 

 The department recognizes there may be a tradeoff between the quantity of publications and the quality of publications.  As reflected in the criteria outlined in sections B-D, a lower quantity of higher-quality publications is valued and a higher quantity of lower-quality publications is valued provided these publications meet the quality threshold within the criteria.   

 The Department values jointly authored scholarship as well as individually authored scholarship; there is no requirement for sole-authored scholarship.  While in some disciplines the order authors are listed is important, in Economics the order of authors is generally alphabetical; not being the “first author” does not imply a lesser role in the authorship of the paper unless explicitly stated.  The candidate’s file should include an indication of his/her relative contribution to each co-authored paper.

 c.  Other Factors Assessed in Evaluating the Research Portfolio

 While emphasis is placed on peer reviewed journal articles,  the successful funding of external grants, published books, published book chapters, and prestigious invited research presentations are also valued as part of  the research portfolio of a candidate.  In evaluating the quality of research grants, both the dollar amount of the grant and the selectivity of the funding agency will be considered.  Finally, as noted in the section on assessing the quality of peer-reviewed research, the external reviewers’ evaluations of the importance and quality of the research are used in evaluating the totality of the research portfolio.  While the criteria in sections B, C, and D focus on peer-reviewed journal articles, a strong record in grants, books, book chapters, presentations, or exceptionally strong external letters may cause a research portfolio to be rated higher; similarly, external letters that indicate the record is not as strong as the criteria below would suggest may cause a research portfolio to be rated lower.

 2.  Methods for Assessing Teaching

 In evaluating a candidate’s teaching portfolio, both course design and course delivery is considered.  Greater emphasis is placed on teaching near the tenure decision than earlier teaching.

 a. Assessing the Quality of Course Design

 Course design focuses on the structure of the course, assessments, and content.  The quality of the course design will be assessed based on the peer teaching reviews, quantitative and qualitative results in student surveys, course materials such as syllabi, exams, and assignments, and the faculty narrative about teaching. 

 i) Effective Course Design

A faculty member will have an effective course design if the course is designed in a manner that provides the student knowledge and skills required for basic application of the course content.  Examples of this include a course that is organized, covers the appropriate content, maintains currency, and has appropriate assessments.  It is expected that all courses will meet the standard of effective course design.  Major design issues raised in earlier evaluations are expected to be addressed and improved upon.

 ii) Exemplar Course Design

Exemplar course design exceeds the effective standard with courses that have intentional planning designed to help students achieve significant learning.  While the nature of an exemplar course design may vary depending on the class enrollment, subject content, and program, examples of this may include a variety of assessment tools specifically suited for the course, the integration of material from a variety of sources, design features that facilitate advanced application of the course content, etc.  The faculty member must articulate in the narrative on teaching the intentional planning that went into the course design. 

 b.  Assessing the Quality of Course Delivery

 Course delivery focuses on the act of teaching, including what happens in front of the classroom and other student interactions.  For fully on-line courses, it includes the communication of material in any multimedia materials included in the course and interactions with students throughout the course.  The quality of course delivery will be assessed based on peer teaching reviews and quantitative and qualitative results in student surveys.

 i) Effective Course Delivery

A faculty member will have effective course delivery if they are able to convey course content that provides the students knowledge and skills required for basic application of the course content.  For example, this includes being understandable, clear, organized, and respectful.  Effective course delivery should be demonstrated by peer reviews and student evaluations that demonstrate effective communication of course material.  Major delivery issues raised in earlier evaluations are expected to be addressed and improved upon.   

 ii) Exemplar Course Delivery

Exemplar course delivery exceeds the effective standard with delivery that helps students achieve significant learning.  Exemplar course delivery should be demonstrated by peer reviews and student evaluations that demonstrate exceptional communication of course material. 

 c.  Other Factors Assessed in Evaluating the Teaching Portfolio

 Although emphasis is put on the quality of course design and course delivery, published research on teaching, participation in teaching conferences, and professional development in teaching are also valued as part of the teaching portfolio of a candidate.  While the criteria in sections B, C, and D focus on quality of course design and delivery, a strong record in these other factors may cause a teaching portfolio to be rated higher.  However, it is not necessary to have any of these other factors in order to meet the criteria below.

 3.  Methods for Assessing Service

 In assessing a faculty member’s service record, service to the department, college, campus, university, community, and profession will be considered.  Both the quantity of service and the quality of service are important.