In Search of the Perceived Quality and Impact of Accredited South African Law Journals: Exploring the Possibility of a Ranking System. A Baseline Study: 2009 - 2014
DOI:
https://doi.org/10.17159/1727-3781/2018/v21i0a3459Keywords:
ranking, journals, law, quality, impact, subsidy, Department of Higher Education and Training, public universitiesAbstract
The DHET Research Output Policy (2015) indicates that there has been a change in the government’s approach to research funding. Previously all research published in any accredited journal was rewarded equally. A decision has been taken, however, that a shift will be made towards rewarding better quality and higher impact peer-review research. Additional mechanisms such as biometric/bibliometric data, including citations, assessments by discipline-specific panels of experts and/or post-publication reviews may be used to determine the quality and impact of publications. The policy notes that the DHET may distinguish between "high" and "low" impact journals after proper consultation.
This article highlights the need for consultation by the legal fraternity with the DHET about the implementation of these possible mechanisms in the light of the special considerations applicable to the evaluation of law journals: most journals publish mainly local legal content, there is a limited number of active legal academics, the nature of legal research is not empirical, and a premium is placed on the writing of books.
The research evaluates the available data between 2009 and 2014 in an attempt to assess if it would be appropriate to introduce a legal journal ranking system in South Africa. The article discusses direct and indirect forms of quality evaluation to inform possible ranking systems. This includes the data from the ASSAf expert panel evaluation of law journals in 2014 and other bibliometric data based on whether the journal is featured in international accredited lists, the size of its print-run, author prominence, rejection-rate, usage studies, and evaluations based on citations. An additional ranking system is considered, based on the five best outputs submitted to the National Research Foundation by applicants applying for rating.
The article concludes that a law journal ranking system would be inappropriate for South Africa. None of the systems meet the minimum requirements for a trustworthy ranking of South African law journals, as the data available are insufficient, non-verifiable and not based on objective quality-sensitive criteria. Consultation with the DHET is essential and urgent to avoid the implementation of inappropriate measures of quality and impact assessmen
Downloads
References
Bibliography
Literature
Altbach 2001 Higher Education
Altbach PG "Academic Freedom: International Realities and Challenges" 2001 Higher Education 205-219
ASSAf Report on Strategic Approach to Research Publishing
ASSAf Report on a Strategic Approach to Research Publishing in South Africa (ASSAf Pretoria 2006)
ASSAf Report on Grouped Peer Review of Scholarly Journals in Law
ASSAf Report on Grouped Peer Review of Scholarly Journals in Law and Related Legal Fields (ASSAf Pretoria 2014)
Brophy 2006 Connecticut Law Review
Brophy AL "Law [Review]'s Empire: The Assessment of Law Reviews and Trends in Legal Scholarship" 2006
Connecticut Law Review 101-107
Budden et al 2008 Trends Ecol Evol
Budden AE et al "Double-Blind Review Favours Increased Representation of Female Authors" 2008 Trends Ecol Evol 4-6
Campbell, Goodacre and Little 2006 J L & Soc'y
Campbell K, Goodacre A and Little G "Ranking of UK Law Journals. An Analysis of the Research Assessment Exercise (RAE) 2001 Submissions and Results" 2006 J L & Soc'y 335-363
Carnelley 2015 Obiter
Carnelley M "Identifying Predatory Open-Access Academic Journal Publishers, in Light of the South African Department of Higher Education and Training's Decision to Retrospectively De-accredit Certain Journals" 2015 Obiter 519-538
Crespi 1997 Int'l Law
Crespi GS "Ranking International and Comparative Law Journals: A Survey of Expert Opinion" 1997 Int'l Law 869-886
Crespi 1998 Wm & Mary Envtl L & Pol'y Rev
Crespi GS "Ranking the Environmental Law, Natural Resources Law and Law Use Planning Journals: A Survey of Expert Opinion" 1998 Wm & Mary Envtl L & Pol'y Rev 273-297
Crespi 1999 Fla St U L Rev
Crespi GS "Ranking Specialised Law Reviews: A Methodological Critique" 1999 Fla St U L Rev 837-849
Currie and Pandler 2011 J Bank Finance
Currie RR and Pandler GS "Survey. Finance Journal Rankings and Tiers: An Active Scholarly Assessment Methodology" 2011 J Bank Finance 7-20
Curtis 2008 Globalisation, Societies and Education
Curtis B "The Performance‐Based Research Fund: Research Assessment and Funding in New Zealand" 2008 Globalisation, Societies and Education 179-194
Datt, Tran and Tran-Nam 2009 ATF
Datt K, Tran A and Tran-Nam B "Ranking Tax Journals – The Way Forward" 2009 ATF 341-369
DHET Research Output Policy
Department of Higher Education and Training Research Output Policy (DHET Pretoria 2015)
DHET White Paper
Department of Higher Education and Training White Paper for Post-School Education and Training: Building an Expanded, Effective and Integrated Post-School System (DHET Pretoria 2013)
DHET Directorate Communiqué
Department of Higher Education and Training Directorate: University Policy and Development Support Communiqué 2 of 2015 (DHET Pretoria August 2015)
Doyle 2004 Leg Ref Serv Q
Doyle J "Ranking Legal Periodicals and some other Numeric Uses of the Westlaw and Lexis Periodical Databases" 2004 Leg Ref Serv Q 1-396
Editorial 2012 EJIL
Editorial 2012 "Impact Factor – The Food is Bad and What's More There is not Enough of It: EJIL – The Beginning of the Existential Debate; Masthead Changes; In this Issue" 2012 EJIL 607-612
Editorial 2015 Nature
Editorial 2015 "Nature Journals Offer Double-Blind Review" 2015 Nature 274
Eisenberg and Wells 2014 Economic Inquiry
Eisenberg T and Wells MT "Ranking Law Journals and the Limits of Journal Citation Reports" 2014 Economic Inquiry 1301-1314
European Union Expert Group on Assessment of University-based Research Assessing University-based Research
European Union Expert Group on Assessment of University-based Research Assessing University-based Research (Publications Office of the EU Luxembourg 2010)
Fuld 1953 NYU L Rev
Fuld SH "A Judge Looks at the Law Review" 1953 NYU L Rev 915-921
George and Guthrie 1999a Fla St U L Rev
George TE and Guthrie C "An Empirical Evaluation of Specialised Law Reviews" 1999 Fla St U L Rev 813-836
George and Guthrie 1999b Fla St U L Rev
George TE and Guthrie C "In Defence of Author Prominence: A Reply to Crespi and Korobkin" 1999 Fla St U L Rev 877-896
Given, Kelly and Wilson "Bracing for Impact"
Given LM, Kelly W and Wilson R "Bracing for Impact: The Role of Information Science in Supporting Societal Research Impact" in Proceedings of the 78th ASIS&T Annual Meeting: Information Science (6-10 November 2015 St Louis, MO) 1-10
Grossman 2003 Colum J Gender & L
Grossman JL "Feminist Law Journals and the Ranking Conundrum" 2003 Colum J Gender & L 522-531
HEFCE Research Excellence Framework
Higher Education Funding Council for England Research Excellence Framework 2014: Overview Report by Main Panel C and Sub-panels 16 to 26 (HEFCE Bristol 2015)
HEFCE Research Assessment Exercise 2001
Higher Education Funding Council for England Research Assessment Exercise 2001 Law Panel General Overview (HEFCE Bristol 2001)
Jarvis and Coleman 1997 Arizona L Rev
Jarvis RM and Coleman PG "Ranking Law Reviews: An Empirical Analysis Based on Author Prominence" 1997 Arizona L Rev 15-24
Jarvis and Coleman 2007 L Libr J
Jarvis RM and Coleman PG "Ranking Law Reviews by Author Prominence – Ten Years Later" 2007 L Libr J 573-588
Kahn 2011 SAJS
Kahn M "A Bibliometric Analysis of South Africa's Scientific Outputs – Some Trends and Implications" 2011 107(1/2) SAJS 1-6
Korobkin 1999 Fla St U L Rev
Korobkin R "Ranking Journals: Some Thoughts on Theory and Methodology" 1999 Fla St U L Rev 851-876
Liefner 2003 Higher Education
Liefner I "Funding, Resource Allocation and Performance in Higher Education System" 2003 Higher Education 469-489
Mallard, Lamont and Guetzkow 2009 Sci Technol Hum Values
Mallard G, Lamont M and Guetzkow J "Fairness as Appropriateness. Negotiating Epistemological Differences in Peer Review" 2009 Sci Technol Hum Values 573-606
Marashi et al 2015 SAJS
Marashi S-A et al "Can Scientific Journals be Classified Based on their 'Citation Profiles'?" 2015 SAJS 1-3
Maru 1976 Am B Found Res J
Maru O "Measuring the Impact of Legal Periodicals" 1976 Am B Found Res J 227-249
McWhirter Legal 100
McWhirter DA The Legal 100: A Ranking of Individuals Who Have Most Influenced the Law (Birch Lane Press Secaucus 1998)
Ministry of Education Policy and Procedures
Ministry of Education Policy and Procedures for Measurement of Research Output of Public Higher Education Institutions (Department of Education Pretoria 2003)
Moed, Van Leeuwen and Reedijk 1998 J Doc
Moed HF, Van Leeuwen Th N and Reedijk J "A New Classification System to Describe the Ageing of Scientific Journals and Their Impact Factors" 1998 J Doc 387-419
Mouton Bibliometric Analysis
Mouton J A Bibliometric Analysis of the State of Research at UKZN. Final Report (Centre for Research on Evaluation, Science and Technology Stellenbosch 2015)
Mouton and Valentine 2017 SAJS
Mouton J and Valentine A "The Extent of the South African Authored Articles in Predatory Journals" 2017 SAJS 1-9
NRF/RISA Evaluation and Rating
NRF/RISA Evaluation and Rating. Guidelines and Instructions for Completing the Application for Evaluation and Rating via the NRF Online Submission System (NRF/RISA Pretoria 2015)
OCLC Research Assessment Regimes
Online Computer Library Centre A Comparative Review of Research Assessment Regimes in Five Countries and the Role of Libraries in the Research Assessment Process. A Pilot Study (OCLC Truro 2009)
Osterloh and Frey Research Governance in Academia
Osterloh M and Frey BS Research Governance in Academia: Are There Alternatives to Academic Rankings? (Institute for Empirical Research in Economics Zurich 2009)
Perry 2006 Va J Law Technol
Perry R "The Relative Value of American Law Reviews: A Critical Appraisal of Ranking Methods" 2006 Va J Law Technol 1-40
Pouris and Pouris 2015 SAJS
Pouris AEM and Pouris A "An Assessment of South Africa's Research Journals: Impact Factors, Eigenfactors and Structure of Editorial Boards" 2015 111(3/4) SAJS 1-8
Pritchard 1998 Minerva
Pritchard RMO "Academic Freedom and Autonomy in the United Kingdom and Germany" 1998 Minerva 101-124
Ramsay and Stapledon 1997 MULR
Ramsay I and Stapledon GP "A Citation Analysis of Australian Law Journals" 1997 MULR 676-692
Smyth 1998 U Tas LR
Smyth R "Academic Writing and the Courts: A Qualitative Study of the Influence of Legal and Non-Legal Periodicals in the High Court" 1998 U Tas LR 164-182
Smyth 2012 UNSWLJ
Smyth R "Who Publishes in Australia's Top Law Journals?" 2012 UNSWLJ 201-245
Steinbuch 2009 Loy LA L Rev
Steinbuch R "On the Leiter Side: Developing a Universal Assessment Tool for Measuring Scholarly Output by Law Professors and Ranking Law Schools" 2009 Loy LA L Rev 87-123
Svantesson 2009 Legal Studies
Svantesson DJB "International Ranking of Law Journals – Can it be done and at What Costs?" 2009 Legal Studies 678-691
Svantesson and White 2009 Bond LR
Svantesson DJB and White P "Entering an Era of Research Ranking – Will Innovation and Diversity Survive?" 2009 Bond LR 173-191
Van Gestel 2015 Legal Studies
Van Gestel R "Sense and Non-Sense of a European Ranking of Law Schools and Law Journals" 2015 Legal Studies 165-185
Van Gestel and Vrancken 2011 GLJ
Van Gestel R and Vrancken J "Assessing Legal Research: Sense and Nonsense of Peer Review Versus Bibliometrics and the Need for a European Approach" 2011 GLJ 901-929
WITS Research Publication Incentive (RINC) Policy
University of the Witwatersrand Faculty of Commerce, Law and Management Faculty Research Committee Research Publication Incentive (RINC) Policy (WITS Johannesburg circa 2015)
Yoon 2013 JLA
Yoon AH "Editorial Bias in Legal Academia" 2013 JLA 309-339
Internet sources
Akademie van Wetenschappen 2014 http://www.knaw.nl/nl/actueel/nieuws/
wetenschapsorganisaties-presenteren-nieuw-evaluatieprotocol-voor-onderzoek
Akademie van Wetenschappen 2014 Standard Evaluation Protocol 2015 – 2021 http://www.knaw.nl/nl/actueel/nieuws/wetenschapsorganisaties-presenteren-nieuw-evaluatieprotocol-voor-onderzoek accessed 10 October 2017
Elsevier date unknown https://www.elsevier.com/solutions/scopus
Elsevier date unknown About Scopus https://
www.elsevier.com/solutions/scopus accessed 10 October 2017
IBSS 2013 http://media2.proquest.com/documents/IBSS+
Editorial+Policies+and+ Principles.pdf
International Bibliography of the Social Sciences 2013 IBSS Editorial Policies and Principles http://media2.proquest.com/ documents/
IBSS+Editorial+Policies+and+Principles.pdf accessed 10 October 2017
London School of Economics 2010 http://blogs.lse.ac.uk/impactofsocial
sciences/files/ 2010/11/chart3.jpg
London School of Economics 2010 Maximising the Impact of Academic Research http://blogs.lse.ac.uk/impactofsocialsciences/files/2010/11/chart3.jpg accessed 10 October 2017
NSD date unknown https://dbh.nsd.uib.no/publiseringskanaler/OmKriterier.
action?requestlocale=en
Norwegian Register for Scientific Journals, Series and Publishers date unknown Criteria for Inclusion of New Scientific Publication Channels https://dbh.nsd.uib.no/publiseringskanaler/OmKriterier.action?request_locale=en accessed 10 October 2017
PBRF date unknown http://www.tec.govt.nz/funding/funding-and-performance/
New Zealand Performance-Based Research Fund date unknown Funding and Performance http://www.tec.govt.nz/Funding/funding-and-performance/ accessed 10 October 2017
SciELO date unknown http://www.scielo.org.za/avaliacao/avaliacao_en.
htm
Scientific Electronic Library Online date unknown 'SciELO SA Criteria': Criteria, Policy and Procedures for Admission and Permanence of Scientific Journals in the South African Collection of SciELO http://www.scielo.org.za/avaliacao/avaliacao_en.htm accessed 10 October 2017
Testa 2016 http://wokinfo.com/essays/journal-selection-process/
Testa J 2016 Journal Selection Process http://wokinfo.com/essays/journal-selection-process/ accessed 10 October 2017
Tome and Lipu 2004 http://citeseerx.ist.psu.edu/viewdoc/download?doi=
1.1.897.6809&rep=rep1&type=pdf
Tome L and Lipu S 2004 Research and Development Discussion Paper #6: Indicators of Journal Quality http://citeseerx.ist.psu.edu/viewdoc/download
?doi=10.1.1.897.6809&rep=rep1&type=pdf accessed 19 December 2017
UKZN Research Office date unknown http://research.ukzn.ac.za/DoHETAccredited Journals.aspx
UKZN Research Office date unknown DHET Accredited Journals http://research. ukzn.ac.za/DoHETAccreditedJournals.aspx accessed 10 October 2017
Washington and Lee University 2009-2014 http://lawlib.wlu.edu/LJ/
Washington and Lee University 2009-2014 Law Journals: Submissions and Ranking, 2009 – 2014 http://lawlib.wlu.edu/LJ/ accessed 10 October 2017
WITS date unknown https://www.wits.ac.za/sajhr/#sthash.OYn340c9.dpuf
University of the Witwatersrand date unknown South African Journal on Human Rights https://www.wits.ac.za/sajhr/#sthash.OYn340c9.dpuf accessed 10 October 2017
LIST OF ABBREVIATIONS
AHRLJ African Human Rights Law Journal
Am B Found Res J American Bar Foundation Research Journal
Am Sociol Rev American Sociological Review
Annual Survey Annual Survey of the South African Law
Arizona L Rev Arizona Law Review
ASSAf Academy of Science of South Africa
ATF Australian Tax Forum
Bond LR Bond Law Review
CARSA Child Abuse Research in South Africa
CCMA Commission for Conciliation, Mediation and Arbitration
CESifo Centre for Economic Studies Publishing Group
Chi-Kent L Rev Chicago-Kent Law Review
CILSA Comparative and International Law Journal of Southern Africa
Colum J Gender & L Columbia Journal of Gender and the Law
Conn L Rev Connecticut Law Review
DHET Department of Higher Education and Training
EJIL European Journal of International Law
Fla St U L Rev Florida State University Law Review
GLJ German Law Journal
Haifa L Rev Haifa Law Review
HEFCE Higher Education Funding Council for England
IBSS International Bibliography of the Social Sciences
ILJ Industrial Law Journal
Ind LJ Indiana Law Journal
Int'l J Legal Info International Journal of Legal Information
Int'l Law The International Lawyer
ISI Thomson Reuters Web of Science (formerly
ISI Web of Knowledge)
IT Information Technology
J Bank Finance Journal of Banking and Finance
J Doc Journal of Documentation
J L & Soc'y Journal of Law and Society
JJS Journal of Juridical Science
JLA Journal of Legal Analysis
L Libr J Law Library Journal
LDD Law, Democracy and Development
Leg Ref Serv Q Legal Reference Services Quarterly
Loy LA L Rev Loyola of Los Angeles Law Review
Merc LJ South African Mercantile Law Journal
MULR Melbourne University Law Review
NMMU Nelson Mandela Metropolitan University
NRF National Research Foundation
NSD Norwegian Register for Scientific Journals,
Series and Publishers
NYU L Rev New York University Law Review
NZ New Zealand
OCLC Online Computer Library Centre
PBRF New Zealand Performance-Based Research Fund
PER Potchefstroom Elektroniese Regstydskrif
RISA Research and Innovation Support and Advancement
SACJ South African Journal of Criminal Justice
SAJELP South African Journal of Environment al Law and Policy
SAJHR South African Journal of Human Rights
SAJS South African Journal of Science
SALJ South African Law Journal
SAYIL South African Yearbook of International Law
Sci Technol Hum Values Science, Technology and Human Values
SciELO Scientific Electronic Library Online
Stell LR Stellenbosch Law Review
THRHR Tydskrif vir Hedendaags Romeins-Hollandse Reg
Trends Ecol Evol Trends in Ecology and Evolution
TSAR Tydskrif vir die Suid-Afrikaanse Reg
U Tas LR University of Tasmania Law Review
UK United Kingdom
UKZN University of KwaZulu-Natal
UNSWLJ University of New South Wales Law Journal
US / USA United States of America
Va J Law Technol Virginia Journal of Law and Technology
WITS University of the Witwatersrand
Wm & Mary Envtl L & Pol'y Rev William and Mary Environmental Law and
Policy Review
Published
Issue
Section
License
Copyright (c) 2018 Marita Carnelley
This work is licensed under a Creative Commons Attribution 4.0 International License.
.png){kind=spacer}
