Format Aside: Applying Beall’s Criteria to Assess the Predatory Nature of Both OA and Non-OA Library and Information Science Journals

Format Aside: Applying Beall’s Criteria to Assess the Predatory Nature of Both OA and Non-OA Library and Information Science Journals

Jeffrey Beall’s blog listing of potential predatory journals and publishers, as well as his Criteria for Determining Predatory Open-Access (OA) Publishers are often looked at as tools to help researchers avoid publishing in predatory journals. While these Criteria has brought a greater awareness of OA predatory journals, these tools alone should not be used as the only source in determining the quality of a scholarly journal. Employing a three-person independent judgment making panel, this study demonstrates the subjective nature of Beall’s Criteria by applying his Criteria to both OA and non-OA Library and Information Science journals (LIS), to demonstrate that traditional peer-reviewed journals could be considered predatory. Many of these LIS journals are considered as top-tier publications in the field and used when evaluating researcher’s publication history for promotion and tenure.

Introduction

While the open access (OA) movement is growing throughout academia and has gained traction in some disciplines, there is still skepticism on the part of researchers who question the quality of OA publishing and may associate OA journals with “predatory” journals. Jeffrey Beall defines “predatory” journals as those that “exploit the gold open-access model for their own profit” while not engaging in a rigorous peer review process.1 Beall developed a tool on his blog to help researchers identify potential predatory journals and publishers. This tool is a list of potential predatory journals and publishers, as well as the Criteria for Determining Predatory Open-Access Publishers.

The authors of this interpretive analysis (hereafter referred to as the “analysts”) believe that users need to be cautioned about the subjective nature of the Criteria. They recommend discretion because the criteria used are so general that an evaluator might label a scholarly journal as “predatory” when it is not or may disregard the article as being without merit solely because it is published in a journal found on Beall’s List. Promotion and tenure committees may even look to this list when reviewing tenure candidate publications, thereby influencing future research.2 Although Beall noted that such committees need to “decide for themselves how importantly or not to rate articles published in these journals in the context of their own institutional standards and/or geocultural locus,”3 there is the potential to misconstrue and misapply the Criteria in terms of its objectivity. Indiscriminate application could affect promotion and tenure decisions, especially those decisions concerning faculty members in emerging fields of scholarship where needs for publishing outlets are met with OA journals.4

In this study, the analysts argue that the individual elements of Beall’s Criteria are so general in design that evaluators might broadly apply them as measures of illegitimacy to any refereed academic journal, regardless of publishing model. By applying Beall’s Criteria to a list of both OA and non-OA well-regarded refereed academic LIS journals, this study demonstrates the subjective nature of his Criteria. While the Criteria may serve as an aid to researchers’ selecting publication outlets, this Criteria is presently not adequate for making such determinations because its elements focus more on the publisher and/or journal attributes rather than the quality of articles published by the outlet.

Literature Review

The rising cost of journal subscriptions at the end of the 20th century created a “serials crisis” rendering libraries unable to afford to purchase all of the scholarly literature needed to support the research and educational missions of their institutions. However, by the 1990s, the World Wide Web became increasingly available to the general population and enabled the proliferation of online journals and, in turn, the growth of the OA movement.

According to Suber, OA scholarship represents an attempt to circumvent traditional obstacles to access.5 This practice of making content freely available to access online is significantly different from the traditional for-profit publication model that libraries were required to deal with to purchase access to scholarly information. Some publishers have transformed themselves from being content providers to their readers to being service providers to their authors. These publishers charge authors a fee to publish in their journals as a way of offsetting publication costs.6 Charging publication fees to authors is not a new concept. Many print journals have been charging authors page charges to offset the cost of publishing.7 Shen and Björk contended that basic journal practices and elements of article publication, including peer review, have not changed appreciably.8

Because some OA journals charge a fee, despite having no print costs to cover, the practice of OA journals charging fees has opened up debate about questions of quality, and whether authors are simply paying to have their work published (rather than publishing quality work). For example, a survey by Swan and Brown that compared the opinions of authors who had published in OA journals and those who had not yet published in OA showed that non-OA authors perceived OA journals “to be of lower reputation and prestige.”9 It can only be assumed that OA authors did not perceive OA journals the same way. Coonin and Younce also found, among social sciences and humanities researchers, the impression that “[OA] may be a less acceptable model, whether because research is often undertaken without substantial grant funding, or because (to some) it may sound a bit too close to ‘vanity press’ publishing.”10 John Bohannon highlighted his argument concerning OA quality in October 2013 when he performed his “sting operation” on OA journals. Bohannon composed a fraudulent paper about a lichen molecule that seemed to have cancer-fighting properties and submitted the paper to 304 OA journals. Even though he wrote the paper in a way that journal referees should have rejected it immediately, 157 OA journals accepted the paper.11 Davis pointed out that this high acceptance rate was not surprising, since “nearly half of the publishers were listed on Beall’s List of predator publishers,” further lending validity to Beall’s List as an authoritative standard.12 However, according to Bohannon, “Nearly half of all of the fee-charging open access publishers listed in the DOAJ [Directory of Open Access Journals] failed this test.”13 Because of this, the DOAJ is performing a reevaluation and cleanup of journals listed in their directory, having removed 3,300 of approximately 10,000 titles.14

Jeffrey Beall argues that there are some publishers (not all) manipulating this “author-pay model” for their own profit.15 Beall writes in his blog that these “fly-by-night” publishers operate as vanity presses for the sole purpose of generating income and do not follow professional publishing practices by disregarding peer review and digital preservation, and by spamming professional e-mail lists to solicit article submissions.16 Pickler et al. support Beall’s belief that predatory publishers engage in deceptive practices, including making false assertions about the publication’s status within a profession and providing inadequate or bogus information concerning the publisher or editorial board.17 According to Beall, in 2013 there were 225 publishers and 126 stand-alone journals he identified as “predatory,” while in 2015 this number increased to 693 publishers and 507 stand-alone journals.18 This indicated that the number of questionable journals and publishers is on the rise, causing much negative publicity for the OA movement.

In an attempt to shed light and encourage discussion about this problem, Beall continues to develop a set of Criteria for Determining Predatory Open-Access Publishers to help researchers determine if a journal is predatory. He uses these criteria to update continually a list of publishers that he believes are “potential, possible, or probable predatory scholarly open-access publishers.”19 In addition to having developed the Criteria, Beall hosts a website on which he provides a list of publishers and stand-alone journals that he considers to have failed one or more of these criteria. Beall’s Criteria recognizes the (1) Code of Conduct for Journal Publishers and (2) Principles of Transparency of Best Practice in Scholarly Journals, both published by the Committee on Publication Ethics (COPE), as foundational documents.20 COPE, which adapted the latter document from one used by the DOAJ, uses it for the purpose of membership evaluation.21 It is important to note, however, that COPE membership is open to journal editors regardless of whether or not their journal is OA, and that these two documents are not written specifically to be applied to OA publishers.22 Beall’s Criteria for Determining Predatory Open-Access Publishers is currently in its third edition. It contains 27 criteria divided into four categories: editor and staff, business management, integrity, and other. Beall provides individual criteria under these categories but does not include further explanations or definitions of the criteria themselves. Beall also includes a category titled Poor Journal Standards that consists of 26 additional criteria. However, since, according to Beall, these additional criteria “are considered to be reflective of poor journal standards,” but “do not equal predatory criteria,”23 the researchers chose not consider them in this study.

A number of researchers have criticized Beall’s List. Shen and Björk, for example, examined Beall’s List by using “a multistage stratified sampling method to take a look into the predatory publisher and journals on Beall’s List and generated their development trend over time.”24 They found that the problems caused by predatory journals are rather limited and regional, seeing the problem as created by institutions in developing countries trying to compete with the academic excellence viewpoint focusing on “how often one publishes” rather than on “what is published,” creating a market for dubious journals. Other researchers like Crawford believe Beall’s lists are subjective resources.25 Berger and Cirasella support Crawford’s findings, stating, “Imperfect English or a predominantly non-western editorial board does not make a journal predatory.”26 Despite these criticisms, the Criteria Beall uses to create his list is a good foundation for librarians to use when thinking about and identifying predatory journals and has been used by research as the foundation of critical evaluation of OA journals.27

Research Questions

The purpose of this study was to assess the subjectivity of Beall’s Criteria for Determining Predatory Open-Access Publishers, the same criteria by which he determines his lists. This task was accomplished by applying the 27 criteria to a collection of LIS journals, both OA and non-OA, which are generally considered to be of high quality because of their inclusion in a recognized source for identifying academic journal rankings. Again, the analysts did not consider Beall’s category area of criteria titled Poor Journal Standards in this study. They were not looking to declare any of these LIS journals as “predatory.” On the contrary, they have published in several of the LIS journals from the collection. Instead, the analysts investigated whether respected and long-established journals would pass Beall’s Criteria as if they were newly emerging journals without an established reputation. The analysts asked the following questions during this assessment:

  1. How many members of the population fail one or more of Beall’s Criteria?
  2. What are the percentages of failure for Beall’s Criteria by area (editor and staff, business management, integrity, and other)?
  3. What are the average ages of the failed journals?
  4. What type of publisher (association, commercial, or university press) had the largest number of failed journals?
  5. To what degree did the analysts agree in terms of their decision-making when applying Beall’s Criteria?
  6. What do the previous questions tell us about Beall’s List as a tool for evaluating scholarly journals?

Methodology

To analyze the subjectivity of Beall’s Criteria, the analysts examined a collection of scholarly journals in the field of LIS to determine if evaluators of the individual titles could perceive them as having predatory journal tendencies when applying Beall’s Criteria. The collection of journals tested against Beall’s Criteria came from the InCites Journal Citations Reports (JCR) section in Thomson Reuters’ Web of Science database. The JCR provided a category, information science & library science, consisting of 87 entries, seven of which were OA journals. At the time of retrieval, these 87 entries comprised the entire population of the database’s information science & library science topic category. At the time of this study, the most recent year of JCR listings compiled was 2014. It was not the analysts’ intention to analyze the JCR collection to make statistically generalizable claims concerning the applications of Beall’s Criteria to any larger population of LIS journals. The analysts selected the JCR list because it provides a convenient ranked collection of journals that also represents an assumed level of quality in the LIS community through its ranking of journals having Thomson Reuters Impact Factors.28

The analysts implemented an interpretive approach consisting of a three-person panel making independent judgments to determine whether a journal from the population of JCR LIS journals qualifies to be categorized as a “potential, possible, or probable predatory journal” according to Beall’s Criteria for Determining Predatory Open-Access Publishers.29 Other researchers have made use of panels of expert judges in both quantitative and qualitative research. Saljo, for example, used both quantitative and qualitative judging techniques to study learning approach and outcomes.30 Douglas, Douglas, McClelland, and Davies used a panel of three independent judges to determine levels of student satisfaction and dissatisfaction in U.K. higher education. Use of this technique allowed for making claims concerning the reliability of the judging process and the outcomes.31

During the course of a one-week period, the panel rated the JCR LIS entries separately. The three analysts are all academic librarians possessing ALA-accredited master’s degrees in LIS and 22 years’ total combined experience as professional librarians. They have authored a combined total of 23 refereed publications in peer-reviewed journals—four of which are published in journals listed in the JCR collection—and have themselves refereed 16 total articles for academic journals. The analysts were cognizant that their dual role as both authors and judges increased the potential for bias in any resulting analysis. They recognized, however, that bias is an inherent aspect of interpretive research and that conscious reflection throughout the research process is a method for mitigating such bias. To minimize bias, the analysts took part in a briefing session concerning the importance of reflexivity prior to beginning the analysis.

The analysts applied the 27 Criteria for Determining Predatory Open-Access Publishers to each journal title. They performed this task independently of each other, with no additional instructions for how to apply the Criteria beyond those given in Beall’s Criteria itself. That is, they were each required to interpret and apply the Criteria themselves. None of the three analysts had previously served as either editors or board members for any of the journals under consideration.

There was no time limit for applying the Criteria to journals during the evaluation process. There was no discussion allowed concerning possible interpretations of Beall’s Criteria, or as to whether or not a journal or its publisher met any of the 27 criteria of the Criteria. There was no discussion concerning the quality of any of the journals being reviewed. The only discussion allowed among the analysts during this period of review was only what was needed to determine and agree upon the identity of a particular listed journal if there was any doubt as to that identity (for example, if more than one journal exists with the same title).

Analysts recorded a YES or NO judgment for each of the 27 criteria, with YES meaning that their analysis of the journal indicated that it fit the description of a “potential, possible, or probably predatory publisher” according to the particular criteria, and NO meaning that it did not sufficiently meet the individual analyst’s interpretation of the particular criterion. All three of the analysts were unaware of each others’ decisions throughout the entire judging process.

Following completion of the qualitative judgment of the 87 entries, the three analysts’ decisions were tabulated for each journal. Individual criterion were assigned an overall YES designation (meeting the criteria for a “potential, possible, or probable predatory journal”) or a NO designation (failing to meet the criteria for a “potential, possible, or probably predatory journal”) depending on which of the two possible outcomes received the majority of votes (that is, two or three votes out of three). The analysts recorded the number of YES versus NO votes for each journal for further analysis.

Results

Of the 87 journal titles analyzed, the analysts removed six entries from the study due to journal title duplication, title changes, or the determination that the entry did not actually represent a peer-reviewed academic journal. The remaining 81 entries counted as individual journal titles (see appendix for a list of the 81 journals evaluated). Of these 81 journals examined, 45 of the journals failed at least one of Beall’s Criteria (again, this determination of failure required at least two of three YES votes on the part of the analysts). A closer look at these 45 journals in table 1 shows that journals can and have failed more than one criteria in the study, with 18 of the 45 journals (45%) failing more than one criteria.

Table 1

Journals That Failed One or More Criteria

Number of Journals

Number of Criteria Failed

27

1

10

2

4

3–6

4

7–9

Looking closer at the types of publishers: association, commercial, or university press, we see that all types of publishers failed at least one criterion (see table 2). In cases where journal publishers were affiliated with an academic institution, the analysts classified those as university presses. Commercial presses published the majority of failed journals (52%). It should be noted that the 17 instances of failed journals does not represent 17 different commercial presses, but that certain presses had multiple failed journal titles.

Table 2

Types of Publishers That Failed One or More Criteria

Type of Publisher

Number of Journals That Failed

Association Press

8

Commercial Press

17

University Press

8

Table 3 shows the number of criteria failed by age of journal, demonstrating that all of the journals that failed a criterion are at least 10 years old. The median age of journals in the ISI example is 32 years.

Table 3

Age and Number of Journals that Failed One or More Criteria

Age of the Journal

Number of Journals

Total Number of Criteria Failed

50–150 years old

9

28

30–49 years old

18

21

10–29 years old

18

49

0–9 years old

0

0

Beall divides his Criteria into four criteria areas. Table 4 highlights the percentage of total failed journals in each criteria area. The criteria area for editor and staff, which has nine specific criteria, had the largest failure percentage (77.78%). The criteria area for integrity, which encompasses seven specific criteria, was not the cause for any journal failure.

Table 4

Percentage of Total Failed Journals by Criteria

Criteria Area (Number of Criteria in Area)

% of Total Failed Journals

editor and staff (9)

77.78%

business management (6)

33.34%

integrity (7)

0%

other (6)

16.67%

Table 5 illustrates the number of journals that could be perceived as predatory by the top five failing criteria. The most failed criterion was under the criteria area business management: “Has no policies or practices for digital preservation, meaning that if the journal ceases operations, all of the content disappears from the internet.” Twenty-two journals failed this criterion. Three out of five of the top five most failed criteria fell into the editor and staff criteria area.

Table 5

Number of Failing Journals in the Top Five Failed Criteria

Failed Criteria

Criteria Area

Number of Failing Journals

Has no policies or practices for digital preservation, meaning that if the journal ceases operations, all of the content disappears from the internet.

business management

22

No single individual is identified as any specific journal’s editor.

editor and staff

17

No academic information is provided regarding the editor, editorial staff, and/or review board members (such as institutional affiliation).

editor and staff

13

There is little or no geographical diversity among the editorial board members, especially for journals that claim to be international in scope or coverage.

editor and staff

12

Have a “contact us” page that only includes a web form or an e-mail address, and the publisher hides or does not reveal its location.

other

11

Interanalyst Agreement Statistics

The analysts calculated interanalyst agreement statistics for individual criterion as well as the criteria areas. They excluded two individual criteria from the ranking because they had too many missing cells to get an accurate agreement percentage rating. These excluded criteria were “The publisher falsely claims to have its content indexed in legitimate abstracting and indexing services or claims that its content is indexed in resources that are not abstracting and indexing services” (criteria area: integrity) and “Publish papers that are not academic at all, e.g. essays by laypeople, polemical editorials, or obvious pseudo-science” (criteria area: other).

All three analysts agreed either YES or NO 87.5 percent of the time. According to table 6, the most disagreement occurred in the criteria area of editor and staff (76%). The highest area of agreement was integrity (98%).

Table 6

Criteria Ranked From Least Referee Agreement to Most

Criteria Area

% of Agreement

editor and staff

76%

business management

86%

integrity

98%

other

90%

Table 7 lists the individual criteria in order of percentage of agreement. The highest percentage of disagreement occurred for the criterion, “the publisher’s owner is identified as the editor of each and every journal published by the organization” (criteria area: editor and staff).

Table 7

Individual Criteria in Order of Percentage of Agreement

Individual Criteria

Criteria Area

Analysts Agree %

The publisher’s owner is identified as the editor of each and every journal published by the organization.

editor and staff

0

Have a “contact us” page that only includes a web form or an e-mail address, and the publisher hides or does not reveal its location.

other

57

Has no policies or practices for digital preservation, meaning that if the journal ceases operations, all of the content disappears from the internet.

business management

60

There is little or no geographical diversity among the editorial board members, especially for journals that claim to be international in scope or coverage.

editor and staff

76

No single individual is identified as any specific journal’s editor;

editor and staff

81

No academic information is provided regarding the editor, editorial staff, and/or review board members (such as institutional affiliation).

editor and staff

Provides insufficient information or hides information about author fees, offering to publish an author’s paper and later sending an unanticipated “surprise” invoice.

business management

84

The journal does not identify a formal editorial / review board.

editor and staff

86

Demonstrates a lack of transparency in publishing operations.

business management

Evidence exists showing that the editor and/or review board members do not possess academic expertise to reasonably qualify them to be publication gatekeepers in the journal’s field.

editor and staff

88

The editorial board engages in gender bias (i.e., exclusion of any female members).

editor and staff

The journals have an insufficient number of board members,(e.g., 2 or 3 members),have concocted editorial boards (made up names), name scholars on their editorial board without their knowledge or permission or have board members who are prominent researchers but exempt them from any contributions to the journal except the use of their names and/or photographs.

editor and staff

87

Does not allow search engines to crawl the published content, preventing the content from being indexed in academic indexes.

business management

92

Two or more journals have duplicate editorial boards (i.e., same editorial board for more than one journal).

editor and staff

94

Provides insufficient information or hides information about author fees, offering to publish an author’s paper and later sending an unanticipated “surprise” invoice.

business management

The publisher asks the corresponding author for suggested reviewers and the publisher subsequently uses the suggested reviewers without sufficiently vetting their qualifications or authenticity. (This protocol also may allow authors to create faux online identities to review their own papers.)

integrity

95

The publisher dedicates insufficient resources to preventing and eliminating author misconduct, to the extent that the journal or journals suffer from repeated cases of plagiarism, self-plagiarism, image manipulation, and the like.

integrity

97

Use boastful language claiming to be a “leading publisher” even though the publisher may only be a startup or a novice organization.

other

Begins operations with a large fleet of journals, often using a common template to quickly create each journal’s homepage.

business management

99

The name of a journal is incongruent with the journal’s mission

integrity

The name of a journal does not adequately reflect its origin (such as a journal with the word “Canadian” or “Swiss” in its name when neither the publisher, editor, nor any purported institutional affiliate relates whatsoever to Canada or Switzerland).

integrity

In its spam e-mail or on its website, the publisher falsely claims one or more of its journals have actual (Thomson-Reuters) impact factors, or advertises impact factors assigned by fake “impact factor” services, or it uses some made-up measure (such as view factor), feigning/claiming an exaggerated international standing.

integrity

Republish papers already published in other venues/outlets without providing appropriate credits

other

Operate in a Western country chiefly for the purpose of functioning as a vanity press for scholars in a developing country (such as using a mail-drop address or P.O. box address in the United States, while actually operating from a developing country).

other

Provide minimal or no copyediting or proofreading of submissions.

other

Discussion

What do the data tell us about Beall’s List as a tool for evaluating both OA and traditional model scholarly journals? First, might evaluators consider these established, well-regarded LIS journals as possible predatory journals? According to the analysts’ application of Beall’s Criteria, yes, even traditional model journals might classify as predatory. Again, the panel was not applying the criteria to identify LIS journals from the JCR collection as predatory (and to avoid any misreading of method, this research report does not directly identify by name individual journals with how they fared against any of the criteria). The primary motivation of the exercise was to determine the validity of Beall’s Criteria and whether these journals, which are arguably considered high quality and high impact, would fail some elements of Beall’s Criteria.

Consequently, the present authors question if Beall’s List suggests a possible bias against OA journals and favors traditional model journals in a way that is not defensible, which previous research has also addressed.32 As a result, the authors find it necessary to consider why Beall’s Criteria aims predominantly at OA publishers when established journals based on traditional models can fail at a high rate. Future research might analyze a list of OA journals generally considered to be of high quality to compare traditional and OA models according to Beall’s Criteria.

Second, although Beall acknowledges the subjectivity of his Criteria, going as far as to state that his lists consist solely of his opinion,33 this study affirms this subjectiveness and raises questions concerning their application to publisher and journal evaluation. Due to the rate of disagreement, the use of a three-person, independently working panel of analysts highlights the subjectiveness of Beall’s Criteria. This calls into question its indiscriminate use by librarians who provide the list to students and faculty as a resource without explaining the limitations of the list and of the Criteria. This is especially important regarding promotion and tenure committees across academic institutions who may use Beall’s Criteria as a way to disqualify articles published in journals that the committee determined fail some Criteria but are not truly predatory. Evaluators should consider a variety of resources when determining the quality of a journal, publisher, or article. Such tools might include JCR, the DOAJ, the Open Access Scholarly Publishers Association, Cabell’s Directory of Publishing Opportunities, and subject specialist knowledge.

The present authors recognize that there are predatory journals in the OA publishing model. However, it would be prudent to caution researchers to understand that, as noted by Berger and Cirasella, there are also low-quality journals found in the traditional publishing model.34 Therefore, the present authors recommend that librarians remain cognizant of these liabilities, but they agree with the conclusion of Smith that librarians should stress to users that OA and traditional journals deserve equal consideration when making decisions concerning quality.35

Beall’s Criteria serves an important purpose as a tool for information literacy and evaluation. However, since Beall ties it directly to his predatory journal and publishers lists, the Criteria becomes counterproductive to the two objectives of information literacy and evaluation. Because evaluators often interpret the lists as determinative of a journal or publisher’s predatory status, this impedes the application of the Criteria because it represents the subjective employment of the Criteria by Beall himself. The indiscriminate use of the Criteria and the lists derived from it effectively obscures this subjective nature and alienates the reader from the process. Additionally, the title of Beall’s benchmarking tool, Criteria for Determining Predatory Open-Access Publishers, is misleading to the reader. The title’s orientation toward only OA journals potentially biases readers against OA publishers and journals. In addition, Beall’s use of “Criteria for Determining” in the title of his evaluation tool further muddies the subjective nature of the Criteria. According to the Oxford English Dictionary, to “determine” something means “To end in (a termination, conclusion, or result)” as well as “To bring to an end a dispute, controversy, or doubtful matter; to conclude, settle, decide, fix.”36 The present authors argue that, while the Criteria may serve as an aid to this end, it is presently not adequate for making such determinations and they question the limited scope of the Criteria. Information literacy and evaluation goes beyond the structure of the journal and its publishing practices to include the assessment of the actual information content. This is extremely important for the work of promotion and tenure committees, as they need to assess the actual information content rather than simply relying on a subjective criterion such as “There is little or no geographical diversity among the editorial board members, especially for journals that claim to be international in scope or coverage.” Beall’s Criteria does not adequately account for the subject knowledge and subject evaluation expertise necessary for determining the quality of the published content; instead, it focuses on traditional publication standards as interpreted through the lens of a librarian (specifically the lens of Beall and all of his possible biases).

Therefore, while the authors of this study are not devaluating Beall’s Criteria, they advocate the development of additional criteria that focus on subject content, particularly if evaluators might use such a list of criteria as a means for making decisions regarding promotion and tenure across university departments.

Conclusion

This study demonstrates the subjective nature of the Criteria by which Beall constructs his lists. Furthermore, it highlights the finding that well-regarded academic journals, whether OA or not, can be considered as possible predatory journals, even when LIS professionals apply the Criteria. The researchers recommend placing emphasis upon training evaluators to apply the Criteria itself, as opposed to relying on a list of journals derived from the Criteria’s application developed under one person’s opinion. The problem with relying on any list is that when an information specialist presents a list to a reader, the reader might view this list uncritically, assuming it to be 100 percent correct. The reader may rely solely on the list, even with the author’s “warnings,” and not consider the information content presented within the journal itself. However, as this study has demonstrated, even librarians disagree about the interpretation and application of the Criteria, and this study’s three analysts did not even consider the quality of the content found within the pages of the journals. After all, scholarly communication is ultimately driven by the information within the journal. The researchers put forth that only experts in a specific field of study can truly identify fraudulent information in their field of expertise, whether or not that journal finds itself on any predatory journal list.

However, if a list of potential predatory publishers and journals is given, an evaluation process needs to be developed that is transparent to the reader so that the reader will know and understand why certain criteria did not pass muster and, therefore, why the journal failed the evaluation. Beall’s lists are deficient in this area, as he provides only names and links to journal publisher websites that have failed. Identifying the reasons a journal failed would shine light on the Criteria and would help the reader to reconsider if the journal is truly predatory, possibly causing them to further consider the information within the journal.

This study shows the subjective nature of Beall’s Criteria, as well as the subjective nature of the lists created from the application of the Criteria. Such criteria provide a starting point for a discussion on predatory aspects of academic publishing. Nevertheless, as librarians, our duty is to refrain from offering up these lists as the final word on predatory journals. Rather, it is our responsibility to (1) use such lists and criteria as tools for teaching faculty to be proactive about evaluating what journals to publish in and (2) to ensure that newer journals, which are often OA, are not disqualified unfairly from consideration as part of quality scholarly output. This latter point is especially important if promotion and tenure committees use Beall’s Criteria without considering the subjective nature of their application, or without also including supplemental evaluative measures of journal quality. The authors of this paper concur with Berger and Cirasella that librarians “are key stakeholders in scholarly and professional conversations reimagining various aspects of scholarly communication,”37 and the present study emphasizes this role in the area of predatory publishing.

Appendix: InCites Journal Citation Reports Information Science & Library Science Category

1

African Journal of Library Archives and Information Science

2

Aslib Journal of Information Management

3

ASLIB Proceedings

4

Australian Academic & Research Libraries

5

Australian Library Journal

6

Canadian Journal of Information and Library Science: La Revue Canadienne ses sciences de l’information et de bibliothéconomie

7

College & Research Libraries

8

Database for Advances in Information Systems

9

Electronic Library

10

Ethics and Information Technology

11

European Journal of Information Systems

12

Government Information Quarterly

13

Health Information and Libraries Journal

14

Informacao & Sociedade-Estudos

15

Informacios Tarsadalom

16

Information & Culture

17

Information & Management (Elsevier)

18

Information and Organization

19

Information Development

20

Information Processing & Management

21

Information Research: An International Electronic Journal

22

Information Society

23

Information Systems Journal

24

Information Systems Research

25

Information Technology & People

26

Information Technology and Libraries

27

Information Technology for Development

28

International Journal of Computer-Supported Collaborative Learning

29

International Journal of Geographical Information Science

30

International Journal of Information Management

31

Investigacion Bibliotecologica

32

Journal of Academic Librarianship

33

Journal of Computer-Mediated Communication

34

Journal of Documentation

35

Journal of Global Information Management

36

Journal of Global Information Technology Management

37

Journal of Health Communication

38

Journal of Information Science

39

Journal of Information Technology

40

Journal of Informetrics

41

Journal of Knowledge Management

42

Journal of Librarianship and Information Science

43

Journal of Management Information Systems

44

Journal of Organizational and End User Computing

45

Journal of Scholarly Publishing

46

Journal of Strategic Information Systems

47

Journal of the American Medical Informatics Association

48

Journal of the Association for Information Science and Technology

49

Journal of the Association for Information Systems

50

Journal of the Medical Library Association

51

Knowledge Management Research & Practice

52

Knowledge Organization

53

Law Library Journal

54

Learned Publishing

55

Library & Information Science Research

56

Library Collections Acquisitions & Technical Services

57

Library Hi Tech

58

Library Journal

59

Library Quarterly

60

Library Resources & Technical Services

61

Library Trends

62

Libri

63

Malaysian Journal of Library & Information Science

64

MIS Quarterly

65

MIS Quarterly Executive

66

Online Information Review

67

portal: Libraries and the Academy

68

Profesional de la Informacion

69

Program-Electronic Library and Information Systems

70

Reference & User Services Quarterly

71

Research Evaluation

72

Restaurator-International Journal for the Preservation of Library and Archival Material

73

Revista Espanola de Documentacion Cientifica

74

Scientometrics

75

Serials Review

76

Social Science Computer Review

77

Social Science Information sur les sciences sociales

78

Telecommunications Policy

79

Telematics and Informatics

80

Transinformacao

81

Zeitschrift fur Bibliothekswesen und Bibliographie

Note: The journals are listed in alphabetical order after six entries were removed from the study due to duplication, ceased publication, or the perception that the entry did not represent a peer-reviewed academic journal.

Notes

1. Jeffrey Beall, “Predatory Publishing: Overzealous Open-Access Advocates Are Creating an Exploitative Environment, Threatening the Credibility of Scholarly Publishing,” The Scientist 26, no. 8 (2012), available online at www.the-scientist.com/?articles.view/articleNo/32426/title/Predatory-Publishing [accessed 1 December 2016]. For a description of the degrees of OA, including “gold,” see Peter Suber’s “Open Access Overview,” available online at http://legacy.earlham.edu/~peters/fos/overview.htm [accessed 1 December 2016].

2. Martin S. Fiebert, “A Look at Open Access Publication and Beall’s List of ‘Predatory’ Journals,” Global Journal of Interdisciplinary Social Sciences 3, no. 4 (2014): 5–6.

3. Jeffrey Beall, “List of Publishers,” previously available online but has since been removed. See https://www.insidehighered.com/news/2017/01/18/librarians-list-predatory-journals-reportedly-removed-due-threats-and-politics [accessed 11 December 2017].

4. Diane Harley, Sophia Krzys Acord, Sarah Earl-Novell, Shannon Lawrence, and C. Judson King, “Assessing the Future Landscape of Scholarly Communication: An Exploration of Faculty Values and Needs in Seven Disciplines” (UC Berkeley: Center for Studies in Higher Education, 2010), available online at https://escholarship.org/uc/item/15x7385g [accessed 1 December 2016].

5. Peter Suber, “Open Access Overview,” available online at http://legacy.earlham.edu/~peters/fos/overview.htm [accessed 24 January 2016].

6. Charlotte Haug, “The Downside of Open-Access Publishing,” New England Journal of Medicine 368, no. 9 (2013): 791–93, doi:10.1056/NEJMp1214750.

7. Sally Morris, “The True Costs of Scholarly Journal Publishing,” Learned Publishing 18, no. 2 (2005): 115–26.

8. Cenyu Shen and Bo-Christer Björk, “‘Predatory’ Open Access: A Longitudinal Study of Article Volumes and Market Characteristics,” BMC Medicine 13, no. 1 (2015): 1, doi:10.1186/s12916-015-0469-2.

9. Alma Swan and Sheridan Brown (Key Perspectives, Ltd.), “Joint Information Systems Committee ( JISC) and the Open Society Institute (OSI) Journal Authors Survey Report” (2004), available online at www.jisc.ac.uk/media/documents/themes/infoenvironment/acf655.pdf [accessed 1 December 2016].

10. Bryna Coonin and Leigh Younce, “Publishing in Open Access Journals in the Social Sciences and Humanities: Who’s doing it and Why?” in Pushing the Edge: Proceedings of the Fourteenth Annual Meeting of the Association of College and Research Libraries, March 12-15, ed. Dawn Mueller (Chicago: American Library Association, 2009), 85-94.

11. John Bohannon, “Who’s Afraid of Peer Review,” Science 342, no. 6154 (2013): 60-65.

12. Phil Davis, “Post Open Access Sting: An Interview with John Bohannon,” The Scholarly Kitchen, available online at http://scholarlykitchen.sspnet.org/2013/11/12/post-open-access-sting-an-interview-with-john-bohannon/ [accessed 25 January 2016].

13. Ibid.

14. “DOAJ News Service: News, Updates & Developments from DOAJ,” available online at https://blog.doaj.org/ [accessed 11 December 2017].

15. Jeffrey Beall, “Predatory Publishers Are Corrupting Open Access,” Nature 489, no. 7415 (2012): 179, doi:10.1038/489179a.

16. Jeffrey Beall, “Scholarly Open Access,” available online at http://scholarlyoa.com [accessed 25 January 2016].

17. Rita Pickler, Jane Noyes, Lin Perry, Brenda Roe, Roger Watson, and Mark Hayter, “Authors and Readers Beware the Dark Side of Open Access,” Journal of Advanced Nursing (2014), doi:10.1111/jan.12589.

18. Jeffrey Beall, “Beall’s List of Predatory Publishers 2015,” previously available online but has since been removed. See https://www.insidehighered.com/news/2017/01/18/librarians-list-predatory-journals-reportedly-removed-due-threats-and-politics [accessed 11 December 2017].

19. Ibid.

20. Committee on Publication Ethics, “Principles of Transparency and Best Practice in Scholarly Publishing,” available online at http://publicationethics.org/files/Principles%20of%20Transparency%20and%20Best%20Practice%20in%20Scholarly%20Publishing.pdf [accessed 22 January 2016] and “Code of Conduct,” Code of Conduct, available online at http://publicationethics.org/files/Code%20of%20conduct%20for%20publishers%20FINAL_1_0_0.pdf [accessed 22 January 2016].

21. Committee on Publication Ethics, “Principles of Transparency and Best Practice in Scholarly Publishing.”

22. Committee on Publication Ethics, “About COPE,” available online at http://publicationethics.org/about [accessed 22 January 2016].

23. Ibid.

24. Shen and Björk, “‘Predatory’ Open Access,” 1.

25. Ibid, 28–31.

26. Walt Crawford, “Ethics and Access 1: The Sad Case of Jeffrey Beall,” Cites & Insights 14, no. 4 (2014): 1–14; Monica Berger and Jill Cirasella, “Beyond Beall’s List: Better Understanding Predatory Publishers,” College & Research Libraries News 76, no. 3 (2015): 132–35.

27. Nerissa Nelson and Jennifer Huffman, “Predatory Journals in Library Databases: How Much Should We Worry?” Serials Librarian 69, no. 2 (2015): 169–92, doi:10.1080/0361526X.2015.1080782.

28. James Testa, “The Thomson Reuters Journal Selection Process,” available online at http://wokinfo.com/essays/journal-selection-process/ [accessed 10 February 2016].

29. Jeffrey Beall, Criteria for Determining Predatory Open-Access Publishers, 3rd ed. (2015), available online at https://scholarlyoa.files.wordpress.com/2012/11/criteria-2012-2.pdf [accessed 1 December 2016].

30. Roger Saljo, “Learning Approach and Outcome: Some Empirical Observations,” Instructional Science 10 (1981): 47–65, doi:10.1007/BF00124566.

31. Jacqueline Ann Douglas, Alexander Douglas, Robert James McClelland, and John Davies, “Understanding Student Satisfaction and Dissatisfaction: An Interpretive Study in the UK Higher Education Context,” Studies in Higher Education 40, no. 2 (2015): 329–49, doi:10.1080/03075079.2013.842217.

32. Crawford, “Ethics and Access 1: The Sad Case of Jeffrey Beall,” 1–14.

33. Kristen Wilson, “Librarian vs. (Open Access) Predator: An Interview with Jeffrey Beall,” Serials Review 39, no. 2 (2013): 125-28); Jeffrey Beall, “Disclaimer,” available online at https://jeffreybeall.com/disclaimer/ [accessed 11 December 2017].

34. Monica Berger and Jill Cirasella, “Beyond Beall’s List: Better Understanding Predatory Publishers,” College & Research Libraries News 76, no. 3 (2015): 132–35.

35. Beverly Dowdy, David Crotty, Beth Bernhardt, Paige Hall Smith, and Jan Mayo, “Challenges and Opportunities of Open Access: A Panel Discussion,” Serials Review 40 (2014): 188–90.

36. Oxford English Dictionary Online, s.v. “determine,”available online at www.oed.com/view/Entry/51244?redirectedFrom=determine [accessed 7 April 2016].

37. Berger and Cirasella, “Beyond Beall’s List,” 135.

*Joseph D. Olivarez is Serials Cataloging/Metadata Librarian, Stephen Bales is Humanities and Social Sciences Librarian, Laura Sare is Government Information Librarian, and Wyoma vanDuinkerken is Director of the Joint Library Facility, all at Texas A&M University Libraries; e-mail: jolivare@library.tamu.edu, sbales@library.tamu.edu, lsare@library.tamu.edu, wvanduin@library.tamu.edu. ©2018 Joseph D. Olivarez, Stephen Bales, Laura Sare, and Wyoma vanDuinkerken, Attribution-NonCommercial (http://creativecommons.org/licenses/by-nc/4.0/) CC BY-NC.

Copyright Joseph D. Olivarez, Stephen Bales, Laura Sare, Wyoma vanDuinkerken


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Article Views (Last 12 Months)

No data available

Contact ACRL for article usage statistics from 2010-April 2017.

Article Views (By Year/Month)

2018
January: 1628
February: 396
March: 449
April: 215
May: 137
June: 118
July: 102
August: 57
September: 61
October: 100
November: 36
2017
April: 0
May: 29
June: 6
July: 10
August: 3
September: 7
October: 127
November: 31
December: 14