Literature Reviews

Don't Rely on Bad Research!

Guarding Against Retracted Articles

Pay attention to retraction notices when collecting and reading articles. The use and continued citation of bad research can have a long-term effect on the fundamental credibility of the scholarship.

Retractions in Scholarly Publishing

An article is retracted by a journal when the journal no longer considers it credible. Articles are retracted for a variety of reasons, including:

• Special issue scam articles - special issues of known journals edited by guest editors, which can be much more lax in their editorial standards than the journal's typical practice. (Else, 2021)
  • Note: Special issues have a long history in scholarly publishing as a way to publish research on new and specific topics. The problem arises with the more porous nature of special-issue publishing, which has been taken advantage of by scammers who pose as guest editors and accept bad papers, as well as those who pose as authors and submit bad papers.
• Paper mill articles - articles claimed as writing credits by the listed author but not actually written or researched by them
• Citation fraud - claiming paper citations that do not exist or do not cite the paper in question
• Plagiarism - using the work of others without attribution
• Fake peer review - a lack or substandard peer review claimed as legitimate
• Honest error discovered by the author

Articles in need of retraction are often identified by volunteer researchers who recognize the signs of bad research. Open peer review commenters can also call attention to the problem.

References

Retraction Identification Tools

Zotero: Retraction Catcher

Zotero Desktop includes a folder that collects retraction notices for articles added to your library if it detects any. Review this folder periodically to make sure you're not using bad research.

Scopus and Web of Science

The citation indexes Scopus and Web of Science include a means to filter for or filter out retracted articles.

Retraction Watch Database

The research news blog Retraction Watch has compiled a database of retracted articles, available on its own and also incorporated into CrossRef data.

Search for an article or author to determine retraction status.

Identifying Bad Research Papers

Signifiers of Questionable Authorship

As you find and read scholarly articles and conference papers, you may notice the following signs of papers written by paper mills or AI:

• "Tortured phrases": Overwrought ways of phrasing common terms. likely as a result of the use of automated programs that rewrite text to avoid charges of plagiarism. Coined by Guillaume Cabanac, a computer scientist, and coauthors (Else, 2021).
  • Examples of these phrases:
    • Instead of artificial intelligence: Counterfeit consciousness
    • Instead of deep neural network: Profound neural organization
• In addition to the tortured phrases, Van Noorden (2023, p. 467) also lists the following based on research conducted by Adam Day, owner of Clear Skies, a company that makes a machine-learning tool called Papermill Alarm that identifies questionable papers:
  • "Text that follows a common template;
  • Suspicious e-mail addresses that don’t correspond to any of a paper’s authors;
    • E-mail addresses from hospitals in China (because the issue is known to be so prevalent there);
  • Identical charts that claim to represent different experiments;
  • Citations of other paper-mill studies; and
  • Duplicate submissions across journals."

References