| Fake publications in science are becoming an increasingly large challenge. While estimates start at "only" 2%, more recent estimates suggest a rate of 16% or higher. However, automatic detection methods are not necessarily reliable or can only detect certain features as fake. Due to this, it is relevant to educate users on potential problems in the information they find online. This includes the search for publications. In this paper, we analyze the validity of using LLMs as a first-level auditor to provide insights for users to determine whether a publication could be questionable. We developed a frontend to visualize the results of this audit. We found that even with just the title and abstract, it is possible to assign sensible risk scores and sentence-based explanations using only LLMs. We conclude that LLMs can be used as an auditor especially in combination with the output of other classifiers. |
*** Title, author list and abstract as submitted during Camera-Ready version delivery. Small changes that may have occurred during processing by Springer may not appear in this window.