September 22, 2013

A few more shoes left to drop

Over the last year, the Licthenthaler retractions scandal (and its ramifications for our field) has tended to come up as a conference mealtime discussion topic — at least when I'm at a conference with European innovation scholars. Last week’s open innovation workshop in Bath was no exception. However, unlike at most conferences, the topic also boiled out into the open during a plenary discussion.

In my opening keynote, I had mentioned the opportunity for more outbound open innovation research, given that half of the Licthenthaler retractions were on this topic. This was news to some people. Although the scandal is well known among German business academics and open innovation scholars, it turns out there were a few attendees who hadn’t heard about the 13 retracted articles by Ulrich Licthenthaler and his former habillitation supervisor Holger Ernst, nor the three articles that were accepted but withdrawn prior to online publication.

At the closing session, a few doctoral students and faculty asked about the new rules in the post-Licthenthaler world. Here let me offer an assessment of what it means and also some thoughts on what’s next (or what’s left).

Research Policy and its Standards

As it turns out, last month students at one doctoral consortium at the Academy in Orlando read June’s editorial by Research Policy editor Ben Martin. The abstract summarizes the problem facing the journal and innovation studies more broadly:
This extended editorial asks whether peer-review is continuing to operate effectively in policing research misconduct in the academic world. It explores the mounting problems encountered by editors of journals such as Research Policy (RP) in dealing with research misconduct. Misconduct can take a variety of forms. Among the most serious are plagiarism and data fabrication or falsification, although fortunately these still seem to be relatively rare. More common are problems involving redundant publication and self-plagiarism, where the boundary between acceptable behavior (attempting to exploit the results of one’s research as fully and widely as possible) and unacceptable behavior (in particular, misleading the reader as to the originality of one’s publications) is rather indistinct and open to interpretation. With the aid of a number of case-studies, this editorial tries to set out clearly where RP Editors regard that boundary as lying.
On the first page, Martin provides the broader context:
[W]e know the pressures of academic competition are rising, whether for tenure, research funds, promotion or status, which may mean that more researchers are tempted to cut corners… The use of performance indicators based on publications, citations, impact factors and the like may also be adding to the temptation to stray from previous conventions regarding what constitutes appropriate research behavior or to attempt to surreptitiously ‘stretch’ the boundary between appropriate and inappropriate behavior. …

There are worrying signs that research misconduct is on the increase. The number of retractions of published papers by journals has increased more than 10-fold in a single decade – from around 30 a year in the early 2000s to some 400 in 2011. … Moreover, the majority of retractions are seemingly the consequence of research misconduct rather than simple error.

With regard to the particular problem of self-plagiarism and related activities described below, the number of academic articles referring to ‘self-plagiarism’, ‘salami publishing’, ‘redundant publication’ or ‘duplicate publication’ has risen nearly five-fold from 170 in 2000 to 820 in 2012. More and more editorials are appearing in which journal editors complain about the growing burden being imposed on them as they attempt to detect, evaluate and sanction research misconduct in its various forms.
Martin noted that the journal only faced an “occasional” problem of research misconduct, until 2007, when it stumbled across the scandal of a dozen or more plagiarized articles published by Hans Gottinger†. (The scandal was jointly investigated by Research Policy and Nature).

After listing various retracted and withdrawn articles, the sixth page of Martin’s editorial refers to the Licthenthaler case (emphasis mine):
More recently, an even more complicated case was brought to the attention of RP Editors by two individuals who independently had been asked to review papers by the same author (a professor at a European university) submitted to two other journals. They discovered that the author concerned had published an astonishing total of over 60 journal articles since 2004. Since this number was too great to handle, the two reviewers concentrated their attention on 15 articles published in leading journals over the period 2007–2010 (including three published in Research Policy), all of which formed part of a single stream of research emerging from a survey of over 100 firms in Europe that the author had conducted. They found that in these papers, similar analyses had been carried out with differing combinations from a large set of variables (sometimes relabeled, to add to the confusion) with no references to indicate the prior papers the author had already produced on the same broad theme. Moreover, in some cases, a given variable was treated as a dependent variable and in others as an independent variable. Perhaps more worryingly, variables that were demonstrated to be significant in some papers were then (presumably deliberately) omitted in the analysis reported in other papers. The author was asked for an explanation. This explanation was deemed unsatisfactory by the RP Editors, with the result that two of the RP papers[31] had to be formally retracted.

[Footnote 31: At a late stage in the investigation, it also became apparent that in one of these RP papers the degree of statistical significance of several of the claimed findings had been misreported or exaggerated. Whether this was simply the result of ‘accidental’ mistakes, as the author claimed, is unclear. However, the fact that similar problems have since been confirmed in several other papers by this author makes this less plausible as an explanation.]
The workshop participants asked what the new rules are for multiple publication from the same data. How does one avoid self-plagiarism and salami slicing? I think Martin and RP have spelled out more clearly than anyone else what these rules are. Drawing from his editorial, his public comments and from my current experience as a RP guest editor, let me paraphrase it into two guidelines.

First, would a reasonable reviewer (or reader) conclude that this article deserves publication if all previous or parallel articles were visible at the same time? Second, does it appear that the authors have withheld from the editor (if not the blinded manuscript) full disclosure of all related work? As Martin concluded:
Failure to provide all pertinent information in the full version implies a premeditated attempt by the author(s) to deceive the journal as to the level of originality of the paper. As such, it represents grounds for the summary rejection of a paper.
† Martin’s editorial doesn’t mention the names of any transgressors. When I asked him about it, he said it was because some of the journal’s actions were public (i.e. retracted articles) and some were not (rejected articles); since he couldn’t list all the names, he decided to list none of them.

Further Sanctions

The Licthenthaler story seems to be winding down. With his habillitation and Lehrbef√§higung withdrawn by WHU earlier this month, what’s left is the results of the investigation by Mannheim, his current employer. The university issued a brief press release in July; as with the WHU announcement, it was issued only in German so here is my composite (computer-aided) translation:
Allegations of scientific misconduct against Professor Dr. Ulrich Lichtenthaler:
University examines the way forward
Press release of 31 July 2013

After the Permanent Commission for the investigation of allegations of scientific misconduct at the University of Mannheim has sent its final report to the allegations of scientific misconduct against Prof. Dr. Ulrich Lichtenthaler, the university now is considering the commission report.

"For legal reasons, in particular allowing for the right to fairness to Prof. Dr. Ulrich Lichtenthaler, I cannot yet make any statement about the contents of the 170 page report or any possible consequences" said the Rector of the University Mannheim, Prof. Dr. Ernst-Ludwig von Thadden. The report was given to Prof. Dr. Lichtenthaler. "The public has a legitimate interest in full disclosure of the allegations and will be informed of further steps by to the extent legally possible," said the rector of the university.

After the Rector of the University of Mannheim received allegations of scientific misconduct against Dr. Lichtenthaler in the summer of 2012, a responsible commission of inquiry of the university was immediately called. Since 24 July 2012, the Commission has made a considerable commitment of its members to deal with the allegations against Prof. Lichtenthaler. Among other things the Commission has commissioned external reports, interviewed respondents and performed its own extensive evaluations. By letter of 21 July 2013, the Commission sent its final report to the Rector. This completes the work of the Commission in accordance with no. 4.3 of the guidelines of the University of Mannheim for safeguarding good scientific practice. The Rector will check any further action.
Further Retractions

That’s not to say that the retractions are over. They are still trickling in; Martin notes the varying degrees of concern (if not integrity) by the editors of the affected journals:
In the case of the extensive self-plagiarism by the German author, other journals were slow to react when alerted to the problem, and in at least one case, the eventual retraction of an article by this author was justified rather vaguely in terms of ‘data problems’ rather than giving details of the specific form of misconduct involved.
According to someone who’s read the various Licthenthaler articles, five articles have problems comparable to those of retracted articles, but are at journals that have not yet announced any decision regarding Licthenthaler’s publications
  • Lichtenthaler, Ulrich (2009). “Outbound open innovation and its effect on firm performance: examining environmental influences,” R & D Management, 39 (4): 317-330. DOI: 10.1111/j.1467-9310.2009.00561.x
  • Lichtenthaler, Ulrich (2010). “Organizing for external technology exploitation in diversified firms,” Journal of Business Research, 63 (11): 1245-1253. DOI: 10.1016/j.jbusres.2009.11.005
  • Lichtenthaler, Ulrich & Holger Ernst (2008). “Intermediary services in the markets for technology: Organizational antecedents and performance consequences,” Organization Studies, 29 (7): 1003-1035. DOI: 10.1177/0170840608090531
  • Lichtenthaler, Ulrich & Miriam Muethel (2012). “The role of deliberate and experiential learning in developing capabilities: Insights from technology licensing,” Journal of Engineering and Technology Management, 29 (2): 187-209. DOI: 10.1016/j.jengtecman.2011.10.001
  • Lichtenthaler, Ulrich & Miriam Muethel (2012). “The Impact of Family Involvement on Dynamic Innovation Capabilities: Evidence From German Manufacturing Firms,” Entrepreneurship Theory and Practice, 36 (6): 1235-1253. DOI: 10.1111/j.1540-6520.2012.00548.x
A sixth article at a top journal has been investigated, but the results (and any corrective action) have yet to be announced.

What happens after such retractions? It appears that the scientific field partially (but not entirely) self-corrects on retracted articles. As Jeff Furman, Kyle Jensen and Fiona Murray reported in their 2012 Research Policy study of retracted medical research:
Our core results [imply] that annual citations of an article drop by 65% following retraction, controlling for article age, calendar year, and a fixed article citation effect. … The effect of retraction does appear to be stronger in the most recent decade than in prior decades, although the large, statistically significant impact of retractions on future citations does not appear to be only induced by modern IT. The results … suggest that papers retracted between 1972 and 1999 experienced a 63% decline in citations after retraction, while those retracted since 2000 experienced a 69% decline in citations.
One would hope that in the future, after a retraction any subsequent citations would rapidly decline to zero. Fortunately, online publication (unlike dusty library print collections) allows prominent marking of the retraction status for a previously published article.

References

Jeffrey L. Furman, Kyle Jensen, Fiona Murray, “Governing knowledge in the scientific community: Exploring the role of retractions in biomedicine,” Research Policy 41, 2 (March 2012): 276–290. doi: 10.1016/j.respol.2011.11.001

Ben Martin, “Whither research integrity? Plagiarism, self-plagiarism and coercive citation in an age of research assessment,” Research Policy 42, 6 (June 2013): 1005-1014. doi: 10.1016/j.respol.2013.03.011

No comments: