A Tale of Two Investigations
Taking A Closer Look at a Pair of HHS ORI Misconduct Reports
My last article detailed efforts to track down more information about the agonizingly slow investigation into possible research misconduct by Dr. Hoau-Yan Wang at the City University of New York. It primarily focused on public records requests from a variety of state and federal institutions (SEC, HHS, CUNY, CCNY). Because the investigation is still active, it’s exceedingly unlikely that any records which directly pertain to it will be released, so I decided to focus my energy into pursuing public records from two closely related items in the meantime. The first item, as discussed in the prior article, involves records from concluded journal investigations for which Dr. Wang provided some representation of raw data; understanding how each journal arrived at its decision is critical for our evaluation of whether or not they actually performed a thorough and thoughtful investigation. The second item involves documentation from similar investigations conducted through HHS ORI; understanding how federal authorities arrive at a determination of research misconduct is critical for our evaluation of how seriously to take the allegations. This article tackles the second of these two items via a records release by HHS.
Last year, Retraction Watch reported on the recently announced administrative action against cancer researcher Toni M. Brand, who HHS ORI determined engaged in research misconduct after a pair of investigations by the two institutions where she worked during the time that the research was produced and published—UW-Madison and UCSF. Retraction Watch also reported about an “egregious delay” by the editors at Science Signaling, who took three years to retract a paper by Dr. Brand which had been flagged extensively on PubPeer, even though the investigation at UW-Madison had deemed that no misconduct had occurred and that “these issues likely did not affect the overall conclusions of the study.” And yet HSS ORI seems to disagree, having specifically referenced this paper among those with “recklessly falsified or fabricated data” in its final report (a good reminder that “affecting the conclusions of a study” is not the barometer for misconduct).
There appeared to be both a litany of unanswered questions about how and why the UW-Madison investigation declined to assert that a handful of serious and unaddressed issues constituted misconduct, as well as an opportunity to glean information about how HHS ORI and/or institutions analyze some of the very same concerns that have been raised in Dr. Wang’s publications.
I submitted a FOIA request to HHS, and received the partially redacted Investigation Reports to HHS ORI from UCSF and UW-Madison. The entirety of the 122-page document is at the link for your perusal—there is quite a lot in there. What follows is an overview of the most salient information that I was able to parse out of the pair of reports.
Investigation I: UW-Madison (DIO 6609)
On June 18, 2018 the Research Integrity Officer assigned to the investigation at UW-Madison submitted a final report to HHS ORI which stated the following:
After reviewing the documentation and analysis provided by the Investigation Committee, the Responsible Institutional Official, Sarah Mangelsdorf, Provost and Vice Chancellor for Academic Affairs, concurs with their conclusion that research misconduct has not been committed.
The gist of the report is encapsulated quite well in the summary, which states that the RIO and committee found “evidence of data duplication or misrepresentation, which the investigation committee found to result from the carelessness of Dr Brand during her preparation of 7 manuscripts.” In other words, just a series of careless, but honest, errors. However, a summary dated May 18, 2018 sent by a redacted group (likely the investigation committee) to Senior Associate Dean Richard L. Moss and copied to Brian Fox, the RIO, is far more circumspect. Quite a bit of this report is redacted, but we are given a good overview of the findings, including that “the committee concluded that data duplication or data misrepresentation occurred in all of these [seven] manuscripts” identified by complainants, despite the fact that “the Investigation Committee has determined that research misconduct was not committed.”
If I’m reading the summary correctly, it seems that the committee drafted an inquiry report and sent it to [redacted] and the respondent for comment, but [redacted] identified more instances of misconduct, so the committee was asked to broaden the scope. Then the committee identified yet more concerns beyond even the original set of manuscripts. An inquiry report was finally submitted which “identified an extensive pattern of irregularities, which led the committee to determine that there was a possibility that Dr. Brand engaged in reckless behavior that constitutes intentional falsification of data.”
In the investigation, the committee found that the respondent “consistently used practices that were inappropriate for generating figure panels for publication, including repetitive copying and pasting techniques” and “sometimes adjusted the contrast of individual images in figure panels to enhance their appearance (although the data itself were not obviously altered with an intent to deceive).” The committee also noted that “In some instances […] we found clear differences between the original data and the published data, but we found that the scientific conclusions made in the manuscripts were not affected” and also that “in a minority of cases, original data could not be found, which the committee found to be unacceptable.” All emphases mine.
The chasmic difference between the substance of the report and its headline is, for lack of a better word, bonkers. The rationale for the committee’s and RIO’s decision not to believe their own eyes appears to depend on the age-old platitude that the malfeasance did not “alter the scientific interpretation of the manuscript.” Personally, I believe the entire apparatus at UW-Madison deserves ample scrutiny for punting on what appears to be a pretty clear case of misconduct. The failure to get this right—at least according to the HHS ORI administrative action—allowed this researcher to continue on and fabricate additional research at a higher profile institution. This is unacceptable. It’s also incredibly disheartening to know that if it weren’t for the much more thorough and less wide-eyed investigation by UCSF, these records would likely be locked away out of reach from FOIA—investigations that end in exoneration are generally considered sealed records for privacy reasons. This, too, is unacceptable. I imagine that this is why the journal editor, when asked to comment, told the UW-Madison RIO that “I think we have the right to refuse disseminating any information.”
Investigation II: UCSF (DIO 6776)
In the next position held by Dr. Brand, as a postdoc at UCSF, a second investigation was commenced which led to a report compiled on January 15, 2021, about two and a half years after the final report was submitted by UW-Madison. The headline of the report this time is that committee found that the respondent “recklessly committed research misconduct.” This report, at over 50 pages, is far more comprehensive and incisive than the UW-Madison report. The report focuses on just four figures with Western blot duplications, but the UCSF committee investigated each of them thoroughly.
For example, the committee asked for the raw data for the duplicated panels labeled 93-VU-147T and Cal27 shown in false color below. The raw data then appears to show the TU138 blot labeled on the gel as Cal27, raising even more questions. There is a devastating quote accompanying this figure in which the respondent replied “Oh, weird. Okay. Yeah. I'm not sure about what — why that is. I see what you're saying now, and I need to go back and figure out why that was used there.“
The raw data provided by the respondent is included in the report:
There is quite a bit of detail like this throughout the report—far too much for me to delve into here. One particular case will be quite familiar for those following the Cassava controversy, though, in which the respondent reused a flipped version of a Western blot. The committee found that she “falsified and/or fabricated data appearing in Supplemental Figure 3A of the author's 2018 Cancer Research paper by using the same image, vertically flipped, to represent different experimental results in the third and fourth panels of the third cell line (SCC90).” This despite some amount of incredulity from the committee given that the results could stand just fine with these panels omitted!
The amount of detail in the UCSF committee report and the meticulousness of their process is laudable. The report includes an abundance of commentary about the committee’s diligence, for example when they express skepticism at the claim that “PDX lysates are hard to come by” by noting that “what's not plausible about [Dr. Brand's] explanation is that PDXes are a renewable resource. You don't run out of a PDX. You might run out of that lysate but then you simply go back and you take it from the freezer and you grow it again and you make more.”
Mainly, though, the committee simply asked for the raw data and then examined it carefully. The summary of the findings starts off stating that “the committee is not convinced by Dr. Brand's claim that the errors contained in the allegations arose due to faulty copying and pasting methods. As discussed in the Analysis and Findings section above, in many instances Dr. Brand was unable to produce the primary and replicate data that she meant to use for the incorrectly duplicated images” and that she “provided additional incorrect images and data that revealed more potential falsifications within the figures.” The UCSF committee took exception to some of the exact items that UW-Madison let slide, like a blot that “had been resized or cropped and brightness and contrast adjusted.” All emphases mine.
Lastly, I’ll point out that the UCSF committee properly acknowledged the pattern of irregularities that UW-Madison simply ignored. The UCSF report noted that “while volume of data may have contributed to the issue, for the reasons already discussed, the errors are unlikely to all have been accidental” and go on to mention that “at least four other published papers on which Dr. Brand is the first author have had to be corrected due to image duplication, which was indicated in the publicly available correction notices. In those instances, Dr. Brand was also unable to provide original data for some of the questioned figures.”
HHS ORI Reports: What We Learned
Something that this report makes clear is that forensic methods for detecting things like duplication or splicing in Western blots—much like what we saw throughout the Citizen Petition filed against Cassava or the analyses of Dr. Bik—should be taken very seriously as signals of concern that require further investigation. Beyond that, as the UCSF report clearly demonstrates, the actual raw data is required in order to do a proper analysis. Accepting data that is not a high resolution scan of the original gel, much like what we’ve seen The Journal of Neuroscience or IBRO journal Neuroscience do, is a dereliction of duty. And further, if the “raw data” that is transmitted itself contains more concerns, that’s quite a strong signal that something is amiss. These things are complex, sure, but it seems that what presents the biggest challenge for institutions and journals performing investigations is not the technical analysis, but rather the challenge of simply believing the overwhelming amount of evidence showing that a colleague may have acted inappropriately.