There are lots of ways to try and ruin it for everyone. Covid restrictions that make no sense are a proven successful strategy for this, demanding the Doing of More. Monkeypox offers exciting additional ruining potential. The surprise winner, however, is an old Alzheimer’s paper that it seems the entire field of drug development
Re: the Statnews piece about stopping holding oversight board meetings --
One of the things that is extremely odd, but true, about living in DC is that you realize there are shockingly few "close the loop" mechanisms. A surprisingly large number of Congressionally-mandated reports just aren't done because the person responsible retired, and if they're done, no one reads them (see, e.g., https://www.washingtonpost.com/sf/national/2014/05/03/unrequired-reading/). And there's no central reporting, as far as I'm aware, of board meetings occurring.
Perhaps this is because these reports and board meetings aren't actually (unfortunately) about the content, but just about the gesture, so that's on net good. But then we're just saying that we're incentivizing the production of empty gestures.
Alternate explanation for the fact that knowing more about Covid correlates with being more worried about Covid: the modal person who knows something about Covid knows it from watching/reading the MSM (rather than following data directly), which has been (sometimes deliberately and maliciously) promoting maximalist Covid fearmongering for years now. If Covid is constantly framed by your major news source as an "end of the world" level pandemic, you're more likely to believe that.
As for the Alzheimer's fraud: my trust in science was already pretty low in January 2020 due to the replication crisis, and over the course of the past two and half years it has reached ~zero for anything from the past 30 years or so, unless it has been turned into engineering. The rot is too deep.
The Alzheimer's fraud seems very bad, but it's impact/importance in the field/research seems to have been dramatically overblown by the media, see here:
One interesting tidbit of the alzheimers story which you did not touch on here. Schrag's discovery was a direct consquence of suspicious truth-seeking short sellers investigating Simufilam. See the quote from the Science article below:
While Schrag is rightly being praised for his role in the discovery, the real heros here are the short sellers here who led the process of uncovering this fraud (and who seemingly used Schrag as an independent expert to verify their own findings/suspicions). You can find their 42 page research report submitted to the FDA in Aug 2021 (under the cover of an attorney) here: https://www.regulations.gov/document/FDA-2021-P-0930-0004
It's worth mentioning that for items like this the FDA has a public commenting process whereby interested parties can submit their thoughts on such matters for consideration. From a cursory review, the comments are unsuprisingly filled with fda lawyers and medical professionals ridiculing the fraud of the short sellers and their motivated research. Go figure. (posted a couple below for fun, don't mean to be too hard on the specific commenters, since their assumptions here may be usually correct, but I also have to imagine that some of them were themselves making use of motivated reasoning...
Last thing I'll note is that in addition to the obvious importance of this discovery, the RAT/EA community should be thrilled that this discovery is in effect maybe the most important real world discovery from 'prediction markets'. For those unfamiliar with the financial markets for early stage biotechs, they are really the closest thing to scaled prediction markets out there. For single-drug companies that are in clinical trials, investors are literally making a asymmetric probabilistic bets on the outcome of trial success, and so as one might suspect the best investors in the space tend to be teams of PhDs conducting their own analysis to derive a P (success) and then betting if their expectations are different than the market. Happy to elaborate further on specifics here when I have time if people are interested.
So who are the short sellers that deserve our praise? As noted in the article, the two short sellers are neuroscientists who submitted the complaint anonymously through an attorney.
Using regulatory databases and some google-fu, it seems rather likely that the initial short seller (or one of them) was Granite Point Capital.
If we using an SEC filing aggregator we can see that three funds had large net put short positions in Cassava in 2H 2021, when the complaint was first filed. These are LMR Partners, Perceptive (one of the very best biotech focused funds), and Granite Point.
Granite Point is smaller, with only ~$500M in AUM, is seemingly biotech focused and active in aggresive short selling (just look up their litigation). A look on linkedin at their partner group shows multiple PhDs/MDs.
Most notably, I stumbled upon this 2019 class action where Labaton, the niche law firm firm filing the cassava, is representing Granite Point, which seems to make it pretty likely that Granite were again working with Labaton in their 2021 complaint against Cassava. Its very possible that they were part of a syndicate or that there was an independent short seller pitching this to multiple funds, but the labaton tie seems to make it very likely they were involved.
"In August 2021, Matthew Schrag, a neuroscientist and physician at Vanderbilt University, got a call that would plunge him into a maelstrom of possible scientific misconduct. A colleague wanted to connect him with an attorney investigating an experimental drug for Alzheimer’s disease called Simufilam. The drug’s developer, Cassava Sciences, claimed it improved cognition, partly by repairing a protein that can block sticky brain deposits of the protein amyloid beta (Aβ), a hallmark of Alzheimer’s. The attorney’s clients—two prominent neuroscientists who are also short sellers who profit if the company’s stock falls—believed some research related to Simufilam may have been “fraudulent,” according to a petition later filed on their behalf with the U.S. Food and Drug Administration (FDA)."
Sample Comments from FDA Complaint
"This is a self serving, deceptive and destructive petition that is not filed on behalf of the human beings this drug is intended to benefit, it is filed on the behalf of one or few short sellers who have gained financially at the expense of investors and future patients.
The petition had made reference to a “whistleblower” which is typically used to refer to someone inside the company. This “whistleblower” was later revealed to be someone outside the company that held a short position in the stock. The scientific queries within this do not seek the truth, they are poorly researched and a scapegoat for this unethical practice of stock manipulation."
"FDA please act expeditiously and firmly deny this abusive and collusive CP against Cassava Sciences by Labaton Sucharow. Already the Journal of Neurosciences have confirmed Cassava’s foundational research paper and found no evidence of manipulation and only 1 human error that doesn’t impact the scientific conclusion. Also, Yale researchers have recently confirmed PTI-125’s moa and affinity binding in mice studies for hard to treat seizures. No patients have been harmed by PTI-125 and you are already aware of the breakthrough improvements AD patients are experiencing. You must not allow any further delay of your decision bc there is an obvious short and distort campaign brought on by bad actors in collusion against Cassava. I believe the SEC is in the process of looking into the matter as many investor complaints (including mine) have been brought to the office of Enforcement. Cassava’s phase 3 SPA trials must not be further impacted."
Turns out a couple of short sellers beat the Journal of Neurosciences. Shows where the real truth-seeking analysis is being done.
The whole field of amyloid research in the so-called sporadic Alzheimer's disease was an idiotic boondoggle right from the start. The fraud is just the icing on the cake.
"Big booster energy. The official stats say ~50% of the vaccinated are boosted (and 10% of them were originally J&J which muddies things) and this has it at 59%, reflecting the bias of people who respond to polls I suppose. "
This discrepancy is mostly due to the official stats being bad, not survey bias. The CDC also believes that 90% of adults have had at least one dose of the vaccine, which is very hard to believe, compared to the poll's answer of 30%, which is plausible. The CDC can't reliably identify which shots are second/third/fourth doses, so they report lots of extra first doses, making the data more or less worthless.
You should be following Elizabeth Bik for research fraud stuff if you aren't already. She's very good at catching this stuff and these fraud-detection organizations already exist.
Though a worthwhile question is "why is peer review failing at this?"
Possibly because when you read a bunch of papers critically, finding no issues, Bayes perniciously dulls your senses. Maybe the panel for obvious frauds should require forced retirement after a year of service.
Or maybe just have a red team that assumes fraud, and speculates on most likely methods for data manipulation in every piece.
Papers commonly include a hand-wavy errors and limitations section. A para on areas of possible data manipulation, or ways the author voluntarily tied their hands to prevent that would be nice. (Did disinterested parties consult on any sudden outlier exclusions? Did you lock your data after collection or register it with a third party? Did you overcollect in a way that would allow shopping for p-values?)
Some risk this would become boilerplate too, but who knows, that might quickly force preregistration and data escrow as a norm.
Re: the Statnews piece about stopping holding oversight board meetings --
One of the things that is extremely odd, but true, about living in DC is that you realize there are shockingly few "close the loop" mechanisms. A surprisingly large number of Congressionally-mandated reports just aren't done because the person responsible retired, and if they're done, no one reads them (see, e.g., https://www.washingtonpost.com/sf/national/2014/05/03/unrequired-reading/). And there's no central reporting, as far as I'm aware, of board meetings occurring.
Perhaps this is because these reports and board meetings aren't actually (unfortunately) about the content, but just about the gesture, so that's on net good. But then we're just saying that we're incentivizing the production of empty gestures.
Alternate explanation for the fact that knowing more about Covid correlates with being more worried about Covid: the modal person who knows something about Covid knows it from watching/reading the MSM (rather than following data directly), which has been (sometimes deliberately and maliciously) promoting maximalist Covid fearmongering for years now. If Covid is constantly framed by your major news source as an "end of the world" level pandemic, you're more likely to believe that.
As for the Alzheimer's fraud: my trust in science was already pretty low in January 2020 due to the replication crisis, and over the course of the past two and half years it has reached ~zero for anything from the past 30 years or so, unless it has been turned into engineering. The rot is too deep.
The Alzheimer's fraud seems very bad, but it's impact/importance in the field/research seems to have been dramatically overblown by the media, see here:
https://twitter.com/samuel_marsh/status/1550883407441416193?t=UpvL61XeS30RAq2DC6Kxvg&s=19
One interesting tidbit of the alzheimers story which you did not touch on here. Schrag's discovery was a direct consquence of suspicious truth-seeking short sellers investigating Simufilam. See the quote from the Science article below:
While Schrag is rightly being praised for his role in the discovery, the real heros here are the short sellers here who led the process of uncovering this fraud (and who seemingly used Schrag as an independent expert to verify their own findings/suspicions). You can find their 42 page research report submitted to the FDA in Aug 2021 (under the cover of an attorney) here: https://www.regulations.gov/document/FDA-2021-P-0930-0004
It's worth mentioning that for items like this the FDA has a public commenting process whereby interested parties can submit their thoughts on such matters for consideration. From a cursory review, the comments are unsuprisingly filled with fda lawyers and medical professionals ridiculing the fraud of the short sellers and their motivated research. Go figure. (posted a couple below for fun, don't mean to be too hard on the specific commenters, since their assumptions here may be usually correct, but I also have to imagine that some of them were themselves making use of motivated reasoning...
Last thing I'll note is that in addition to the obvious importance of this discovery, the RAT/EA community should be thrilled that this discovery is in effect maybe the most important real world discovery from 'prediction markets'. For those unfamiliar with the financial markets for early stage biotechs, they are really the closest thing to scaled prediction markets out there. For single-drug companies that are in clinical trials, investors are literally making a asymmetric probabilistic bets on the outcome of trial success, and so as one might suspect the best investors in the space tend to be teams of PhDs conducting their own analysis to derive a P (success) and then betting if their expectations are different than the market. Happy to elaborate further on specifics here when I have time if people are interested.
So who are the short sellers that deserve our praise? As noted in the article, the two short sellers are neuroscientists who submitted the complaint anonymously through an attorney.
Using regulatory databases and some google-fu, it seems rather likely that the initial short seller (or one of them) was Granite Point Capital.
If we using an SEC filing aggregator we can see that three funds had large net put short positions in Cassava in 2H 2021, when the complaint was first filed. These are LMR Partners, Perceptive (one of the very best biotech focused funds), and Granite Point.
Granite Point is smaller, with only ~$500M in AUM, is seemingly biotech focused and active in aggresive short selling (just look up their litigation). A look on linkedin at their partner group shows multiple PhDs/MDs.
Most notably, I stumbled upon this 2019 class action where Labaton, the niche law firm firm filing the cassava, is representing Granite Point, which seems to make it pretty likely that Granite were again working with Labaton in their 2021 complaint against Cassava. Its very possible that they were part of a syndicate or that there was an independent short seller pitching this to multiple funds, but the labaton tie seems to make it very likely they were involved.
https://www.labaton.com/hubfs/Joint%20Declaration%20of%20Carol%20C.%20Villegas%20and%20Adam%20M.%20Apton%20ISO%20Final%20Appro....pdf
Quote from Science Article:
"In August 2021, Matthew Schrag, a neuroscientist and physician at Vanderbilt University, got a call that would plunge him into a maelstrom of possible scientific misconduct. A colleague wanted to connect him with an attorney investigating an experimental drug for Alzheimer’s disease called Simufilam. The drug’s developer, Cassava Sciences, claimed it improved cognition, partly by repairing a protein that can block sticky brain deposits of the protein amyloid beta (Aβ), a hallmark of Alzheimer’s. The attorney’s clients—two prominent neuroscientists who are also short sellers who profit if the company’s stock falls—believed some research related to Simufilam may have been “fraudulent,” according to a petition later filed on their behalf with the U.S. Food and Drug Administration (FDA)."
Sample Comments from FDA Complaint
"This is a self serving, deceptive and destructive petition that is not filed on behalf of the human beings this drug is intended to benefit, it is filed on the behalf of one or few short sellers who have gained financially at the expense of investors and future patients.
The petition had made reference to a “whistleblower” which is typically used to refer to someone inside the company. This “whistleblower” was later revealed to be someone outside the company that held a short position in the stock. The scientific queries within this do not seek the truth, they are poorly researched and a scapegoat for this unethical practice of stock manipulation."
"FDA please act expeditiously and firmly deny this abusive and collusive CP against Cassava Sciences by Labaton Sucharow. Already the Journal of Neurosciences have confirmed Cassava’s foundational research paper and found no evidence of manipulation and only 1 human error that doesn’t impact the scientific conclusion. Also, Yale researchers have recently confirmed PTI-125’s moa and affinity binding in mice studies for hard to treat seizures. No patients have been harmed by PTI-125 and you are already aware of the breakthrough improvements AD patients are experiencing. You must not allow any further delay of your decision bc there is an obvious short and distort campaign brought on by bad actors in collusion against Cassava. I believe the SEC is in the process of looking into the matter as many investor complaints (including mine) have been brought to the office of Enforcement. Cassava’s phase 3 SPA trials must not be further impacted."
Turns out a couple of short sellers beat the Journal of Neurosciences. Shows where the real truth-seeking analysis is being done.
Here is something I wrote about AD and amyloid a long time ago:
http://triviallyso.blogspot.com/2009/07/malleus-amyloidarum.html
The whole field of amyloid research in the so-called sporadic Alzheimer's disease was an idiotic boondoggle right from the start. The fraud is just the icing on the cake.
"Big booster energy. The official stats say ~50% of the vaccinated are boosted (and 10% of them were originally J&J which muddies things) and this has it at 59%, reflecting the bias of people who respond to polls I suppose. "
This discrepancy is mostly due to the official stats being bad, not survey bias. The CDC also believes that 90% of adults have had at least one dose of the vaccine, which is very hard to believe, compared to the poll's answer of 30%, which is plausible. The CDC can't reliably identify which shots are second/third/fourth doses, so they report lots of extra first doses, making the data more or less worthless.
You should be following Elizabeth Bik for research fraud stuff if you aren't already. She's very good at catching this stuff and these fraud-detection organizations already exist.
patio11 (Patrick McKenzie) is one of my favorite people to follow!
I was really sad that his last startup – Stockfighter – folded before I got a chance to play with it!
If you're not following it already, his new blog is great: https://bam.kalzumeus.com/
"Contrast this with Norway’s approach." – LOL; those lucky Norwegians!
I'd second a submission of "checking for obvious frauds" to Open Philanthropy.
https://www.causeexplorationprizes.com/
Though a worthwhile question is "why is peer review failing at this?"
Possibly because when you read a bunch of papers critically, finding no issues, Bayes perniciously dulls your senses. Maybe the panel for obvious frauds should require forced retirement after a year of service.
Or maybe just have a red team that assumes fraud, and speculates on most likely methods for data manipulation in every piece.
Papers commonly include a hand-wavy errors and limitations section. A para on areas of possible data manipulation, or ways the author voluntarily tied their hands to prevent that would be nice. (Did disinterested parties consult on any sudden outlier exclusions? Did you lock your data after collection or register it with a third party? Did you overcollect in a way that would allow shopping for p-values?)
Some risk this would become boilerplate too, but who knows, that might quickly force preregistration and data escrow as a norm.