The Famous Medical Doctor and the Unknown Twitterer
Or how Texas COVID-19 Cases and Deaths by Vaccination Status Dashboard is NOT credible
Happy New Year! The following post is an interruption to the scheduled German births data programming ;) I just need to get this spat on Twitter out of my system…
This story starts when the following tweet by Dr Roger Seheult, ‘famous’ for his MedCram online educational medical content aimed primarily at med students, caught my attention:
Note the prominently bolded figures:
In the most recent 28 days with available data, compared to fully vaccinated Texans, unvaccinated Texans were:
13x more likely to test positve for COVID-19
14x more likely to die of a COVID-19 associated illness
Wow! In the era of Omicron variants such Relative Rate Ratios (nice CDC epdemiology primer on these calculations if you like :) are a throwback to the heady days of 95% vaccine efficacy claims. If true, unvaccinated being 14x more likely would equate to 93% vaccine efficacy). Well, my statistical spider sense was immediately tingling because even the MSM accepts those days are long gone, though I doubt they ever existed. Others were understandably equally skeptical but the good Doctor also tweeted this in response:
Unfortunately he was not being ironic in his use of incredible, and if you haven’t spotted it already, I will show that the headlined figures are quite simply NOT credible.
A little background: As I have mentioned in a previous post, my final damascene moment during the pandemic was when the my home state (Bavaria, Germany) introduced 2G regulations which restricted admission and participation in much of social life to only those who were vaccinated (Geimpft in German) or recovered (Genesen). Apart from the obvious ethical, moral, and legal issues involved in such discriminatory social exclusion, what drove me bonkers was the fraudulent case incident rates which were cited in parliament to support such measures. I had spotted (with the inspiration of Twitter sleuths and Substack friends) that the official Bavarian Health Department’s figures for unvaccinated cases were simply far too improbable to be true - and yes, I was right, they were wrong and had vastly exaggerated the unvaccinated case rates. TLDR: It’s not the first time I’ve noticed when case rates look dodgy.
So I fired off a couple of quick tweets expressing skepticism about the figures quoted, for example, questioning the testing frequency and associated positivity which are critical to correctly interpreting such data; and then I visited the Texas Health and Human Services’ COVID-19 Cases and Deaths by Vaccination Status Dashboard to check out the data.
The first thing that instantly struck me from just eyeballing the graphs as displayed was that they clearly did NOT show a 13x or 14x times rate differences - indeed not at any stage in 2022 would such 28-day ratios be accurate. With zero math, and put very simply: the “unvaccinated” curves would have to be 13x times higher that the “fully vaccinated” for those figures to be true. E.g. if the vaxxed case incidence rate was 10, then the unvaxxed would have to be 130 (and that ratio would have to be average for a 28-day period!). For those inclined, I have included a little math at the end of the post.
In the above tweet I not only pointed out the inaccurate 28-day Incidence Rate Ratios (or 28-day comparison as Texas Health also refers to them) are way off with respect to the available data but I also highlighted something the good Doctor had seemingly ignored, namely that the “boosted” rates were apparently WORSE than the “fully vaccinated” rates!? Note: the figures in green compare “unvaccinated” with “fully vaccinated” and the figures in orange compare “unvaccinated” with “boosted” so both figures use “unvaccinated” as reference. (I messed up a little by not selecting 11th November from both datasets, in my defense, the interactive graph was tricky on my smartphone)
The good Doctor did not like this factual observation and insisted that “boosted” rates were still better than “unvaccinated”. Sure, the Texas data does show this BUT I was intrigued that their data was ALSO showing worse rates for “boosted” than “fully vaccinated”. Was this confirmation of the Cleveland Clinic Study suggesting likelihood of infection was greater, the more doses received? - covered multiple times on Substack last week. Or just a case of waning efficacy? - alternative take.) This posed the legitimate question should someone double-dosed (“fully vaccinated” in Texas) go for that 3rd dose? This is a question thrown up by the Texas data cited by the good Doctor and should be worthy of consideration
The good Doctor’s condescending response (there’s lots more of that down thread) is that the “boosted” population is not comparable with the “unvaccinated” and contrary to the commonly observed healthy-vaccinee effect (a form of selection bias) where it is recognised that often the most health-conscious are the most conscientious about getting their jabs - annual flu jabs, etc. - whereas the most socially and health disadvantaged (who are actually in greatest need of medical care) are too often underrepresented in public health initiatives.
The irony of discluding and dismissing comparisons because of confounding effects involved with different population sets will hopefully not be lost on the reader as the good Doctor’s original tweet was ALL about such comparisons. There are much stronger indicators or factors to compare and predict Covid outcomes such as age, comorbidities, etc. -vaccination status is just one of many variables. And unfortunately WE DO NOT ACCURATELY KNOW the make up of any of the Texan “unvaccinated”, “fully vaccinated”, and “boosted” populations, so such comparisons always have to be treated with great care.
Ho hum.. unfortunately the 13x difference is NOT real - as anyone who can analytically eyeball a graph can readily see! (Also, see the simple math involved at the end of this post if you like.)
Now we come to what for me is the perhaps most interesting part about this Twitter spat: how do the “experts” interact and respond to “non-experts” online when their knowledge is questioned? The good Doctor refuses to honestly examine the data after I have clearly pointed out the inconsistencies between the Texas data and the Texas comparison ratios and instead goes to bat for Texas Health (and presumably the vaccines). He remains steadfast, dismissive, and ultimately condescending. This seems to be a sad trend observable again and again
I am accused of a form of cherry-picking, ie. slectively excluding data because it does not confirm my supposed biases. Of course, the good Doctor's original post was presumably because it confirmed his pro-vaccine bias ;)
This is a typical “I know what I am talking about", followed by “who are you to question the (great state of Texas’) experts" - a mix of arguing to authority and simultaneously an ad hominem questioning my math skills.
You know the debate has ended when one side starts LoLing (Laughing Out Loud) in response to the arguments of the other. But, hey, maybe that is just the limitation of Twitter as a platform for discussion of facts and details. Or maybe it us an indictment if the expert class and their guardianship of civil debate on subjects of important public interest.
In the last tweet the good Doctor condescendingly dismisses the legitimacy of Walgreens data and my observation that the Walgreens Covid-19 Index Dashboard, which is actually quite detailed including both test positivity and test proportion broken out by several vaccination statuses, and strongly conflicts with the Texas headline figures:
Any analytical thinker seeks alternative approaches or references with which to guage and compare results, and especially for results that don’t seem right. It lets you know if you are in the right ballpark, it's about looking for and noticing signals, correlations, confirmations, refutations, etc. The Walgreens data is up-to-date and suggests there are currently minimal differences in “unvaccinated” rates versus other cohorts.
But this isn’t all about Texas Health messing up their comparison ratios to the benefit of vaccinees - although that is certainly newsworthy! Neither is this post intended as a take down of @RogerSeheult. Please do not misunderstand, I actually quite like him and I have watched many of his videos and have found him to be very knowledgeable, an excellent communicator, and often willing to explore alternative approaches or interpretations. For me, this is symptomatic and highly tepresentative of countles discussions I have had online and IRL. People (myself included?) have extreme difficulty processing information which contradicts their beliefs and biases and the most common response is time and again to just double down.
This Twitter spat is a microcosm of the institutional inflexibility and intransigience in the face of inconvenient data or results. Á la the Titanic, the institutional ship of covid vaccines is not for turning, they have plotted their course and that is the direction we must steam ahead. Why? - because that approach was initially believed clear, because they believe they are unsinkable, because riff raff cannot subordinately correct the admiralty, because the ship has tragically too great a momentum to agilely react to updated incoming data…
Finally, here is one of the good Doctor's retorting tweets
This amounts to: anyway, I am not culpable because it is not my data, it's Texas' data I'm just tweeting it. Guess he didn't like my comment that it amounted to misinformation.
Oh, well, perhaps all is not lost, another online medical educator, the youtuber Dr John Campbell has admirably changed course over 2022 as more and more data has rolled in. Hopefully 2023 will be a year where the data speaks louder than institutional dogma, inertia, and indifference. I wish you and yours all a good slide (German joke) into the new year. May the truth prevail in 2023 and all who seek to walk in its light.
By the way, this is how my Twitter spat ended. I contacted the relevant bodies in Texas and I look forward to a retraction/correction appearing on their website soon :)
The data is publicly available and anyone can download for themselves the data via the interactive charts provided on the Texas Health website.
==Skip this last part if you don’t like mathematics==
Recall, (un)vaccinated incidence rates are calculated with middle-school math:
Incidence Rate = 100,000 x number of cases ÷ (un)vaccinated population size
Of course, the tricky part is accurately knowing the population sizes as Prof. Norman Fenton has excellently shown in his videos and articles on this topic with relation to UK rates by vaccination status.
28-day Incidence Rate Ratio = 28-day average of (unvaccinated rate ÷ vaccinated rate)
And yes, Texas Health’s own website confirms my understanding of this matter:
28-day rate comparisons, also known as the Incidence Rate Ratio, are calculated as the incidence rate among unvaccinated cases divided by the incidence rate among vaccinated cases and the incidence rate among boosted cases.
This stuff ain’t rocket science, even an anonymous twitterer can figure this out - (although admittedly I studied Electronic and Electrical Engineering at university in my youth, so I have quite a solid education in rigourous analytical thinking, math, statistics, etc.)
==End of Math - end of post==
==Updated 01.01.2023: I replaced some tweet links with screenshot images==
[UPDATE 06.03.2023:] I’ve had a pretty weird exchange in a comments thread of someone else’s substack – a post whose data failings and mistaken interpretations I had highlighted in my previous comments and also in my own substack post “Stop. Think. Check for Confirmation Bias!”. I know it’s not good to feed belligerent trolls, but here I go... a pro-vaxx fanatic (handle: asf32aa) has obviously gone to great effort to try to debunk my post here and yet all they have succeeded in doing is confirming the inconsistencies and failings in the Texas Health Dashboard which I had originally pointed out! I guess I should be grateful for that – so, thank you to the troll =)
Screenshots of an old Texas Health PDf (no link to source file) show that Texas Health should be correctly calculating Age-Adjusted Incidence Rate Ratios. The problem seemingly lies with their dashboard.
Now my response to the trollish accusations of my “errors”:
>> 1. you assumed that 'latest 28 days' means the most recent in the plot, it doesn't. They use data from at least 30 days prior to the most recent date in the plot after an embargo period.
Nope. In my mails to Texas Health I explicitly asked them to clarify the 28-day period for the data they were using. I highlighted a recent data point for Dr Seheult to emphasise the discrepancy between the values of the rates plotted and the ratios reported. As I pointed out in my post, at no point in 2022 did the plotted rates indicate the ratios as reported in the comparison rates (maximum unvaxxed:fully-vaxxed in 2022 was in fact 3x).
>> 2. You are using the crude age adjusted rates (your 41-something and 17-something etc) in the plot pop-up. See the newest data on deaths and use the 'totalrate' column to get the current 19x deaths on the current dash board.
What is a “crude age adjusted rate”??? - I always thought crude rates meant not age-adjusted. Anyway, I used the data values Texas Health plotted in their graph as tabulated in the COVID-19 Cases and Deaths by Vaccination Status Dashboard. If a graph is labelled Overall Age-Adjusted COVID Rates by Vaccination Status with Rates per 100,000 then I “assume” that’s what is being plotted. Fine, the new dashboard plots monthly values of a new data column “totalrate” for death rates, but the old dashboard plotted weekly values of Case and Death Rates from the plot pop-up (which correspond roughly to the old PDF).
Also, very funny Texas Health have scrubbed the Fully-Vaxxed data after it showed Boosted with worse rates than Fully Vaccinated =)
>> 3. You assumed that the math was wrong but the plot is wrong.
Wrong again. I repeated in my tweets, in my substack post, and in my mails to Texas Health that the graphs clearly did not equate to the comparison rates reported. Either the data provided in the dashboard and the plots of the Case Rates per 100K were faulty or the 28-day Comparison Rate calculations had to be. I never excluded the possibility the plots were wrong although the contemporaneous rate ratios from other dashboards suggested the incidence rate ratios in the graphs looked reasonable.
>> 3. continued… You can check this yourself: Do the integral on the online dashboard for the 28days in the pdf document for the sep21-oct21 time period (I assumed a binning of 4 weeks) and you'll get the same numerator (vax case rate of around 1767 cases per 100k), but you'll notice the denominator (the vax rate) is off by a factor of 3 in the plot. They are plotting some crude rate or doing some other adjustment for the fully vaxed case.
Oh dear, Texas Health dashboard really was/is a mess! (Even compared with CDC’s dashboard showing incidence rates by vax status.)
>> 3. continued… Otherwise it should match the PDF in that area. As for the Nov (really Sep) 2022: Notice the very similar shape (though both lines are lower) So take the 2-5X that you admit to in your post, scale the blue line down 3x and you get your ~13x that you're looking for.
Nope. For example, the case incidence rate ratios (unvaxxed:fully-vaxxed) indicated by the dashboard plotted data hadn’t exceeded 2.5x since March 2022. But the troll thinks if one assumes they are plotting “crude age-adjusted rates”, or assumes “crude rates” in age-adjusted graphs, or if one assumes they are doing “some other adjustment” to the rates for Fully Vaccinated and if we “scale” the denominator values then the graphs should match the 13x comparison rates reported, i.e. if we change the dashboard data, the results are correct! ROTFLMAO
==TLDR:==
Texas Health messed up their dashboard.
It was obvious from just eye-balling the graphs.
A hot-shot online medical content creator couldn’t recognise the glaring inconsistency.
A pro-vaxx troll read my post and still couldn’t recognise the glaring inconsistency.
The troll read the post again, then found an old Texas Health PDF and thought it was a slam dunk.
The troll makes ad-hoc assumptions to make the results fit the data, so it’s all good – LOL!
Remember, it doesn’t matter if Texas Health correctly calculated their comparison rates (using data different to the data they provided and plotted in their dashboard!?) - my original observations and criticisms stand.
Thank you for having the talent and the patience to go through claims like this. Your post would be excellent educational material in itself.
I suggest you visit @EthicalSkeptic twitter feed. He has done the best work on modeling and forecasting on Covid by far. According to him, the vaccinated is 2.6x more likely to be infected than the unvaccinated. My personal experience also bears it out.