What we've got here is failure to communicate – and adequately educate!
By Keith Devlin @profkeithdevlin
The first part of the title for this month’s essay is the oft-repeated line from the 1967 movie Cool Hand Luke, starring Paul Newman and distributed by Warner Brothers - Seven Arts. Actor Strother Martin’s iconic line as the prison warden, "What we've got here is failure to communicate", is one of the most famous movie quotes of all time.
A radio broadcaster I‘ve worked with in the past mentioned to me recently that she’d been reading some of my recent online posts on mathematical issues connected to the COVID-19 pandemic, in particular my posts on “Devlin’s Angle”. After we’d finished our brief exchange, I looked back to remind myself which specific topics I have covered. I was surprised to see that I’ve written eight coronavirus-related posts since the pandemic began (in the US) in March of last year. Specifically, my posts on April 1, May 1, June 1, and December 7 in 2020, and those on February 1, March 5, August 3, September 3 in 2021 were all related to COVID-19, though in some cases the post’s title did not explicitly reflect that connection.
It’s not that I decided I’d use my MAA mouthpiece to evangelize the (many) connections between mathematics and the management of epidemics. Rather, my posts simply reflected my personal interests as a mathematician, living through a major global pandemic. The fact is, I tend to see many (maybe most) things in my life through mathematical eyes.
Promotional flyer for Sunil Singh’s new book, to be published later this year by IMPress.
Talking of which, let me alert you to a forthcoming book by former math teacher Sunil Singh in which he describes his own experiences seeing the world through mathematical eyes: Chasing Rabbits: A Curious Guide to a Lifetime of Mathematical Wellness. I read it in proof form, and loved it.
Singh’s background as a math teacher differs from mine as a research mathematician, but we are both very self-aware of how our mathematical experiences changed the way we see the world and operate in it. Look out for the book when it comes out later this year.
But I digress. The point I was leading up to was observe that, after such a run, I had decided it was time to give the “math and the pandemic” theme a break for a while. But like the James Bond movie franchise, it just keeps coming back with a different cast.
This time, however, it’s not about mathematics per se, but more generally science (and mathematics).
More accurately still, my focus is going to be on how science and mathematics are done in real-world contexts and how they are used to work on issues and problems in the world.
The particular pandemic-related story that got my mathematician’s wheels turning once again was the recent decision to make COVID-19 vaccine booster shots available to three categories of adults who had their initial pair of Pfizer vaccines at least six months ago. Namely (and this recommendation is available on the CDC website):
Those 65 years and older
Those having underlying medical conditions
Those who live or work in high-risk settings
Links on the website provide explanations of what those second and third categories mean.
What made the announcement of these guidelines into a news story was the path that led to their adoption, every step of which was covered in detail by the media in real time.
For those of us who live in the world of science (and I include mathematics as a part of science in this essay), there was nothing of note. It was just science as usual. (More specifically, it was a routine illustration of the way science is used to decide public policy, or make business decisions, or inform military or defense strategies. More on that momentarily.) But to most of the journalists, and many of their readers and viewers, it seemed anything but routine. Cue this month’s title and the lead image.
The reason you are reading about this on the MAA website is that the episode highlighted the vast chasm of misunderstanding that separates many people outside the world of science from those of us within. Plainly, we do a poor job of making future citizens aware of what science is, how it is done, and how it works. Given the high – and sometimes, as in this instance, life-and-death – stakes that accompany the use of science in today’s world, this state of affairs is plainly intolerable. Those of us in science and mathematics education need to do better.
The problem, as I see it, is that we mostly teach science and math as if they exist in a vacuum, sealed off from the world around us. We present neatly packaged fragments that can be mastered in small pieces, five, ten, or fifteen minutes at a time, each one with a clean (and hence easily assessed and graded) right-or-wrong answer.
Is there then any wonder that our students graduate to become future citizens thinking that science and mathematics are all about establishing reliable facts (i.e., true statements about the world)?
I’ve addressed this issue in this forum before, most recently in my Devlin’s Angle post of November 1, 2019. As I wrote then, most real-world problems are what are called wicked problems, as opposed to the kind problems science and math teachers present to their students. (A shorter, simpler account, aimed at K-12 mathematics teachers, appeared on my SUMOP website on October 31, 2019.)
The majority of citizens, even if they graduate with mathematics or science college degrees, never encounter wicked problems. They go through their entire mathematics or science education thinking that mathematics and science focus on facts, with the goal being to find the unique, correct answers to well-formulated, precise questions. “What I like about math is that everything is right or wrong, so you always know how you are doing,” is a common student comment.
Yes, that is true for the restricted version of mathematics they are typically presented with, and for the kind problems they meet, but it does not scratch the surface of math used in the wild.
Yes, there is a good – and in my view inescapable – reason for teaching the all-important mechanics of science and mathematics that way. But if we do so at the expense of covering the daily praxis of science and mathematics in society, then we do a grave disservice to our students. The real world of human activities is, for the most part, messy; there are rarely right-or-wrong answers – heavens, there are rarely well defined, clear questions!
In the case of mathematics in particular, let me be clear that I’m not saying we should not continue to help our students learn about the broad field of pure mathematics, a rich domain that can keep you challenged and entertained for a lifetime. It is one of the great arts of humanity. But my present focus is on the practical need to do so in a way that (in addition) leaves our students with a realistic understanding of how mathematics is done and used in the world; what is it capable of, and what its limitations are.
The recent events leading up to the booster shots decision provides a dramatic illustration of our failure as educators to cover mathematics (and more generally science) praxis.
Briefly, the sequence of events that led to the CDC’s decision, a sequence that unfolded rapidly over just a few days, was this.
After examining data provided by Pfizer, the US Food and Drug Administration (FDA) issued an emergency use authorization (EUA) for the Pfizer vaccine to permit administration of a single booster dose, at least six months after completion of the primary series, for the three categories of recipients I listed above.
The matter then passed on to the CDC’s Advisory Committee on Immunization Practices, who, after some considerable discussion, eventually recommended the booster for the first two groups on the FDA’s list, but not the third.
Finally, the Director of the CDC, after consulting with her colleagues, made the decision for the boosters to be made available to all three of the original three categories.
All very routine, and surely unremarkable to anyone who has worked in an area where decisions involve considerations of scientific data. Again, I include mathematics here. In fact, the key data involved in making the decision on boosters was statistical data about risks of serious illness or death from COVID-19 infection. (More on this in a moment.)
But to journalists, for the most part with poor understanding of science and statistics, the process apparently came across as chaotic, confusing, and unclear. Indeed, some went as far as to report that the CDC “ignored the science”, when the fact of the matter is they based their decision on the science, and moreover did so following normal protocols.
Insofar as the CDC “failed” at anything, it was in not doing enough to explain the process in a way that laypersons could readily comprehend. That omission is understandable, since the parties involved in this decision-making process normally go about their work at a slower pace, out of the public eye. For the CDC as for everyone else, managing the fast-moving pandemic under full media gaze has been a learning curve. A steep one.
In fact, I know from interactions I have had with other mathematicians and scientists over the course of my career, that many of my colleagues who work in pure science or mathematics are also not familiar with the way math and science are used in business, commerce, and public administration. (Neither was I in the first half of my career, when I focused entirely on the internal mechanics of pure mathematics. I loved it. Still do, in fact!)
The key distinction to bear in mind with the booster shots story is that “following the scientific evidence or the data” does not mean “do what the scientific evidence or data says”. It cannot. Scientific evidence or data doesn’t say anything. It’s just what the words say it is: evidence or data. To be sure, evidence or data that comes from a scientific study carries a lot of weight, and should be treated as such. But most real life decisions involve other factors in addition to the scientific or mathematical ones. We elect societal representatives to form administrations that make those decisions.
The recent CDC episode actually presents a textbook illustration of how “scientifically-based decisions” are made in public health. (A similar story can be told in all other areas where wicked problems are solved using scientific evidence or data.)
First, there is a division of labor. Different steps of the process are handled by different people or groups, who bring different expertise.
In the booster-shots case, the FDA’s task was to decide if the use of the Pfizer vaccine as a booster was safe. There was a lot of data suggesting it was, but because of the urgency (we are still in the middle of a raging pandemic), there was a huge potential cost in illnesses or deaths if the normal, intentionally slow protocols were followed for authorizing new treatments. Releasing a new treatment always comes down to a judgement call. But in an emergency situation, the rules are relaxed a little, and an “emergency use authorization” is issued, to be replaced by a full authorization when the normally-required, more lengthy evaluation is completed. That occurred when the COVID vaccines were first made available late last year, and the same was true last week for the Pfizer booster shots.
There is still science and data. But not directed at clear-cut yes/no question like, “Is this totally safe?” You cannot say that about any drug. Rather, the scientists and decision makers ask questions like “How low is the risk of harm?” and, in particular, in the case of a vaccine, “How does the risk of a serious adverse reaction or death compare to the risk of serious illness or death as a result of infection by the virus if the vaccine were not available?” Committees of experts like those at the FDA and the CDC look for those kinds of answers.
Yes, the laboratory science, the field tests, and the statistical analyses are all carried out with full scientific and mathematical rigor. There are plenty of hard facts floating around. But in the end, a group of experts has to make a judgement call. That’s what the FDA committee did.
The same thing happened at the next step in the process, when the issue came before the CDC’s Advisory Committee. Their goal was to decide the best way to allocate the booster shots in society. And there you have a full-blown wicked problem.
For starters, what is “best”? Is the goal to prevent as many individuals from serious illness or death? Or is it to ensure the pandemic ends as quickly as possible? Or maybe the goal is to slow down the spread of the virus as much as possible so hospital systems are not overwhelmed? And there are other factors, for example, society’s need to get as many doses of the vaccine manufactured and made available to everyone on the planet as quickly as possible. You can’t have them all; if you focus single-mindedly on one, or just one or two, there is usually a cost to be paid with the others.
It seems clear from the decision the committee came to, together with remarks some of the members made subsequently, that they based their recommendation on making the boosters available only to those who had a statistically high probability of serious illness or death from a COVID infection. In particular, they did not make prevention from infection a high priority. (That was a judgement call.) Hence the exclusion of the FDA’s third category. But the committee were by no means unanimous. As some of them remarked afterwards, it was a close decision.
New York Times story of September 24, 2021. The headline and summary are misleading, since they fail to indicate that the “unusual” aspects arose because the circumstances were unusual. The decision making process actually proceeded as it should, with everyone doing what was required of them.
The New York Times published a story on September 24 that provides some illuminating insight into the Committee’s decision making, that is worth reading, although the story’s headline and some of its phrasing indicates the reporters made the same tacit assumptions that such decisions should be clear cut that I’m arguing against.
As the article reports, the Committee spent two days debating who should get boosters and when, and could not reach agreement as to whether occupational risk should qualify as a criterion. So they left it out.
Finally, the matter was passed to the CDC Director. Her job was to look at the recommendation of the Advisory Committee, take note of the various points the committee raised and discussed, and factor in the government’s policies for managing the pandemic. That’s a different goal, for which the Advisory Committee’s report was important data, but nevertheless just that: input data.
None of this process was about making the “right” decision. It’s a wicked problem. There is no “right” answer. Every step involved weighing the available evidence and coming to a decision. The “hard science evidence and data” that gets fed into the process at the start means that there is a lot more going on than “mere opinions.” It’s with good reason that those of us familiar with scientific and mathematical thinking have confidence in the process. But in the end, science and math cannot make the decision for us.
To be clear, we ignore the evidence and data that science and mathematics tells us at our peril; not taking it (very) seriously generally ends badly, sometimes deadly. But viewing those scientific and mathematical results as “telling us what to do” is also dangerous; indeed arguably more so, since the very scientific and mathematical precision and accuracy of those results can give us a dangerously false impression that our decision is “the right one.” Wicked problems don’t have “right answers.”