How to use science to decide when you don’t know what’s going to happen
Making decisions with imprecise probabilities is a risky business
Making decisions with imprecise probabilities is a risky business
The media tends to like to point out either the risks or the benefits of reopening the economy versus staying locked down. It is rare to see any news outlet take a systematic approach, which is what we hope policymakers are doing. We know stats nerds are. With much of the economy now reopening, we hope for the best but what can we be certain of in uncertain times? Math may have the answer.
In my research on sensors, machine learning, and collaborative robotics, understanding how to make decisions when you don’t know what is going to happen is something I deal with all the time. Mathematics can tell us something about what we’re facing, even when there is so much we don’t know.
Every decision we make comes with an element of risk. Do I buy a Honda or a Toyota? Or should I save that money and take the bus? Is it safe to go to the grocery store or should I stay home and order in? Can I survive on unemployment checks until it is safer to work or should I go back now?
These decisions come with risks. They also come with uncertainty.
You can think of risks as the “known unknowns” and uncertainty as the “unknown unknowns” of life.
One big unknown unknowns that we are all dealing with right now is Coronavirus. I guarantee you that thousands of risk assessment meetings across countless boardrooms went on in late 2019 and early 2020 that made zero mention of a worldwide pandemic and global lockdowns. Could it have been accounted for? Possibly better than it was but not completely.
As a teenager, I was a huge Isaac Asimov fan. Asimov was best known for two things: his rules of robotics and the fictional science of psychohistory, the mathematics of predicting the future which he made a critical character in his Foundation series. Fundamentally, Asimov’s psychohistory was about decision making under uncertainty with the belief that large political movements could be predicted or at least managed.
Chaos theory tells us that we can’t do what Asimov wanted. There are too many unpredictable events that have out-sized impacts. In mathematical terms, they are sensitive to initial conditions. Even something as tiny and insignificant as a single change in a single strand of RNA in a single virus can have an enormous impact on human events. The “butterfly effect” means that nobody can predict even global events too far into the future.
Suppose, however, that there were a perfectly rational being in charge who could make decisions taking all factors into account and maximizing benefit at minimal cost. Knowing that the probabilities of any outcome are unknown and the benefits of certain actions also unknown, could even a super artificial intelligence make the right choice? What sort of mathematics could this being rely upon? What sort of strategy would such a being choose?
In collaborative robotics, we don’t tell the robots exactly what to do. We also don’t just let them figure it out on their own. They are part of a team and teams need to know how to work together. We use a playbook. A playbook is developed over a long period of time, many games, seeing what works and what doesn’t. It also takes an experienced coach to choose the right play for the situation.
Any decision maker always ought to have a playbook in their hands before they make any policy decisions. Making it up as you go along is a recipe for disaster. This playbook needs to be compiled by experts in both the topic at hand and policy making so they know what decisions are feasible. The playbook must constitute the set of all decisions that the policy maker is willing to make.
Still, how do we put data to use to make our choices from the playbook? If one of your experts is a mathematician, they might suggest a handful of mathematical methods for decision making with imprecise probabilities.
The mathematics of decision making under uncertainty (what we call imprecise probabilities) is concerned with making the best decisions given that you don’t know exactly what the worst thing that will happen is or how likely it is. Mathematicians define the best decision as the one that induces the highest gain, a number that measures the overall benefit of making a given decision.
Without uncertainty, the gain of making any choice is easy to calculate, even when the outcomes are random. For example, suppose you give me a fair die with six sides and tell me that if I roll a 6 you will give me $10 but if I roll anything else, I have to pay you $x. What should x be for me to take you up on the offer? It turns out there is a precise mathematical answer, it comes from calculating the “expected value” which is simply the cost or benefit of an outcome times its probability. If you sum up all the expected values of all possible outcomes, you get the total expected value. If the expected value is positive, you will, on balance, benefit. If it is negative, you will, on balance, lose. Casinos know this and so rig all their games to benefit the house in expected value.
In the case of the die, the probability of rolling a 6 is 1/6. The probability of rolling anything else is 5/6. 10 times 1/6 is 10/6. x times 5/6 is 5x/6. Subtract the two, 10/6 minus 5x/6 and you get (10–5x)/6 or 5(2-x)/6. Do a little math and you’ll see that if x is less than $2, I stand to gain. If x is more than $2, you stand to gain. Now, for one roll, I may still gain, even if x is large. Likewise, you might gain even if x is small. But over many, many rolls, on average, the odds are in the favor of the one who has the highest expected value.
If you want to win at any game of chance, all you have to do is figure out what the expected value is and make sure you can play many, many times in a row. This is why most games of chance in casinos and lotteries are rigged so that the “house always wins”. The stock market, on the other hand, usually has a positive expected value. As long as you place your bets on many, many stocks, as with a stock market index fund, you can’t go wrong in the long run.
You can apply expected value to many areas of finance, physics, chemistry, biology, and so on. Evolution has, over billions of years, determined by trial and error, what the expected values of many behaviors and anatomies are and caused our form and function to develop as it has to maximize our chance of survival.
So why not use expected value in decision making? You can, but expected value only works if you know what to expect. What if you don’t? Unlike evolution, we don’t have countless years and billions of lives to sacrifice to find the right play for the moment. Computer models might help with that but models depend on knowing what to expect too; how else would you build one?
Mathematicians have looked at this problem as well and developed a few strategies. If you are ignorant, all you have are your beliefs about reality to make decisions on. These aren’t, however, leaps of faith. Rather they are more like guesstimates. For every decision, you make a guesstimate as to the smallest expected payoff or “utility” for that decision. Likewise, you make a guesstimate for the largest expected cost of not making that decision.
Why do we care about smallest payoffs and largest costs? The reason is because we want to bound our expected payoff so that we know that it will be more than our guess.
For example, suppose I want to bet on a coin toss that is not guaranteed to be fair. The chance of its coming up heads is somewhere between 0.2 and 0.7. I have five dollars that I can divide between heads and tails in anyway I want. My gain is a function of possible outcomes that represents the division. What is the best division?
The expected utility of putting H dollars on heads and T dollars on tails is the smallest of these:
H x 0.2-T x 0.8
T x 0.3-H x 0.7
If I put $2 on heads and $3 on tails, for example, then my expected utilities are either a $2 loss or a $1.90 loss, so the expected utility is negative $2.
If I want to figure out what the best bet is, I need to find out what the maximum value of the minimum values of each of these functions are for every bet. With the constraint that H+T=5, that is pretty simple, it turns out to be the place where the two functions are equal or where the two lines cross.
That point is at H=$2.75 and T=$2.25. It is still expected to lose of course.
This way of making decisions is called “maximin” because we are maximizing minimum gains. It is also called “worst case optimization” decision making because it is so pessimistic [1]. Chess programs have used this to win against human players for decades.
You can also try to optimize the maximum gain. That is called “best case optimization”. That point is the highest point of either of the two lines. In this case, the best case is to put all your money on tails and none on heads. That gives you an estimated gain of about $1.50.
Suppose, however, I want to have a range of good choices instead of having these algorithms pick one for me? If I’m really uncertain of the probabilities then I might want to just filter out the bad ones.
There are a few ways to do that. One observation is that if the gain of making a particular decision D is always as good as all the others but better for just one E, then you should throw E away because it will never beat D. That is called picking an optimal set by “pointwise dominance”.
Once you’ve done that, you can compare each pair of decisions. You subtract their gains and compute the worst case expected values of the decisions as a pair. This is like looking at your decision as choosing one decision over another and seeing what you would lose. The ones that lose against all the other decisions, you throw away. This approach to decision making is called “maximality” because you are just finding a “maximal” set of decisions that are better than others.
Another option is just to look at cases where the expected values between your “best case scenario” and your “worst case scenario” for one given decision are all better than the expected values of another one. In other words, if my best case gain for a lockdown is 5 and my worst case gain is -7 but my best case gain for doing nothing is -8, then I have to throw away doing nothing even if lockdown isn’t my favorite choice. This approach is called “interval dominance” and is another good way to get rid of bad choices.
Our final option is called E-admissibility. In this scenario, we choose decisions that maximize at least one expected utility scenario. That means if I look at the expected utilities for every possible decision, for every set of probabilities that I propose, only those that are the best in one of those are kept scenarios are kept. This is like imagining many possible scenarios and throwing out the decisions that don’t do the best in any of them. The ones left over are the ones that have a shot of being the best choice.
Which approach is best? In general, if you are very uncertain, you don’t want to pick maximax or maximin because you will just get one answer no matter how little you know. E-admissibility, interval dominance, and maximality, however, will give you a number of good decisions in proportion to your uncertainty. Less certain, more answers. It’s like the Cheshire cat said, “if you don’t know where you’re going, any road will do.”
Discounting maximax and maximin, interval dominance tends to leave you with a lot of decisions left over. If all you want is to get rid of decisions that are completely bad, then that’s fine. Otherwise, better choose E-admissible or maximality. These turn out to be the same when you only have two choices. In general, E-admissibility is slightly stronger than maximality. It takes into account every possible scenario, but it could be vulnerable to abuse if you are overly optimistic since, unlike maximality, it does not only look at the most pessimistic scenarios.
In any case, no matter what you choose you won’t be guaranteed not to make a mistake. All you can guarantee is that you made the best choice you could.
[1] Troffaes, Matthias CM. “Decision making under uncertainty using imprecise probabilities.” International journal of approximate reasoning 45, no. 1 (2007): 17–29.