Monday, January 7, 2019

248: A Safe Bet

Audio Link


If you’re geeky enough to listen to this podcast, you’re probably also a fan of the “XKCD” webcomic by Randall Munroe, which bills itself as “A webcomic of romance, sarcasm, math, and language”.   (If not, be sure to check it out at xkcd.com.)    Recently I was especially amused as I browsed comic 1132, titled “Frequentists vs Bayesians”, which contains a hilarious example of what is known as the “Base Rate Fallacy”.

Here’s how the comic goes.   In Frame 1, a character states that he has a detector to tell him if the sun just went nova.   Remember that it takes light from the sun around 8 minutes to reach the Earth, so theoretically if this happened, you might not know yet.   However, the detector always rolls two dice, and if both come up 6s, it lies- giving a 1 in 36 chance of a wrong answer.   The detector has just displayed the word “Yes”, claiming that the sun did indeed go nova.   In the 2nd frame, a character points out that this means there is a 35 in 36 chance that the sun has indeed exploded— and since this is greater than 95%, the “p value” usually accepted as the standard in scientific papers, we must accept this answer as accurate.   In the 3rd frame, another character says “Bet you $50 it hasn’t”.

As is often the case in XKCD comics, this humor works on several levels.   In particular, if ever offered the chance to bet on whether or not the sun has just exploded, I would bet on the “no” side regardless of the odds.  Money just won’t be that useful in a universe where you have less than 8 minutes to live.   I’m also not so sure about the feasibility of the nova-detection machine, though the xkcd discussion page does claim that it might be possible using neutrinos, which are expelled slightly before the actual nova and travel at nearly the speed of light.     Anyway, for the moment let’s assume we’re some kind of faster-than-light capable and nova-immune alien spacefaring society, and think about this bet.

Something probably bothers you about believing the sun has exploded based on the word of a machine that occasionally lies.   But how do you get around the fact that the machine is right 35/36 of the time?    Doesn’t the math tell you directly which side to bet on?  This is the core of the base rate fallacy:   when trying to detect a specific incidence of an extremely rare event, you must consider both the independent probability of the event itself occurring AND the accuracy of your detection method.   In this case, any time we use the hypothetical machine, we are facing essentially four possibilities:   A.  The sun exploded, and our detector tells the truth.  B.  The sun exploded, and our detector lies.  C.  The sun is fine, and our detector tells the truth.   D.  The sun is fine, and our detector lies.   Since the machine said yes, we know we’re in situation A or D.

Now let’s look at the probabilities.   For the moment, let’s assume the sun had a 1 in 10000 chance of going nova.   (It’s actually a lot less than that, since our scientists are very sure our sun has a few billion more years in it, but this should suffice for our illustration.)    Situation A, where the sun exploded and the detector tells the truth, has a probability of 1/10000 times 35/36, or 35/360000.   Situation D, where the sun is fine and the detector lies, has a probability of 9999/10000 times 1/36, or 9999/360000.    So we can see that in this situation, we are 9999/35, or 287 times more likely to be fine than to be facing a nova.     Thus, even if we are all-powerful aliens, we should still be betting on the side that the machine is wrong and the sun is fine.

This comic makes us laugh, but actually makes a very important point.    There are many more concrete applications of this principle of the base rate fallacy in real life, as pointed out by the Base Rate Fallacy;s Wikipedia page.  The classic one is AIDS testing— if, say, a test quoted as “95%-accurate” claims you are HIV-positive, but you are in a very low-risk population, you are probably fine, and should arrange another independent test.   A scarier one is random “95% accurate” breathalyzer tests for drunk drivers— if there are very few drunk drivers on the road, but police set up a roadblock and test everyone, chances are that the innocent non-drunks falsely flagged by the machine will far outnumber the actual drunks.    This actually could apply to any police technique, such as finding terrorists based on profile data, that attempts to identify rare criminals in the general population.      

Another common case of this fallacy that has reached epidemic proportions lately is the use of supposedly “scientific” studies to justify exotic alternative medicine techniques.   For example, suppose you run a study of sick people given homeopathy, a method that violates hundreds of well-understood properties of chemistry and physics, such as Avogadro’s Number and core biochemical reactions.    Let’s say you get results indicating that it works with a “p value” showing a 95% probability that your test was accurate.   You can’t just quote that 95% without taking into account the independent probability that a treatment that violates so many known scientific laws would work— and when you take this into account, the probability that such a study has really given useful information is vanishingly small.     Thus the occasional studies that show good results for these scientifically-infeasible techniques are almost certainly false positives.

So, any time someone is discussing the probability of some extremely unlikely event or result with you in real life, regardless of the context, think about whether you might be ignoring some key factors and taking part in a Base Rate Fallacy.    If that might be the case, take a few homeopathic brain-enrichment pills and listen again to this podcast.

And this has been your math mutation for today.


References:



Friday, December 28, 2018

247: Wombat Geometry

Audio Link

Recently as I was watching TV with my wife, it suddenly occurred to me that we didn’t have an episode yet on one of the most important mathematical news developments of 2018.    Perhaps you’re thinking I should be featuring new developments in high-energy physics, or something about the recently awarded 2018 Fields Medal winners?    True, those would make good topics, but the one I’m thinking of came up as we were watching “Crikey It’s the Irwins”, that show about cute animals at the Australia Zoo.   I am, of course, talking about new advances in our understanding of cubic wombat dung.

Wombats, in case you haven’t seen one, are cute, furry marsupials related to koalas, and can be found mainly in Australia.    And yes, you heard me right— they are the only animals that generate cube-shaped poop.   I thought this was a joke at first, but numerous sources confirm it; you can find many photos online if you’re skeptical.    For a long time it was a mystery how wombats do this.   In real life, cubes are not an easy shape to generate:  in human-run manufacturing, you basically need to start with some kind of cubic mold, or directly cut materials into cube shapes.   So advances in our understanding of how this happens can have real economic benefits and applications to industry.    Recently a team of scientists led by Patricia Yang at Georgia Tech did some new experiments to discover exactly how this works.

You might guess at first that wombat dung is made up of some kind of crystal, since crystals are one major source of regular shapes with sharp angles in nature.   But that’s not it at all— this dung is similar smelly stuff to what other mammals generate, though it is a bit drier due to wombat metabolism, which helps it hold its shape.    You might then guess that there is effectively some kind of extrusion mold in the wombat’s bodily structure, similar to how our factories would generate that kind of shape.   But that’s not it either.   Somehow the wombats generate this cubic dung purely through soft-tissue activity, and don’t have any explicit square or cube shapes visible anywhere in their body.

Yang’s team ordered some roadkill wombat bodies from Australia, and did some experiments where they inflated balloons in their intestines to measure the elasticity, or stretchiness, at various points along their digestive tract.   They then did the same for pigs, as a control.  They found that the pigs’ intestines had roughly uniform elasticity, leading to the roundish dung generated by pigs and most other animals.   But wombats’ intestines were very irregular, containing some more and less stretchy parts, and in particular two groove-like stretchy areas.   Yang believes these are they key to shaping the dung as it travels down the tract.   Of course, there are a lot of followup questions to answer, such as how two stretchier areas lead to a full cubic shape, but the experiments are continuing.

At this point, you probably are also thinking of another question that Yang didn’t address:   *why* the wombats generate dung in this shape.   There must be some evolutionary advantage, right?    According to most online sources, the key is probably in the fact that they use the dung to mark territory.   Popular Mechanics suggests that the cubes can be stacked to build walls, but other sources note that wombats have never been observed to do this.  However, they do often leave their dung markers in precarious locations like on top of rocks or logs— this probably is able to signal to competitors from farther away than if left on the ground.    And if you want something to stay on top of a log or rock, a shape that is less likely to roll away provides a clear advantage.

I think I know the real secret, however.   Deep down in their burrows, wombats like to play Dungeons and Dragons during their downtime from foraging for food.   They must especially like playing wizards, and need lots of 6-sided dice to roll damage for their fireball spells.   Maybe as D&D geeks grow to dominate the human race, we too will develop this evolutionary manufacturing shortcut.    I prefer the smell of plastic dice personally, but I’m probably just less evolved than the common wombat.

And this has been your math mutation for today.


References:




Sunday, December 2, 2018

246: Election Soutions Revisited

Audio Link

Way back in podcast 172, I proposed a solution for the issue of contentious election recounts and legal battles over very close vote totals.   With all the anger, bitterness, and waste that handling these close-margin races causes, I thought it would be good to review this proposal again, and really give it some serious thought.

Our nation collectively spends millions of dollars during each election cycle on such issues, and the number of contentious races seems to have exploded in recent years.   One example is the recent Florida senate race, where over 8 million votes were cast, and the victory margin was within 10000, around an eighth of one percent.  Due to the increasing partisan divide, these close margins are now universally accompanied by accusations of small-scale cheating in local precincts or at the margins.   And we can’t deny that due to the all-or-nothing stakes, and the ability of a relatively tiny set of local votes to swing an election with national ramifications, the temptation for local partisans on both sides to cheat in small ways, if they spot an opportunity, must be overwhelming.   

Now a lot of people will shrug their shoulders and say these contentious battles and accompanying waste are an inevitable consequence of democracy.   But as I pointed out in the earlier podcast, if we think carefully, that’s not quite true.   The first key point to recognize, as pointed out by authors such as Charles Seife, is that there is a *margin of error* to the voting process.    Rather than saying that Rick Scott got 4,099,805 votes and Bill Nelson got 4,089,472 votes, it might make more sense just to say that each got approximately 4.1 million votes.    There are errors due to mishandled ballots, natural wear and tear, machine failures, honest mistakes, and even local small-scale cheating.   Once you admit there is a margin of error, then the silliness of recounts becomes apparent:  a recount is just a roll of the dice, introducing a different error into the count, with no real claim to be more precise or correct.   So under the current system, if you lose a close election, it would be foolish not to pour all the resources you have into forcing a recount.

But if we agree that the voting process has a margin of error, this leads to a natural solution, suggested by Seife:   if the votes are close enough, let’s just agree to forsake all the legal battles, recounts, and bitter accusations, and flip a coin.   It would be just as accurate— just as likely to reflect the true will of the people— and a much cheaper, faster, and amicable method of resolution.   But if you think carefully, you will realize that this solution alone doesn’t totally solve the issue.   Now we will repeat the previous battles wherever the vote is just over the margin that would trigger a coin flip.    For example, if we said that the victory margin had to be 51% or greater in a 1000-vote race to avoid a coin flip, and the initial count gave the victor precisely 510 votes, there would be a huge legal battle by the loser to try to shave off just one vote and trigger his random shot.

This leads to my variant of Seife’s proposal:  let’s modify the system so that there is *always* a random element added to the election, with odds that vary according to the initial vote count.   We will use a continuous bell curve, with its peak in the middle, to determine the probability of overturning the initial result.   In the middle, it would be a 50% probability, or a simple coin flip.   In our 1000-vote election, at the 510 vote mark the probability would be very close to 50-50, but a smidgen higher, depending on how we configure the curve, something like 51%.    Now the difference between a 50% chance of winning and a 49%, or 51%, will probably not seem very significant to either candidate:  rather than fighting a legal battle over the margin, they will probably want to go ahead, generate the random number, and be done with it.    Of course we will agree that once the random die is cast, with the agreement of all parties to the election, both winners and losers accept the result without future accusations or legal battles.   Due to the continuity of the curve, there will never be a case where a tiny vote margin will seem to create life-or-death stakes.   The probability of overturning the election will fall smoothly as the margin increases, until it gets down to approximately zero as the vote results approach 100%.   So Kim Jong Un would still be safe.

You can probably see the natural objection to this scheme: as you move further out from the center of the curve, having *any* possibility of overriding the result starts to seem rather undemocratic.   Do we really want to have a small chance of overturning the election of a victor who got 60% of the vote?   If this system is implemented widely, such a low-probability result probably will happen somewhere at some point.   But think about all the other random factors that can affect an election:  a sudden terrorist attack, mass layoff at a local company, random arrest of a peripheral campaign figure, a lurid tabloid story from a prostitute, a sudden revelation of Math Mutation’s endorsement—- the truth is, due to arbitrarily timed world events, there is always a random factor in elections to some degree.   This is just a slightly more explicit case.   Is it really that much less fair?    And again, think of the benefits:  in addition to saving the millions of dollars spent on legal battles and recounts, the reduction in the need for bitter partisan battles in every local precinct on close elections has got to be better for our body politic.

So, what do you think?    Is it time for our politicians to consider truly out-of-the-box solutions to heal our system?   Maybe if all the Math Mutation listeners got together, we could convince a secretary of state somewhere to try this system out.   Of course, I know I’m probably just dreaming, outright nuclear war in Broward County, Florida is a much more likely solution to this issue.   At least I’m located pretty far outside the fallout zone for that one.

And this has been your math mutation for today.


References:




Sunday, October 7, 2018

245: How Far Apart Are Numbers?

Audio Link

When you draw a number line, say representing the numbers 1 through 10, how far apart do you space the numbers?    You might have trouble even comprehending the question,   If you’ve been educated in any modern school system in a developed country, you would probably think it’s obvious that the numbers are naturally placed at evenly spaced intervals along the line.    But is this method natural, or does it simply reflect what we have been taught?    In fact, if you look at studies of people from primitive societies, or American kindergarten students who haven’t been taught much math yet, they do things slightly differently.   When asked to draw a number line, they put a lot of space between the earlier numbers, and then less and less space for each successive one, with high numbers crowded together near the end.    As you’ll see in the show notes, ethnographers have found similar results when dealing with primitive Amazon tribesmen.  Could this odd scaling be just as ‘natural’ as our evenly spaced number line?

The simplest explanation for this observation might be that less educated people, due to their unfamiliarity with the task, simply don’t plan ahead.   They start out using lots of space, and are forced to squish in the later numbers closer together simply because they neglected to leave enough room.    But if you’re a fellow math geek, you have probably recognized that they could in fact be drawing a logarithmic scale, where linear intervals are proportional to ratios rather than quantities.   So for example, the space between 1 and 2 is roughly the same as the space from 2 to 4, from 4 to 8, etc.    This scale has many important scientific applications, such as its use in constructing a slide rule, the old-fashioned device for performing complex calculations in the days before electronic calculators.    Is it possible that humans have some inborn tendency to think in logarithmic scales rather than linear ones, and we somehow unlearn that during our early education?

Actually, this idea isn’t as crazy as it might sound.   If you think about it, suppose you are a primitive hunter-gatherer in the forest gathering berries, and each of your children puts a pile of their berries in front of you.   You need to decide which of your children gets an extra hunk of that giant platypus you killed today for dinner.    If you want to actually count the berries in each pile, that might take a very long time, especially if your society hasn’t yet invented a place-value system or names for large numbers.    However, to spot that one pile is roughly twice or three times the size of the other can probably be done visually.    In practice, quickly estimating the ratio between two quantities is often much more efficient than trying to actually count items, especially when the numbers involved are large.   So, thinking in ratios, which leads to a logarithmic scale, could very well be perfectly natural, and developing this sense may have been a useful survival trait for early humans.     As written by Dehaene et al, in their study linked in the show notes, “In the final analysis, the logarithmic code may have been selected during evolution for its compactness: like an engineer’s slide rule, a log scale provides a compact neural representation of several orders of magnitude with fixed relative precision.”

The same study notes that even after American children learn the so-called “correct” way to create a number line for small numbers, for a few years in elementary school they still tend to draw a logarithmic view when asked about higher numbers, in the thousands for example.   But eventually their brains are reprogrammed by our educational system, and they learn that all number lines are “supposed” to be drawn in the linear, equally-spaced way.    However, even in adults, additional studies have shown that if the task is made more abstract, for example using collections of dots too large to count or using sound sequences, a logarithmic type of comparison can be triggered.    So, it looks like we do not truly lose this inherent logarithmic sense, but are taught to override it in certain contexts. 
   
I wonder if mathematical education could be improved by taking both views into account from the beginning?   It seems like we might benefit from harnessing this innate logarithmic sense, rather than hiding it until much more advanced levels of math are reached in school.   On the other hand, I could easily imagine young children getting very confused by having to learn to draw two types of number lines depending on the context.    As with everything in math education, there really is no simple answer.

And this has been your math mutation for today.

References:






Saturday, September 8, 2018

244: Is Music Just Numbers?

Audio Link


I was surprised recently to hear an advertisement offering vinyl records of modern music for audio enthusiasts.   In case you were born in recent decades, music used to be stored on these round black discs called “records” in an analog format.    This means that the discs were inscribed with grooves that directly represent the continuous sound waves of the music.   In contrast, the vast majority of the music we listen to these days is digital:   the sounds have been boiled down to a bunch of numbers, which are then decoded by our iPods or other devices to reproduce the original music.    I’ve never noticed a problem with this, but certain hobbyists claim that there is no way a digital reproduction of music can be as faithful as an analog one.   Do they have a point?

The key issue here is the “sampling rate”.   As you would probably guess, there is no way a series of numbers can represent a truly continuous music wave— they are representing sounds at a bunch of discrete points in time, which are played quickly right after each other to produce the illusion of continuity.    These rates are too fast for most of us to discern:  the compact disc standard, for example, is a sampling rate of 44.1 KHz, or about 44,100 samples per second.   This is far more than generally accepted estimates of what our ears can perceive.   It also meets the criteria calculated by Swedish-American engineer Harry Nyquist in the early 20th century, showing that this sampling rate is more than sufficient to faithfully reproduce all sounds within the range of human hearing.    But the small group of audiophiles who insist on listening to vinyl claim they can hear the difference.

These analog boosters of vinyl seem to be relatively vocal in online communities.  They claim that there are subtle effects that make analog-reproduced music seem “brighter”, that digital has a “hard edge”, or other vague criticisms.    There are also claims that the analog method provides an “emotional connection” to the original artist that is destroyed by digital.   The most overwrought of them claim they are saving the world, with statements like, “future generations will be sad to realize that we didn’t preserve the majority of our music, we just made approximate digital records of it.”   Some of them do admit that part of their enthusiasm comes from the rituals of analog music:  leafing through records in a store, carefully pulling it out of the sleeve and placing the needle, etc.  

You can think of audio sampling rates kind of like pixels in computer graphics, the small blocks that make up the images we see on any modern computer screen.  in old computers from the 1980s, you could clearly see the pixels, since computer memory was expensive and the resolution, or density of the pixels, rather poor.    But these days, even professional photographers use digital equipment to capture and edit photos.    If you have purchased wedding pictures or something similar from a digitally-equipped professional, I doubt you have looked at the pictures and noticed anything missing.     I’ve seen this lead to a few arguments with some of my snobbier friends about the need for art museums:   since we can see high-resolution digital reproductions of just about any classic art online, why do we need to walk to an old building and stare at the painstakingly preserved original canvases?    I think the “museum ritual” is kind of similar to the vinyl ritual:  it evokes emotional memories and nostalgia for a past history, but isn’t really necessary if you think it through.

Getting back to the audio question, what we would really like to see are double-blind studies challenging audiophiles to distinguish analog vs digital music without knowing its source.   I was surprised not to see any online— perhaps the logistics of such a study would be challenging, since you would have to use completely non-digital means to transmit the sound from both sources to the listening room, and avoid telltale giveaways like the need to flip records after a while.   But in 2007 the Boston Audio Society did something pretty close, asking a bunch of volunteer audiophiles in a bunch of genres to try to distinguish recordings played at a standard CD sampling rate, 44.1 KHz, vs high-resolution formats recorded in formats like SACD, which sample at around 50 times higher rates.   They would take the high-resolution recordings, and transfer them back to CD format for the experiment, so they were comparing the exact same music at the two sampling rates.   Surely if sounds lost at the standards CD sampling mattered, these improved formats should give much better sound, right?    

As you have probably guessed, their results showed that the massively higher sampling rate made essentially no difference— the volunteer audiophiles could not distinguish the high-resolution and CD-quality recordings any better than chance would predict, at 49.82%.    So, it looks like the human ear really can’t distinguish sampling rates beyond what a CD provides, which after all is not a surprise in relation to the Nyquist calculations we mentioned earlier.      Of course, some New Age types will continue to claim a mystical emotional connection that can only be transmitted through analog, and I’ll have to let them debate that with their spirit guides.    But if you look at the science, you don’t need to comb the garage sales for old record players and unscratched vinyl—digital music should work just fine.

And this has been your math mutation for today.




References:



Sunday, July 29, 2018

243: The Consciousness Continuum

Audio Link


Most of us think of consciousness as a kind of division between several states.   We can be awake, unconscious, or maybe somewhat drowsy on the boundary between the two, and occasionally interact with our out-of-reach “subconscious” without realizing it.   But is there more to the concept?   In his book “The Tides of Mind”, Yale computer scientist & cognitive researcher David Gelernter makes the case that consciousness is more of a continuum:   at any given moment, you are at some point on a spectrum ranging from pure thinking to pure feeling.    You  move up and down the spectrum depending on how focused and logical your thoughts are and how connected with the outside world.  At the bottom, you retreat inward into your mind, and fall asleep.    

So, what does this have to do with mathematics?    Gelernter points out a famous quote by physicist Eugene Wigner about John Von Neumann, who is widely acknowledged as one of the greatest mathematicians of the 20th century:  “Whenever I talked with Von Neumann, I always had the impression that only he was fully awake”.    Now of course, we need to take Gelernter’s use of this quote with a grain of salt, since Wigner was probably speaking in a colloquial sense and not aware of this modern theory.    But he does point out that in this spectrum theory, logical thinking and reasoning is at the very top, indeed the most ‘awake’ portion of that spectrum.   And this might also explain why mathematics tends to be more challenging to the average person than many other human pursuits:   it requires that you keep focused attention, avoiding any temptation towards reminiscence, daydreaming, or emotion.   And if you fall asleep, you probably won’t get any math done.

Of course, there are certain other types of genius that can occur when someone has exceptional abilities farther down the spectrum.   Gelernter also points out the example of Napolean, who was skilled at stirring the emotions of his followers, and could draw vast quantities of past military experiences from his memory to guide his plans and policies.   As a young officer Napoleon is said to have claimed, “I do a thousand projects every night as I fall asleep”.   In other words, in his semi-conscious state near the bottom of the spectrum, he could easily retrieve various scenarios from his memory, varying them and playing with them creatively to discover different ways the next day’s battle might play out.  

The position of emotions and memories in this theory is also somewhat strange.   Gelernter writes, “Emotion grows increasingly prominent as reflective thinking fades and the brightness of memories grows— and by not creating memories, we unmake our experience as it happens”.   In other words, what others might call the subconscious is simply your internal array of memories; rather than having a separate subconscious mind, the illusion of a subconscious is just what results when you are low on the spectrum and increasingly drawing from your memories instead of logically observing the outside world.   Emotions are our internal summaries of the flavor of a set of memories, which become increasingly prominent as we are lower on the spectrum.

One interesting consequence of this theory, in Gelernter’s view, is that “computationalism”, the idea that the human mind might be equivalent to some advanced computer, is fundamentally wrong.   Due to our reliance on the capabilities of the mind to analyze memories in an unfocused way and generate emotions, no computer could replicate this state of being.   I may not be doing the theory justice, but I don’t find this argument very convincing.    Ironically, it might be said that he’s assuming a computer model of a human mind is restricted to a “Von Neumann architecture”, the type of computers that most of us have today, and which seem to directly implement the logical, mathematical thought that occurs at the top of our spectrum.   But there are many alternative types of computers that have been theorized.  In fact,  currently there is explosive growth happening in “neural network” computers, inspired by the design of a human brain.   While I would tend to agree with Gelernter that the spectrum of consciousness would be very hard to model in a Von Neumann architecture, I would still bet that some brain-like computing device will one day be able to do everything a human brain can do.   On the other hand, that might just be a daydream  resulting from my down-spectrum lack of logical thinking at the moment.

And this has been your math mutation for today.

References:


Sunday, June 24, 2018

242: Effort and Impact

Audio Link


I’m always amused by management classes and papers that take some trivial and obvious mathematical model, give it a fancy name and some imprecise graphs, and proudly proclaim their brilliant insights.    Now of course, such models may be a useful tool to help think through daily issues, and should be used implicitly as a simple matter of common sense, but it’s really important to recognize their limitations.   Many limitations come from the imprecision of the math involved, human psychological factors, or the difficulty of translating the many constraints of a real-life situation into a few simple parameters of a model.    I was recently reminded of a classic example of this pitfall, “Impact-Effort Analysis”, when reading a nice online article by my former co-worker Brent Logan.   While the core concepts of impact-effort analysis are straightforward, it is critical to recognize the limitations of trying to apply this simple math to real-life problems.

The basic concept of impact-effort analysis is that, when you are trying to decide what tasks to work on, you should create a list and assign to each an “impact”, the value it will bring to you in the end, and an “effort”, the cost of carrying out the task.   You then graph all the tasks on an impact vs effort graph, which you divide into four quadrants, and see which quadrant each falls into.   For example, you may be considering ways to promote your new math podcast.   (By the way, this is just a hypothetical, who would dare compete with Math Mutation?)     A multimedia ad campaign would be high effort but high impact., the quadrant known as a “big bet”.   Posting a note to your 12 friends on your personal Facebook page would be low effort and low impact, which we label as “incremental”.     Hiring a fleet of joggers to run across the country wearing signboards with your URL would probably be high effort and low impact, which we label the “money pit” quadrant.    And finally, using your personal connections with Weird Al Yancovic to have him write a new hit single satirizing your podcast is low effort and high impact— this is in the ideal quadrant, the “easy wins”.   (And no, I don’t really have such connections, sorry!)

This sounds like a sensible and straightforward idea— who could argue with rating your tasks this way in order to prioritize?    Well, that is true:  if you can correctly assign impact and effort values, this method is very rational.    But Logan points out that out old friends, the cognitive biases, play a huge role in how we come up with these numbers in real life.    

On the side of effort estimation, psychological researchers have consistently found that small teams underestimate the time it takes to complete a task.   Kahneman and Tversky have labeled this the “Planning Fallacy”.   Factors influencing this include simple optimism, failing to anticipate random events, overhead of communication, and similar issues.   There is also the question of dealing with constant pressure of bosses who want aggressive-looking schedules to impress their superiors.     You may have heard the well-known software engineering saying that stems from this bias:   “The first 90% of the project takes the first 90% of the time, and the last 10% of the project takes the other 90% of the time.”

Similarly, on the impact side, it’s very easy to overestimate:  when you get excited about something you’re doing, especially when the results cannot be measured in advance.    In 2003, Kahneman and Lovallo extended their definition of the Planning Fallacy to include this aspect as well.  I find this one pretty easy to believe, looking at the way things tend to work at most companies.    If you can claim you, quote, “provided leadership” by getting the team to do something, you’re first in line for raises, promotions, etc.   Somehow I’ve never seen my department’s VP bring someone up to the front of an assembly and give them the “Worry Wart Award” and promotion for correctly pointing out that one of his ideas was a dumb waste of time.  If your company does that effectively, I may be sending them a resume.

Then there are yet more complications to the impact-effort model.   If you think carefully about it, you realize that it has one more limitation, as pointed out in Logan’s article:   impact ratings need to include negative numbers.   That fleet of podcast-promoting joggers, for example, might cost more than a decade of profits from the podcast, despite the fact that the brilliant idea earned you a raise.    Thus, in addition to the four quadrants mentioned earlier, we need a category of “loss generators”, a new pseudo-quadrant at the bottom covering anything where the impact is below zero.

So what do we do about this?    Well, due to these tendencies to misestimate, and the danger of loss generators, we need to realize that the four quadrants of our effort-impact graph are not homogeneous or of equal size.    We need to have a very high threshold for estimated impact, and a low threshold for estimated effort, to compensate for these biases— so the “money pit” quadrant becomes a gigantic one, the “easy win” a tiny one, and the other two proportionally reduced.    Unfortunately, it’s really hard to compensate in this way with this type of model, since human judgement calls are so big a factor in the estimates.    You might subconsciously change your estimates to compensate.   Maybe if you had your heart set on that fleet of joggers, you would decide that its high impact should be taken as a given, so you rate it just enough to keep it out of the money pit.

Logan provides a few suggestions for making this type of analysis a bit less risk-prone.   Using real data whenever possible to get effort and impact numbers is probably the most important one.    Also in this category is “A/B testing”, where you do a controlled experiment comparing multiple methods, on a small scale before making a big investment.   Similarly, he suggests breaking tasks into smaller subtasks as much as possible— the smaller the task, the easier it is to come up with a reasonable estimate.   Finally, factoring in a confidence level to each estimate can also help.     I think one other implied factor that Logan doesn’t state explicitly is that we need to be consistently agile:   as a project starts and new data becomes available, constantly re-examine our efforts and impacts to decide whether some re-planning may be justified.

But the key lesson here is that there are always a lot of subtle details in this type of planning, due to the uncertainties of business and of human psychology.  Always remember that writing  down some numbers and drawing a pretty graph does NOT mean that you have successfully used math to provide indisputable rationality to your business decisions.  You must constantly take ongoing followup actions to make up for the fact that human brains cannot be boiled down to a few numbers so simply.

And this has been your math mutation for today.

References: