Category Archives: Book Review

Information Dashboard Design – Stephen Few

“Eloquence through simplicity”

One of the largest challenges facing high-level management or leaders is information. A large organization is complicated at the best of times, and the information passed on by lower-level management is distorted by their own biases and interests. Even when you get unbiased information, it may not be clear what matters or what you should pay attention to.

Enter the dashboard. There are many definitions, but Stephen Few uses a relatively simple one: a dashboard is a “visual display of the most important information needed to achieve one or more objectives, consolidated and arranged on a single screen so the information can be monitored at a glance.”

In other words, the reader should be able to look at the dashboard and take in what they need to know in a single glance. They are an overview of performance.

So far so good, and yet many dashboards fail to achieve this purpose. They use misleading diagrams, or colours that provide no information but make it hard to take information in quickly, or have a poor data-to-ink ratio. Some will clutter the display with unnecessary visual effects, or arrange information poorly, or fail to consider exactly what the audience needs to know.

The solution is simplicity. Every pixel in a dashboard should have a purpose, or you should remove it. Dashboards should 1) maximize the data-to-ink ratio, and 2) emphasize the data. IT workers should push back when customers demand poor dashboards, and give them not what they want, but what they need. No dashboard can do everything, and pretending they can will only create confusion.

The book spends most of its time going through examples, which are particularly informative. Few highlights some dashboards that fail, and some that succeed, in a wide variety of contexts. This isn’t a casual read, but if you use dashboards in your own life (and they are surprisingly general in purpose – you could use one for your physical exercise, for example!), it is an excellent one, full of rich advice and good ideas.

The Ethics of Influence – Cass Sunstein

“The private sector inevitably nudges, as does the government. No government can avoid some kind of choice architecture. We can object to particular nudges, and particular goals of particular choice architects, but not to nudging in general.” – The Ethics of Influence

Two young fish are swimming along in the ocean. They pass an older fish, who says “morning boys, how’s the water?” The young fish keep swimming along (like many young fish, they are somewhat rude to their elders), until one turns to the other and says: “What’s water?”

This, says Cass Sunstein, is the story of how we are affected by our environment. The effect is so everpresent we often don’t even know it is there, and it is inescapable. At a time when nudges have been enormously popular with governments (see, for example, the UK Behavioural Insights Team, or the US Social and Behavioral Sciences Team), the question of whether they are ethical is very salient.

Perhaps unsurprisingly given Sunstein’s role in popularizing nudges, The Ethics of Influence argues they are. It advances four key arguments: that individual nudges can definitely be unethical, but that nudging qua nudging is not unethical in the same way that taxes qua taxes are not; that objections are pointless, because nudges are inescapable, such as snowy days increasing how many four-wheel drives are purchased and then returned to car dealers; that it’s possible to set the rules of the game, including nudges, without predetermining outcomes (for example, by making a decision easier across the board); and that in surveys, people prefer educational campaigns to direct nudges, but if they are told that education is less effective, support for nudges increases.

Sunstein’s definition of nudges is quite generous, which can sometimes muddy the waters: he includes all educational campaigns as a nudge, for example. That’s not wrong—after all, it is a non-coercive form of affecting behaviour—but it is very different from the examples that most people have ethical concerns about, such as using opt-out to encourage organ donation. At a few points, the book feels closer to salesmanship than analysis: in a chapter on environmental nudges he repeatedly raises the example of setting printers to double-sided defaults, cutting paper use by 40%. It’s a good and useful example, but if you’re already a critic of nudges, that isn’t what you have in mind.

Ethics of Influence is more academic than some of Sunstein’s recent books, and so readers expecting Wiser, Simpler, or Nudge may be a bit taken aback. It is heavily footnoted and abstract, making it somewhat hard to follow at points. It’s also on a very important topic, and one that is making huge waves in government. Probably not a book for the casual reader, but if you’re already interested in the field, Sunstein is a giant in it, and his arguments are relevant and important.

Disclosure: I read this book as an advance reader copy. You can read more reviews and order it here: The Ethics of Influence.

If we can put a man on the Moon…: Getting big things done in government – William D. Eggers and John O’Leary

“The requirements for achieving great things are two simple but far from easy steps–wisely choosing which policies to pursue and then executing those policies. The difference between success and failure is execution.”

It has become something of a truism that just having good ideas isn’t enough: they also have to be implemented well. Eggers and O’Leary (who should be a comedy duo, based on their names) argue that because we don’t always realize this, we tend to spend a lot of time looking for someone to blame when something goes wrong, rather than realizing the flaw is often systemic. When we try to solve a systems problem by changing an individual, we inevitably fail.

The book maps out how to end up with a good public policy. Start with a good idea; turn that idea into a specific design, often through writing some draft legislation; win approval for the idea (more of an issue in the American system than in Parliamentary ones, in which the party often has a majority already and so the parliamentary step is straightforward); implement it competently; generate the desired results; and over time, re-evaluate and look for ways to do better. All the while, of course, avoiding classic traps like confirmation bias, overconfidence, and complacency.

The book is strongest if you aren’t familiar with the various ‘innovative’ practices going around already. Unfortunately, most of their examples—prizes for innovation, hackathons, the London congestion charge, etc—have been analyzed at length in the past. Eggers and O’Leary come from the American civil service, for which the practices are relatively new, but for other readers most of it won’t seem that exciting. The authors also have a disturbing tendency to make up their own non-intuitive names for things, which can sometimes be confusing. Overall, not a bad book, but probably not new for most readers in the field.

Understanding Variation – Donald J. Wheeler

“Businessmen are finding that while they have more numbers than ever before, they still do not know what these numbers mean.”

“This book was written to help organizations overcome the effects of numerical naivete.”

Understanding Variation is a classic in understanding and measuring how businesses work. Its key lesson is that things vary. When we see that trade is down in February compared to last year, it’s tempting to assume that means something. The media will certainly report it as such, but we really don’t have any reason to do so. For that data to be meaningful would require that last year be ‘normal.’ I’m not sure that’s ever happened.

Variation is a significant challenge for businesses. If a business has bad results one month, should it make changes? Wait? Ignore it? There are no hard and fast rules, though entire disciplines have arisen over what type of results mean something (see: Six Sigma). Even good managers fall victim to regression to the mean: when they see results that are worse than average, they reprimand the employee, and things get better! Of course, when they see good results, they reward the employee, and things get worse. Managers learn to always be mean, rather than the true lesson that things can’t permanently stay above or below average.

Wheeler presents a few key rules, including that no data have meaning apart from their context, and that to detect a signal, filter out the noise. Much of the book, however, focuses on laying out tools that can help businesses identify meaningful variation. Wheeler swears by the control or XmR chart, a time series combined with a graph showing the size of the change each period. It means you can track both level and variability simultaneously, and look for significant changes in either.

The book is a bit outdated now, but the lessons ring true. Variation was and remains a challenge, and one it is far too easy to neglect.

Working Minds – Beth Crandall, Gary Klein, and Robert Huffman

Cognitive Task Analysis “investigates what people know and how they think.” – Working Minds

Some people just seem to have a certain je-ne-sais-quoi. Nurses who know whether a baby is sick before the lab tests are back: firefighters who know when a building is about to collapse: chess grandmasters who can see the right move without thinking about it.

Whether services or manufacturing, more and more of the modern world relies on the knowledge of experts. Nurses, AI specialists, consultants, and a myriad of other professions all require expertise, and it isn’t always clear what exactly that means or how it can be transferred to others.

Enter Cognitive Task Analysis. The goal is to take an expert task and break it into pieces that can be organized, understood, and replicated. Working Minds is a how-to guide to making that happen: the method draws on concept maps, interviews, and other methods of trying to see into the head of experts and write down what they know and the processes they use, even if they aren’t aware of them themselves.

The book is aimed almost exclusively at practitioners, but the method itself is still interesting. The first half introduces a number of useful tools and ideas, and though probably too specialist for most readers, if this is something you’re interested in cognitive task analysis may be a good place to start. Working Minds, though a bit heavy-going at times, serves as a useful introduction to how to conduct one.

Working Minds gets bogged down in the second half, where it tries to address the potential use of cognitive task analysis. When it spends a chapter explaining that this would be useful to teach others how to do the tasks, for example, it feels like largely a waste of space; the benefits were pretty much self-evident.

What is Government Good At? – Donald J. Savoie

“The requirements of contemporary politics and deficiencies in public administration explain in large measure why government is not as good as it once was or needs to be in several areas if it to enjoy the support of citizens.” – Savoie

Our governments do an enormous amount of work on our behalf, and many of those things they do reasonably well. They are also under tremendous strain. Below the ‘fault line,’ says Savoie, operational employees focus on serving Canadians. Above it, however, political pressure forces civil servants to play the blame game, responding to crises rather than getting things done. The problem, he argues, is that the culture of the public service has focused power and prestige above the fault line. Below it, there are fewer and fewer resources and results.

Donald Savoie is a Canadian expert on public administration, and has strong opinions about how governments are and should be run. Despite the name of the book, its focus is mostly on why governments do things badly. If you make it to the last fifty pages, Savoie highlights that governments are best at tasks with wide public appeal or at least comprehension (such as a war), that are visionary in nature (such as going to the moon), and at dealing with wicked problems (largely because no one else can). Useful points, and it’s a shame he didn’t explore them earlier, rather than retreading arguments he has made in other books.

Savoie has a strong position, but I find some of his statistics hard to interpret. He mentions, for example, that 60% of civil servants are effectively back office, doing policy or other work that doesn’t involve dealing with Canadians directly. He roundly condemns this, but I’m not so sure. Is that good? Bad? A number of his statistics are impressive, but hard to interpret.

The book is on an interesting question, and if the title is a false promise, the content is still interesting. A useful work to read and reflect on, but if you were hoping to learn what government is good at, you’ll be disappointed.

Practical Performance Measurement – Stacey Barr

“ Performance measurement shouldn’t be a post-mortem; it should be a health plan.”

Performance measurement (PM) is oft-maligned: organizations see it as useless or a waste of time when they should focus on results. What many miss is that without clear evidence and data on progress, it can be hard to deliver results, or even know when you’ve delivered them. Performance measurement may well be unnecessary when things are going well, but it’s hard to know if they’re going well or not without it.

Stacey Barr developed what she calls PuMP: the Performance Measurement Process, with an extra ‘u’ thrown in to make it easy to pronounce. In the book, she runs through an 8-step pilot to implementing the process in an organization. The method places heavy emphasis on using PM for improvement, not accountability: if you punish people based on what is measured, you’ll end up with useless measures, she points out.

The book is well organized and useful. A few points needed more defense or explanation: her rejection of brainstorming because not all the ideas it produces are good felt more like a PR move than a well-considered argument, and her espousal of XmR graphs as a way of reporting performance seemed overly strong as well. Since the book’s purpose is partly salesmanship for her course, that may play a role.

The key lesson from the book is engagement. Several of the 8 steps in the pilot are about getting buy-in from the organization and ensuring others feel ownership: without engagement, she argues, performance measures will be ignored at best, and more likely rejected out of hand. I suspect she’s right in that: for any process that is that embedded in an organization—including PM but also HR, IT management, and others—if it remains the project of only a few people it will not lead to the culture change necessary to provide results and maximize impact. A lesson more of us could learn.

How to Make Good Teachers – Economist

Are good teachers made or born? Made, says The Economist and an increasing section of the literature, but the public still tends to look for super-teachers rather than train the ones we have in what works.

Early on in humanity’s lifespan, whether you lived or died of a disease was largely in the hands of the heavens. Some people were born with the ability to intercede with the heavens, and they were recognized and elevated early on for their innate talents. They might have extensive knowledge of the theory of how the gods thought and behaved, as well as how they could be convinced to change their minds.

Fast forward, and medicine is more skills-based than theory-based: theory clearly matters, of course, but doctors learn specific techniques that can help improve outcomes and allow patients to recover. Medical schools focus on teaching the concrete details of how the body works and how to treat disease, rather than abstract philosophies about medicine.

Unfortunately, says The Economist, teaching is still in the first state. The overwhelming narrative in education tends to be about accountability; testing and other methods find out how good teachers are, so we can separate the wheat from the chaff. That may have value, but far more useful is to ensure teachers are teaching well, using skills and practices that can be learned. Unfortunately, far too many education degrees focus on abstract theories of pedagogy rather than actually teaching how to teach.

The data highlights how important it is. The things we fight about in public policy – school uniforms, class size, streaming – make effectively no difference to student outcomes. Good teaching, on the other hand, makes all the difference, with good teachers getting students to learn about three times as much material as poor ones. That is the difference between getting 1.5 years of education or 0.5 years each year.

Good teachers ask probing questions of students; assign short writing tasks to check progress and get children thinking; they plan their classes and how to achieve their goals; have classes that are teacher-led but interactive; anticipate errors; and space out the content they cover. Regardless of your theory of teaching, these methods help students learn, in the same way that regardless of your theory of medicine, some practices will help patients recover.

North America has a lot learn from the places that prioritize education, such as Finland, Japan and Singapore. Their systems aren’t perfect either, but we certainly have room for progress.

You can read the full article here.

Wiser – Cass Sunstein and Reid Hastie

“In this book, we begin with a simple question: Do groups usually correct individual mistakes? Our simple answer is that they do not. Far too often, groups actually amplify those mistakes.”

If you want to find out how an appellate judge will vote about an ideologically connected case, many of us would check if they were appointed by a republican or democrat. That’s a pretty good predictor. But in many areas, there is an even better predictor: who appointed the other two judges on the panel?

Being in a group affects what decisions we take. That much is clear. But should we take a decision alone or with a group? Both methods seem to work in some situations, and not work in others. In Wiser, Sunstein and Hastie set out to explore the difference.

In general, using a group does two things. It increases the cost of making the decision, because all group members have to be consulted and negotiated with: and it reduces the cost of errors, because it means the group can aggregate wisdom and ideas to minimize expensive mistakes. Or at least, that’s the theory.

In practice, groups can help or hinder. At their worst, they can amplify the errors of some of their members, get trapped in groupthink, become more polarized and extreme without become more correct, and focus only on shared information rather than the key information only some members possess. Perhaps the biggest concern is happy talk: group members go along with the consensus or say nothing is wrong, instead of providing new, sometimes critical, information. Good group design—red teams, giving equal voice to members, and other simple methods—can minimize these costs.

At their best, groups can guess how many jelly beans are in a jar. Groups are good when you need to forecast an unknown result, whether a presidential election or the number of jelly beans: they can aggregate opinions and if everyone is slightly off, the final result can often be very close because the average minimizes the individual errors.

Though Wiser discusses these themes at length, it is short on clear lessons, and indeed the book itself can feel repetitive or meandering. It is an interesting idea, and an important subject, but not as well or as clearly explored as I would have liked, making it hard to distill or learn from.

The Orenda – Joseph Boyden

“We had magic before the crows came…And we understood our magic. We understood what the orenda implied.”

The orenda is a spiritual energy present in all natural things—humans, animals, plants, rocks, storms. If a hunter did well, his or her orenda was stronger than that of the game: a shaman had great personal orenda.

The Orenda is the story of three people: a Huron warrior; an Iroquois girl captured by the warrior; and a crow, a French missionary sent by his leaders to the Huron village. As the tale of early interaction between Huron and French is told, each of the three struggles wrestles to adapt and accommodate differences, with the two foreigners (Iroquois and French) each forcing the village to change in response to their presence.

Boyden carefully makes his characters complex: none of them are purely good or pure bad, but instead each has their blind spots and flaws. The Orenda takes events that many Canadians may be broadly familiar with and makes them visceral, giving us characters we can empathize with, even understand. The one odd note for me was the detailed descriptions of torture: though I appreciate he wanted to get historical facts right, I found I largely skipped through those sections, particularly after the first one.

The other note he strikes, one which has been controversial, is the issue of roles. He doesn’t paint the First Nations as solely victims: at one point, the narrator asks “what role did I play in the troubles that surround me?” There is a sequence of back-and-forth throughout the novel, as individuals wrong others and are wronged in turn—sometimes they forgive and grow past it, sometimes not.

I read this book in almost one sitting: I’d highly recommend it, though I might also recommend skipping the torture scenes. It won the 2014 Canada Reads Competition.