What kind of sceptic I am (2m36s)

Mike Hall opens by disambiguating sceptic. Not a climate denier, not an anti-vaxxer — a scientific sceptic, in the Merseyside Sceptics Society sense, where the practice is to question claims that lack good evidence rather than to refuse to believe anything established.

"scepticism in the sense we use it refers to the principles of scientific scepticism — the practice of questioning claims which lack good evidence"

"it seems like a really bad idea to accept things as true when there's no good [evidence]"

"our brains are really, really buggy, and they will accept any old nonsense if you present that information in the correct way"

The Thatcher illusion and the bugs in our brains (3m41s)

The 1980 Thatcher illusion demonstrates a structural bug in how our brains do facial recognition. When a photo of Thatcher is upside down with her eyes and mouth inverted, the result looks roughly normal; flip the photo right-side-up and it's grotesque. Magic tricks are fun bugs; cognitive biases are the bugs that matter.

"as software engineers... we spend an awful lot of our time debugging — but we don't always turn that rational, stepwise examination back on ourselves"

"we often fail to recognise that our brains are very flawed and error-prone"

"these cognitive biases are systematic errors in the way that we think"

Confirmation bias and works on my machine (6m46s)

Confirmation bias is the tendency to accept evidence that affirms what we already believe (Arthur Conan Doyle and the Cottingley fairies). In software it shows up as unit tests written to confirm the code works rather than to try to break it, and as the phrase "works on my machine" that sidelines the customer's evidence in favour of the engineer's prior belief.

"we will miss bugs that we don't expect, because our own bias about how we think our code works stops us even looking for those kinds of bugs"

"this phrase drives me up the wall, and I quite often will ban it from teams that I'm working in — works on my machine"

"it prevents us from seeing and addressing flaws in our code"

Appeal to popularity: fashion-based development (10m20s)

50,000,000 Elvis fans can't be wrong is the canonical bad argument. Mr Blobby's records sold well too. In software it shows up as the default stack — Postgres, Node, TypeScript, AWS, microservices, Kubernetes — chosen not because it fits the problem but because it's the popular thing.

"this is what I refer to as fashion-based development"

"popularity is a bad reason to choose a stack — we should be asking: does this solve a problem that we actually have?"

"popularity is the only consideration that is on the table when choosing a tool"

Anchoring in sprint planning and code review (12m54s)

Anchoring is when the first number sets the conversation. The medium popcorn in a cinema exists to make the large look reasonable; Steve Jobs anchored the iPhone's $500 price by valuing an iPod plus a phone at $500 before mentioning the iPhone itself. In sprint planning, I think it's five points makes it harder for anyone to say I think it's thirty; in code review, this is great makes this sucks harder to follow with.

"the medium popcorn isn't really there for you to buy — it's there to make the price of the large popcorn seem more reasonable"

"we're most likely to encounter anchoring as software engineers in things like sprint planning"

"often we can anchor ourselves around that initial idea of 'I think it's this', and we spend all of our time chasing down that initial idea"

Survivorship bias and the Google-talk Q&A (17m30s)

The 1943 US Air Force bomber diagram showing where surviving planes were hit is the canonical example — reinforce where they're being shot is the wrong reading; planes shot in the engine never came back, which is why no dots are there. In software, let's use the same stack Facebook uses is the same fallacy. How did you get into Google? is asking what lottery numbers did you play to win?

"the reason the damage is only in these areas is because these are the areas the plane can take damage and still return to base"

"let's use the technology Facebook used... we know this is a path to success"

"what numbers did you play to win the lottery? Because then I'll play those numbers and then I'll win the lottery as well"

Critical thinking as the brain's debugger (21m37s)

Knowing about biases doesn't make you immune — they're hard-wired. What you can do is manage them with critical thinking: question assumptions (a true one will survive examination), check your sources, prioritise evidence over opinion, practise open-mindedness, change your mind when the evidence does. Treat every day like April Fools' Day is the rough heuristic.

"critical thinking is the debugger for your brain"

"we are very, very good at [scepticism] on April 1st, and we are terrible at that for the whole of the rest of the year"

"a true assumption will hold up under critical examination — the truth has nothing to fear from critical examination"

The W3Schools colorDepth bug (27m14s)

The case study: payments failed on Mac Chrome with an external monitor attached. The cause was that Apple's wide-colour-gamut displays report colorDepth: 30, but the 3DSecure 2.0 spec only accepts a fixed enum of 1, 4, 8, 15, 16, 24, 32 or 48 — which was lifted from a W3Schools page edit in 2013. W3Schools have no authority over web standards. They had spectacular SEO and one editor took the values as gospel and baked them into a payment-card specification.

"W3Schools, despite their name, is not affiliated with the W3C — they have no authority whatsoever on web standards. They are a very well-meaning, very small organisation based out of Norway, that have spectacular SEO somehow"

"somebody at EMVCo, who writes the 3DSecure 2.0 specification, took this information as gospel and baked it into the 3DSecure 2.0 standard"

"this is why you should check your sources — evaluate your sources and the quality of information that you receive"

Question one assumption next week (32m53s)

The closing challenge: pick one assumption your team has been operating under, test it, evaluate the evidence, and see what changes. The goal of software engineering isn't to ship code — it's to solve problems robustly. A culture of questioning and curiosity gets you there faster than power-typing does.

"our goal is not always to ship code — our goal is to solve problems in a robust and adaptable manner"

"fostering a culture of questioning and curiosity within our teams"

"go back, evaluate: is that assumption true? Is this the correct predicate that we have built our code on?"