The big sort: designing a classification for COVID-19

Design with people. Not for people. Before we are German, Irish or Canadian, we are human. And humans think the same way. Dream the same way. Organize the same way. There are mental maps out there in humanity. We just need to discover them. With the Web, we have the platform and the tools to understand these mental maps.

Here’s the initial hypothesis on how humans think COVID-19 should be organized into classes:
• Symptoms, Diagnosis, Spread
• Mental & Physical Wellbeing
• WHO, Government Guidance, Education, Training
• Research, Statistics
• Vaccine, Immunity, Treatment
• Avoiding Infection
• News
• End Date, New Outbreaks

This came about as a result of over 800 people sorting the top tasks that had emerged from a WHO survey of almost 3,000 individuals, families, academics, students, healthcare providers. We discovered what mattered most to people and then we asked them to sort and organize these things. The more people sorted, the more consistent the classes became. We found tremendous consistency between the mental models of healthcare providers, academics and the general public. They organized things in the same way.

Professionals and the public consistently grouped symptoms with diagnosis. Tasks such as likely course of the illness, incubation period and infectiousness were tightly bound in this group.

Transmission-type tasks like virus survival and mutation were less tightly bound but there was enough evidence of grouping for us to create this class: “Symptoms, Diagnosis, Spread.” Maybe Transmission should be a separate class. We need to test.

We also found that while “Myths, Fake News” was grouped with “News”, it was not strong. There were issues with “Infection hotpots” and “At risk vulnerable”; they didn’t clearly fit anywhere. These are what we call “orphan” tasks. There will always be orphan tasks and there will never be a perfect classification. How you deal with exceptions, orphans and tiny tasks is really important. In too many classifications, the tiny task exceptions have far too much influence on the classification design because tiny tasks are often pet projects of senior managers.

Links and classifications are the absolute foundations of the Web. Too often they are designed in a rapid, ad hoc manner. If there’s one thing you can’t design well and quickly it’s a classification. Unfortunately, too often they are designed by ‘five smart people in a room drinking lattes’ over a couple of workshops with lots of Post-its, with the occasional interjection from a hippo or a seagull that flies in, poops an idea and flies off again.

Good classification design comes from slow cooking. It is evidence-based and rigorously tested. It takes time to do well.

From the sorting of tasks we get a hypothesis that must be tested. There are a couple of reasons. Firstly, you need to test how you have named the classes. Over years of doing this we have found that while people are genius at grouping things, they are often idiot at naming the groups. We got suggested class name like: “Other, A, B, C, D”. We need to use judgement to come up with the names and then test.

There are grey lines between where one group ends and where another begins, whether in fact we have one group or should have two. We need to test, test, test. The hypothetical classification above will go through at least three rounds of testing.

We will do this by creating task instructions such as:
• Can you get infected by COVID-19 through the air?
• If someone has had COVID-19 are they safe from getting it again?

We will have a hypothesis about where we expect people to go and then we will ask them to select which class they would click on to find the answer.

Join Gerry McGovern, WHO, and a worldwide team in discussing the results from Round 1 testing of the WHO COVID-19 top level classification. We will then prepare for the Round 2 test.

2 thoughts on “The big sort: designing a classification for COVID-19

  1. Amanda Charlton

    Gerry, I agree classifying information is an essential step for human centered information management. I did the survey, picking the top 5 from the alphabetical list of 78 items. I think the survey design is at high risk of primacy and recency bias (Choi 2005 p9) resulting in sequence anchoring effect (Peake 1989) or serial position effect (Galesic 2009). These biases would predict respondents are more likely to select items in the first half of the alphabet than the middle or second half. I certainly experienced response fatigue and decision fatigue by the end of 78 items. When I realised this item list was so long, and I would be less likely to consider the items at the end of the alphabet, I switched my viewing from a-z to z-a order. Using the inbuilt computer generated item randomisation present in many survey platforms so that items are presented in a different order for every participant would be one solution for primacy bias, but not for decision fatigue.

    Choi BC, Pak AW. Peer reviewed: a catalog of biases in questionnaires. Preventing chronic disease. 2005 Jan;2(1).
    Peake PK, Cervone D. Sequence anchoring and self-efficacy: Primacy effects in the consideration of possibilities. Social Cognition. 1989 Mar;7(1):31-50.
    Galesic M, Bosnjak M. Effects of questionnaire length on participation and indicators of response quality in a web survey. Public opinion quarterly. 2009 Jan 1;73(2):349-60.

    1. Gerry McGovern Post author

      Thanks for your feedback here, Amanda. We always try to randomize the list. Unfortunately, this survey platform did not allow it. We have taken steps to counter this, in that we are running different versions of the survey with a different order.

      The list is deliberately long in order to get the gut instinct to kick in and force people to select what is most important. Even though it seems counter-intuitive that this approach should work, it does. We’re approaching almost 600 of these surveys by now and we’ve never had one to fail.


Leave a Reply

Your email address will not be published. Required fields are marked *