Today’s Morning Buzz is brought to you by Raman Shah, an independent data scientist based in Providence, RI. Connect with Raman on LinkedIn or X/Twitter.
- What I’m reading: “Generalized Additive Models“ by Simon Wood
- A hobby I enjoy: Perfumery
- Who my pets are: Two new-to-me middle-aged lady cats: Foxy the orange tabby and Cupcake the tuxedo
“Additional duties as assigned.”
Sprinkled into job descriptions everywhere, these four words generate laughter, terror, and cool war stories whenever I’m around my friends in local gov. The sheer diversity of the work of public service makes it interesting and fun. It also demands a certain handiness, a collection of weird little skills taught in the wrong parts of school.
Sorting stuff efficiently is a jewel among these weird little skills. It’s a game-changer for anyone who needs to organize physical items. To sort 150 things by hand — a typical additional duty as assigned — goes six times faster with the right approach. I learned this skill early in my path as a data scientist in a computer science class called Data Structures and Algorithms. But this skill is extremely rare among the non-programmers who need it the most. Let’s change that right now.
Recognizing the value of sorting
Sorting stuff is essential. To find something in an unsorted pile of 1,000 items takes, on average, 500 tries. To find it in a sorted pile takes no more than 10 tries as we check the middle of the pile, learn which half of the pile the item is in, then check the middle of that half, halving the work at each step.
Whenever possible, we sort stuff by asking a computer to do it for us, trusting a programmer who took the course. One of my three tiny Excel skills for resilient management of periodic responsibilities is, in fact, the shortcut key to ask Excel to sort stuff.
But additional duties as assigned are often physical: the shoebox full of name tags to lay out for the public meeting you’re running. The mess of century-old records in the basement of Town Hall that your town is, by charter, obligated to look after. During my Ph.D., I taught physical chemistry lab to undergraduates. It would have been a nightmare to hand 150 graded lab reports back to students if I hadn’t alphabetized them beforehand.
Merging two pre-sorted piles of stuff
The algorithm I describe here, a flavor of merge sort, uses the insight that it’s easy to merge two pre-sorted piles of stuff into one sorted pile. It’s easy because we only need to look at the front of each pile as we shuffle them together.
For the sake of illustration, I alphabetized my spice rack. By comparing what’s at the front of two rows of four pre-sorted spices as shown in the green boxes, we can quickly build a sorted row of eight spices. Ajika goes before dill weed:
Then dill weed goes before paprika:
Comparing the front of each of the two rows, one by one, a sorted row of eight spices comes together easily. Eventually, one of the rows of four is used up. The vanilla and vindaloo remaining in the other row go directly onto the end of our sorted row of eight:
Sorting stuff from start to finish
We can efficiently sort any number of things by applying this tactic repeatedly:
- Grab four items, or all of the rest if there are fewer than four left. Sort them however you want. In my experience, humans can sort four things at a glance but start to bog down much beyond that.
- As soon as two sorted groups of four items are available, merge them into eight. As soon as two sorted groups of eight are available, merge them into 16. And so on.
- Once you’re out of new items to grab, merge the remaining pre-sorted groups down into a single, fully sorted collection.
To illustrate, here’s a collection of 18 spices partway through this process.
There’s a sorted row of eight spices and a sorted row of four below it. We grab four more and sort them:
We merge 4+4=8:
We merge 8+8=16:
All that’s left is to merge the last two spices into the row of 16, and the job is done.
Sorting stuff by winging it is harder in a way that gets increasingly out of hand as the job grows. I wrote some simulation code to put numbers to this fact. To sort 18 spices is a tiny job, and it takes only about 1.5 times as much work to wing it as to use merge sort as described above. In real life, I have 45 spices and winging it would have taken about 2.7 times as much work as the merge sort I used. That’s noticeable. To sort 150 resident nametags, basement records, or lab reports, winging it takes about six times as much work. An efficient sorting algorithm at this point is the difference between a modest 30-minute chore and half a day of brain-melting tedium. In practice, without merge sort, stuff is usually left disorganized, ruining someone’s day downstream.
Amazing your colleagues
There’s pride and joy in being handy. It allows us to better serve our neighbors and inspire the people around us to do the same. Being able to sort stuff efficiently with merge sort is a great little superpower to keep around. It’s rare enough and useful enough that you’ll probably find yourself unexpectedly saving the day and amazing your colleagues.
“Additional duties as assigned?” Let’s go.