home -> thoughts -> Goodharting is the Signal

Goodharting is the Signal

​ It is supposedly common practice among wealthy parents to add their children as co-owners on lines of credit from a young age, in order for them to begin their financial life with a strong credit score backed by a decade of strong history. On the face of it, this seems like an odd loophole that has somehow avoided a shutdown by the ratings agencies - few toddlers are able to manage credit lines. Viewed instead as a voluntary transaction of signals, it becomes clear. FICO knows that the kind of parents who understand credit ratings and how to goodhart them will both set their children up with a good understanding of finance and wealth-building, as well as provide a solid base of collateral should their child get in over their head. Those who game the system are the ones who understand it.

​ A similar relationship appears to exist in the hiring pipelines for software engineering positions. Leetcode problems lost much of their signal as a direct proxy for engineering ability long ago, as incredible infrastructure was created for the purpose of training for the problems you are likely to see on a technical interview. And yet, while this class of problems are in decline, they are still widely used. This is possible because the first-order signal of problem solving ability was replaced with the second-order signal of task optimizing ability. 90% of these problems can be solved through reduction by ~10 core algorithms, and so once the skill of reduction is honed and the algorithms committed to memory, the task is solved. This meta-level challenge of recognizing and solving the underlying task is a good proxy for engineering ability, perhaps even more than being able to solve an isolated coding puzzle.

(Disclaimer: I have no data for this, and believe that I would be a hard thing to prove. I’m also unsure if this is explicitly understood by hiring managers, as the path of causality of the signal is irrelevant as long as it still holds value, i.e. good at leetcode => good at engineering)