I still remember The Decoder episode where Nilay Patel was interviewing the CEO of Patreon and he talked about how the adage that "I'll know it when I see it" about porn is actually totally wrong. There are reams and reams of specific documentation detailing exactly what should and shouldn't be considered pornographic from a policy perspective.

And yet it's a concept and idea that has so found so much purchase in our popular culture. The idea that Taste isn't a skill, it's a perspective on the world of something that you'll "know when you see". It's the idea that the reams of knowledge and understanding that have been built up over the decades is worth something, but it's also not something that you can explain or share with any other human being.

It's not all that dissimilar to the way that challenging conversations end with "it depends"! Like we've all discovered some hidden trait of the universe that details are tricky to suss out. Those details are where the hard work begins, not where it ends. And our desire to stop there doesn't reflect an impossibility to understand the world, it reflects our own desire to not push too hard and to not do the work to win small and important nuggets of wisdom.

"Taste" is total fucking bullshit.

Tacit knowledge is real. Tacit knowledge that's the accumulation of experience is real. It's not taste though. And by that I mean, the key differentiator between "having taste" and "not having taste" actually has nothing to do with "good" (whatever that's supposed to mean). I think what you're actually seeing is something much closer to a prediction match or a prediction mismatch.

If you think about a skill as a set of competencies that jumps you from an idea to a goal and an LLM as a device that squashes portions of that goal into a single step, what you're really doing is entering some text that you predict will produce a given output. Your ability to both predict the output, and understand the connection between that output and the next prompt (which is the accumulation of experience and probably your own disposition to the skill of operating an LLM) is "Taste".

The primary fail state I see in engineers (junior, AI-assisted, or otherwise) is an inability to visualize the outline of an end state before starting, and the inability to break work down into a coherent number of steps to get there. What "Taste" is actually doing is post-hoc rationalization. Was the step I took in that iteration towards my goal, or away from it? Were there things that jumped out that feel like they could spin out into chaotic pathways over time? In order to make these analyses you do have to have genuine experience in the field. But it's not "taste" and it's not subjective, at least not insofar as taste is a subjective matter.

The reason this matters, in my opinion, is that it suggests there are probably additional layers of abstraction to be wrung out of the problem solving process. Theoretically it would be possible to grade the output of any given step from an LLM (either within the process itself or within a meta structure like a skill), and then work your way in and out towards a solution. I don't think that abstraction would take away the need for goal setting, but it would probably eliminate many of the things that we currently call "taste" (derogatory) without actually eliminating real taste (predictive alignment with social outcomes).

Goal selection is still really difficult and insight is hard to derive.