Brilliant article! While reading, I wondered if it might also be because data modeling requires a methodical approach. I am very much looking forward to your book! I assume that different types of data modeling require different approaches regarding the way of working. In my work with methods, I have only seen two camps: those who smile at it and those who have completely mastered it. A methodical approach is often ridiculed, especially by managers who prefer to act spontaneously based on gut feeling. For data engineers, it is, of course, very difficult to push for data modeling to be given the time it needs. I think the combination with proper training in stakeholder management (though not the boring kind found in textbooks) could be helpful.
This resonates because I see the same problem in agent systems at scale. You build a nice schema. Agent queries it. Gets technically correct but meaningfully wrong answers.
The agent sees that active_customer table and confidently joins it to orders. But it doesn't know Marketing defines active as logged in last 30 days and Finance defines it as currently paying. Your schema supports both definitions silently.
When you add an agent on top of data where nobody agreed on what things mean, you get a confident system making wrong decisions. I've watched this happen multiple times.
The validation layer has to come first. Before the agent touches data.
That's where your real model lives. Not in the schema.
Sadly, this is how I feel a lot of people learn their data practices: https://www.youtube.com/watch?v=u0acW03inho
This should be mandatory reading for non-technical RevOps leaders. IYKYK
Brilliant article! While reading, I wondered if it might also be because data modeling requires a methodical approach. I am very much looking forward to your book! I assume that different types of data modeling require different approaches regarding the way of working. In my work with methods, I have only seen two camps: those who smile at it and those who have completely mastered it. A methodical approach is often ridiculed, especially by managers who prefer to act spontaneously based on gut feeling. For data engineers, it is, of course, very difficult to push for data modeling to be given the time it needs. I think the combination with proper training in stakeholder management (though not the boring kind found in textbooks) could be helpful.
A different approaches, same mindset
This resonates because I see the same problem in agent systems at scale. You build a nice schema. Agent queries it. Gets technically correct but meaningfully wrong answers.
The agent sees that active_customer table and confidently joins it to orders. But it doesn't know Marketing defines active as logged in last 30 days and Finance defines it as currently paying. Your schema supports both definitions silently.
When you add an agent on top of data where nobody agreed on what things mean, you get a confident system making wrong decisions. I've watched this happen multiple times.
The validation layer has to come first. Before the agent touches data.
That's where your real model lives. Not in the schema.
The best book I ever read on data modelling was William Kent ‘Data and Reality’; published nearly 50 years ago!
I had forgotten the debates in the conference table.
It’s been SO LONG I forgot this used to be part of the job.
One of the biggest parts (that and trying to get the damn campaign passed legal and compliance — banking, man).
I had forgotten the importance of all that work.
Those discussions shaped everything (including power, relationships).
I think a lot about optimization from other perspectives (consumer, brand) but this texture got sanded down, too.
Is the book available for preorder. I think I’m going to love it.
YES, a thousand times yes!!!