We're modelling data as part of data product development, so it follows a product development, maintenance and enhancement lifecycle, rather than being project based. So the operating model for the data product considers who needs to do what when the data changes, when the data model changes, when there's a quality issue with the source data, the pipelines, and the data model, all of it. So I totally agree, data modelling is never one and done.
We take a very similar approach. What we do struggle with are the product boundaries, loosely, what in microservices world is called bounded context.
How do you deal with "my view of their entities" vs "their view of my entities"? This can get mutated super fast unless the org as a whole is speaking that language. Been a recurring theme throughout my journey. Very interested to hear what others think.
Effective practices for shepherding enterprise data are going to remain a dream until their funding is made part of the capital budget. Just like a large factory, the data assets of an enterprise are expensive to create and costly to maintain. Even if the executable code that uses the data is completely replaced, the data will be carried forward, often at significant risk and cost.
Although the value of well-managed data receives frequent and sincere lip service, the mechanism for actually committing the funding required remains mired as a budgeted item, lumped in with all the other data processing cost projections and subject to the same semi-arbitrary cost control variabilities.
In working with many of my clients, I have advocated for data quality and security programs to be managed in the capital budget, where on-going investment is a strategic action. My arguments have yet to succeed. A key contribution to my unbroken record of non-success is the issue of data ownership, which is ceded to multiple enterprise partitions rather than to the enterprise as a whole.
I work in content architecture and write a lot about content modeling. It's very much parallel to what you write about with data modeling, and in fact, I've cited you in my "Model Thinking" newsletter.
There's also a related idea that my manager and I talked about years ago that I've seen mentioned a few places online: Architectural work doesn't do well in sprints. (I chafed at putting content modeling work in sprints.)
For knowledge modeling, the same is true. Design thinking matches modeling work, be it content, taxonomies and ontologies or data modeling.
When I worked as an Information Architect at Amazon, I was also a program manager as my work transcended boundaries and involved accounting for multiple realities.
Super curious to watch how Ellie evolves. Data modeling often happens in silos, artifacts are not widely shared or collaborated on, and it’s disconnected from analysts and business users. I’ve been thinking a lot about how data catalogs can bridge this gap between systems and users and concepts.
But the problem is catalogs are so distant from dev workflows. I think the right data modeling tool could bring the dev workflow closer to these other users. Love seeing they have a really solid sync with Collibra, will be interested to see if that expands to others.
This reframing is nice. As I’ve begun to understand the importance of thinking through the data model of a project first, sometimes I’m stuck in analysis paralysis. Trying to find the perfect model. Keeping in mind that it’s a program that grows, will help me get it to where it needs to be instead of sitting around and thinking just to think.
Knowledge management is 100% about program-sustained and constant. Not a one off, much like data modeling which is a form of process knowledge.
We're modelling data as part of data product development, so it follows a product development, maintenance and enhancement lifecycle, rather than being project based. So the operating model for the data product considers who needs to do what when the data changes, when the data model changes, when there's a quality issue with the source data, the pipelines, and the data model, all of it. So I totally agree, data modelling is never one and done.
This is the way! "Injecting" data modeling into everyday product workflows, and ensuring that the work done at that level feeds a wider enterprise-level understanding of what is where, is how a living and breathing data model can be realized. I wrote something related just yesterday: https://open.substack.com/pub/commonsensedata/p/conceptual-modeling-thinking-beyond?utm_source=share&utm_medium=android&r=38edu3
We take a very similar approach. What we do struggle with are the product boundaries, loosely, what in microservices world is called bounded context.
How do you deal with "my view of their entities" vs "their view of my entities"? This can get mutated super fast unless the org as a whole is speaking that language. Been a recurring theme throughout my journey. Very interested to hear what others think.
Joe -
Effective practices for shepherding enterprise data are going to remain a dream until their funding is made part of the capital budget. Just like a large factory, the data assets of an enterprise are expensive to create and costly to maintain. Even if the executable code that uses the data is completely replaced, the data will be carried forward, often at significant risk and cost.
Although the value of well-managed data receives frequent and sincere lip service, the mechanism for actually committing the funding required remains mired as a budgeted item, lumped in with all the other data processing cost projections and subject to the same semi-arbitrary cost control variabilities.
In working with many of my clients, I have advocated for data quality and security programs to be managed in the capital budget, where on-going investment is a strategic action. My arguments have yet to succeed. A key contribution to my unbroken record of non-success is the issue of data ownership, which is ceded to multiple enterprise partitions rather than to the enterprise as a whole.
I work in content architecture and write a lot about content modeling. It's very much parallel to what you write about with data modeling, and in fact, I've cited you in my "Model Thinking" newsletter.
I 100% agree that modeling is a program. I touched briefly on this same concept about 10 months ago in https://newsletter.collinscontent.com/posts/the-granularity-balance-pocket-approvals-and-specialist-v-generalist as I was finding my own newsletter voice.
There's also a related idea that my manager and I talked about years ago that I've seen mentioned a few places online: Architectural work doesn't do well in sprints. (I chafed at putting content modeling work in sprints.)
For knowledge modeling, the same is true. Design thinking matches modeling work, be it content, taxonomies and ontologies or data modeling.
When I worked as an Information Architect at Amazon, I was also a program manager as my work transcended boundaries and involved accounting for multiple realities.
Super curious to watch how Ellie evolves. Data modeling often happens in silos, artifacts are not widely shared or collaborated on, and it’s disconnected from analysts and business users. I’ve been thinking a lot about how data catalogs can bridge this gap between systems and users and concepts.
But the problem is catalogs are so distant from dev workflows. I think the right data modeling tool could bring the dev workflow closer to these other users. Love seeing they have a really solid sync with Collibra, will be interested to see if that expands to others.
Similar to the Netherlands, a US government program may require multiyear deliverables from its supplier for the contract’s period of performance.
For example, the supplier provides the deliverable to the government every month for the 20-year contract.
Thus, this could be an example of a US company that must treat data modeling as a continuous, multi-decade program.
This reframing is nice. As I’ve begun to understand the importance of thinking through the data model of a project first, sometimes I’m stuck in analysis paralysis. Trying to find the perfect model. Keeping in mind that it’s a program that grows, will help me get it to where it needs to be instead of sitting around and thinking just to think.