Union two or more tables together automatically, AgileData Engineering Pattern #2 - Episode #68
Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern of Unioning two or more tables together automatically.
The Automated Table Unioning pattern automatically combines two or more tables by intelligently looking up column names and data types to generate safe SQL under the covers. It supports disparate data sources, such as those from multiple publishers in a data clean room, and creates a view or incrementally loads the unified data into a physical table while tracking load watermarks to prevent duplicates.
An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.
If you want a copy of the pattern template head over to:
https://agiledata.substack.com/i/167481120/pattern-name
Discover more AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns
If you want to join us on the next podcast, get in touch over at https://agiledata.io/podcasts/#contact
Or if you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.
Subscribe: Apple Podcast | Spotify | Google Podcast | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser | Deezer | Podcast Addict |
Buy the Green Book now!
Simply Magical Data
Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern of Orchestrating Dynamic Data Flows.
The Dynamic Data Flow Orchestration pattern dynamically generates and self-heals data flow manifests (DAGs) at runtime from a central context database, enabling adaptive, cost-effective, and low-latency data processing primarily for micro-batching workloads
An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.
If you want a copy of the pattern template head over to:
https://agiledata.substack.com/i/167144707/pattern-name
Discover more AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns
If you want to join us on the next podcast, get in touch over at https://agiledata.io/podcasts/#contact
Or if you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.
Subscribe: Apple Podcast | Spotify | Google Podcast | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser | Deezer | Podcast Addict |
Buy the Green Book now!
Simply Magical Data
--------
22:19
--------
22:19
Data Engine Thinking Patterns with Roelant Vos - Episode #66
Join Shane Gibson as he chats with Roelant Vos about a number of Data Engine Thinking patterns and his new book Data Engine Thinking.
You can get in touch with Roelant via LinkedIn or over at https://roelantvos.com/blog/
Or you can grab a copy of the Data Engine Thinking book on Amazon or at https://dataenginethinking.com/en/
If you want to download the transcript for the podcast, head over to:
https://agiledata.io/podcast/agiledata-podcast/data-engine-thinking-patterns-with-roelant-vos/#read
If you want want to read a summary generated with GenAI, head over to:
https://agiledata.substack.com/p/data-engine-thinking-patterns-with
Listen to more podcasts on applying AgileData patterns over at https://agiledata.io/podcasts/
Read more on the Agile Data Way of Working over at https://AgileDataGuides.com/
If you want to join us on the next podcast, get in touch over at https://agiledata.io/podcasts/#contact
Or if you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.
Subscribe: Apple Podcast | Spotify | Google Podcast | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser | Deezer | Podcast Addict |
Buy the Green Book now!
Simply Magical Data
--------
52:03
--------
52:03
Data Engineering Patterns with Chris Gambill - Episode #65
Join Shane Gibson as he chats with Chris Gambill about a number of Data Engineering patterns.
You can get in touch with Chris via LinkedIn or view his YouTube channel https://www.youtube.com/@GambillDataEngineering
If you want to download the transcript for the podcast, head over to:
https://agiledata.io/podcast/agiledata-podcast/data-engineering-patterns-with-chris-gambill/#read
If you want want to read a summary generated with GenAI, head over to:
https://agiledata.substack.com/p/data-engineering-patterns-with-chris
Listen to more podcasts on applying AgileData patterns over at https://agiledata.io/podcasts/
Read more on the Agile Data Way of Working over at https://AgileDataGuides.com/
If you want to join us on the next podcast, get in touch over at https://agiledata.io/podcasts/#contact
Or if you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.
Subscribe: Apple Podcast | Spotify | Google Podcast | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser | Deezer | Podcast Addict |
Buy the Green Book now!
Simply Magical Data
--------
1:04:28
--------
1:04:28
The pattern of writing a data book with Shane Gibson and Ramona C Truta - Episode #64
Join Ramona C Truta as she interviews Shane Gibson about the patterns of writing a data book (and they discuss a whole lot of other data things)
You can get in touch with Ramona via LinkedIn or read more on substack https://ramonatruta.substack.com/
Or buy a copy of Shanes book "an Agile Data Guide to Information Product Canvas", and finally fix those data requirement gathering problems.
If you want to download the transcript for the podcast, head over to:
https://agiledata.io/podcast/agiledata-podcast/the-pattern-of-writing-a-data-book-with-shane-gibson-and-ramona-c-truta/#read
If you want want to read a summary generated with GenAI, head over to:
https://agiledata.substack.com/p/the-pattern-of-writing-a-data-book
Listen to more podcasts on applying AgileData patterns over at https://agiledata.io/podcasts/
Read more on the Agile Data Way of Working over at https://AgileDataGuides.com/
If you want to join us on the next podcast, get in touch over at https://agiledata.io/podcasts/#contact
Or if you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.
Subscribe: Apple Podcast | Spotify | Google Podcast | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser | Deezer | Podcast Addict |
Buy the Green Book now!
Simply Magical Data
Sverige