2827: Data, Decisions, and Dagster: Nick Schrock's Blueprint for Engineering Excellence

Nick Schrock, the innovative mind behind Dagster Labs and the renowned co-creator of GraphQL, joins me on Tech Talks Daily. Nick takes us through his illustrious journey from his foundational days at Facebook, where he spearheaded the Product Infrastructure team, to his visionary leap into solving some of the most pressing issues facing data and ML engineering today through Dagster, his open-source data orchestration platform. Nick shares insights from his experience at Facebook, elaborating on how internal tools like React and GraphQL revolutionized the company's development practices and set new benchmarks for the developer community worldwide. His transition from Facebook to founding Dagster Labs was driven by a deep-seated desire to address the complexities and inefficiencies in data infrastructure, a challenge he identified as a critical pain point for engineers across industries. Throughout the conversation, Nick delves into the core areas of data orchestration, highlighting the importance of enabling practitioners to have end-to-end ownership of data pipelines without the need for a centralized team. This approach, he argues, is pivotal in the era where data and ML engineering are becoming fundamental to decision-making processes both in human and business contexts. Much of the discussion is dedicated to exploring the future of open source in the SaaS-dominated landscape and the operational convergence of ML, AI, and data engineering. Nick emphasizes the delicate balance required in managing an open-core business model and shares personal anecdotes about the "Engineering Founder's Dilemma" — the intricate dance between leading the vision and running the company. Listeners will gain a unique perspective on the evolution of data platforms and engineering, underscored by Nick's advocacy for a robust, community-driven approach to open-source development. He sheds light on the challenges and rewards of building a platform like Dragster, which aims to simplify and democratize data infrastructure for companies of all sizes. Nick also advises technical founders on maintaining equilibrium between their visionary roles and their companies' operational demands. This episode is a deep dive into the mechanics of data orchestration and a masterclass in leadership, innovation, and the transformative power of open-source projects in addressing complex engineering challenges.  

Om Podcasten

The speed of technological change continues to race ahead at breakneck speed, but it will never move this slowly again. In 2024, businesses and their employees will be expected to get to grips with AI, Machine Learning, Quantum Computing, Extended Reality, Cloud Computing, Edge Computing, Sustainable Tech, Green Computing, Cybersecurity, 5G, Blockchain, Crypto, IoT, Autonomous Everything, Human Augmentation, Data Sovereignty, Privacy, AI ethics and the Digital Transformation of everything. Navigating these uncharted digital waters can feel daunting and overwhelming for business leaders, IT teams, and employees in every sector. Especially when scrolling down newsfeeds full of hype, buzzwords, and confusing technical jargon/acronyms. Does this sound familiar? Welcome to Tech Talks Daily, a podcast uniting business and technology. Tech Journalist Neil C. Hughes brings you the latest insights into business technology, digital transformation, and the evolving business landscape and IT. This technology podcast focuses on real-world applications and business strategies around digital transformation. From AI's impact on business operations to navigating the complexities of cloud computing and cybersecurity, Neil explores how technology reshapes industries and redefines personal and business success in 2024. The lineup of guests includes tech leaders, CEOs, entrepreneurs, visionaries, and even the occasional celebrity who has walked the path of tech innovation. High-profile guests share their startup stories, their journey through digital transformation, and how technology has revolutionized their businesses and lives. Whether you want to understand how to harness technology for business growth or stay informed, Tech Talks Daily is your daily dose of inspiration and insight. Join the conversation and be part of a community shaping the future of business and technology.