Scaling Machine Learning with Data Mesh // Shawn Kyzer // Coffee Sessions #116
MLOps Coffee Sessions #116 with Shawn Kyzer, Principal Data Engineer at Thoughtworks (Spain), Scaling Machine Learning with Data Mesh co-hosted by Adam Sroka. // Abstract You can't just get something done by using tools. You need to go much deeper than that and it is very clear that Data Mesh is the same thing. You have to educate the organization about the movement. In this session, Shawn broke down the cultural piece of data mesh and how many parallels there are with the MLOps Movement when it comes to the cultural side of MLOps. // Bio Shawn is passionate about harnessing the power of data strategy, engineering, and analytics in order to help businesses uncover new opportunities. As an innovative technologist with over 13 years of experience, Shawn removes technology as a barrier and broadens the art of the possible for business and product leaders. His holistic view of technology and emphasis on developing and motivating strong engineering talent, with a focus on delivering outcomes whilst minimising outputs, is one of the characteristics which sets him apart from the crowd. Shawn’s deep technical knowledge includes distributed computing, cloud architecture, data science, machine learning, and engineering analytics platforms. He has years of experience working as a consultant practitioner for a variety of prestigious clients ranging from secret clearance level government organizations to Fortune 500 companies. // MLOps Jobs board https://mlops.pallet.xyz/jobs MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://odsc.com/speakers/scaling-machine-learning-with-data-mesh/ https://docs.google.com/presentation/d/1rVtltHkRkP_JaGZdkAS3U_SXfr5Gg-RP980FKXh0YNU/edit?usp=sharing --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Adam on LinkedIn: https://www.linkedin.com/in/aesroka/ Connect with Shawn on LinkedIn: https://www.linkedin.com/in/shawn-kyzer-msit-mba-b5b8a4b/ Timestamps: [00:00] Introduction to Shawn Kyzer [00:43] Takeaways [04:00] Data Mesh for ML projects [11:22] The signal for the exploratory part of a new modeling project [14:13] Ownership and centralization [16:20] Lack of technology and some implementations literature [17:10] Python stronghold from Microsoft blogs [23:09] Integration with self-serve data platform [25:31] Starting a platform team [30:04] Quick wins [32:09] Metrics monitoring [34:18] Metrics break up [38:32] Limit to capabilities and not worth doing [41:39] Culture and technology holds [44:03] Setting the foundation [46:53] Unforeseen benefits [52:19] Lightning question