DataTalks.Club

• Trends in Data Engineering – Adrian Brudaru

  In this podcast episode, we talked with Adrian Brudaru about ​the past, present and future of data engineering.About the speaker:Adrian Brudaru studied economics in Romania but soon got bored with how creative the industry was, and chose to go instead for the more factual side. He ended up in Berlin at the age of 25 and started a role as a business analyst. At the age of 30, he had enough of startups and decided to join a corporation, but quickly found out that it did not provide the challenge he wanted.As going back to startups was not a desirable option either, he decided to postpone his decision by taking freelance work and has never looked back since. Five years later, he co-founded a company in the data space to try new things. This company is also looking to release open source tools to help democratize data engineering.0:00 Introduction to DataTalks.Club1:05 Discussing trends in data engineering with Adrian2:03 Adrian's background and journey into data engineering5:04 Growth and updates on Adrian's company, DLT Hub9:05 Challenges and specialization in data engineering today13:00 Opportunities for data engineers entering the field15:00 The "Modern Data Stack" and its evolution17:25 Emerging trends: AI integration and Iceberg technology27:40 DuckDB and the emergence of portable, cost-effective data stacks32:14 The rise and impact of dbt in data engineering34:08 Alternatives to dbt: SQLMesh and others35:25 Workflow orchestration tools: Airflow, Dagster, Prefect, and GitHub Actions37:20 Audience questions: Career focus in data roles and AI engineering overlaps39:00 The role of semantics in data and AI workflows41:11 Focusing on learning concepts over tools when entering the field 45:15 Transitioning from backend to data engineering: challenges and opportunities 47:48 Current state of the data engineering job market in Europe and beyond 49:05 Introduction to Apache Iceberg, Delta, and Hudi file formats 50:40 Suitability of these formats for batch and streaming workloads 52:29 Tools for streaming: Kafka, SQS, and related trends 58:07 Building AI agents and enabling intelligent data applications 59:09Closing discussion on the place of tools like DBT in the ecosystem🔗 CONNECT WITH ADRIAN BRUDARULinkedin -  / data-team   Website - https://adrian.brudaru.com/ 🔗 CONNECT WITH DataTalksClubJoin the community - https://datatalks.club/slack.html Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/... Check other upcoming events - https://lu.ma/dtc-events LinkedIn -  /datatalks-club   Twitter -  /datatalksclub   Website - https://datatalks.club/

  --------  

  56:59
• Competitive Machine Leaning And Teaching – Alexander Guschin

  In this podcast episode, we talked with Alexander Guschin about launching a career off Kaggle.About the Speaker: Alexander Guschin is a Machine Learning Engineer with 10+ years of experience, a Kaggle Grandmaster ranked 5th globally, and a teacher to 100K+ students. He leads DS and SE teams and contributes to open-source ML tools.0:00 Starting with Machine Learning: Challenges and Early Steps 13:05 Community and Learning Through Kaggle Sessions 17:10 Broadening Skills Through Kaggle Participation 18:54 Early Competitions and Lessons Learned 21:10 Transitioning to Simpler Solutions Over Time 23:51 Benefits of Kaggle for Starting a Career in Machine Learning 29:08 Teamwork vs. Solo Participation in Competitions 31:14 Schoolchildren in AI Competitions42:33 Transition to Industry and MLOps50:13 Encouraging teamwork in student projects50:48 Designing competitive machine learning tasks52:22 Leaderboard types for tracking performance53:44 Managing small-scale university classes54:17 Experience with Coursera and online teaching59:40 Convincing managers about Kaggle's value61:38 Secrets of Kaggle competition success63:11 Generative AI's impact on competitive ML65:13 Evolution of automated ML solutions66:22 Reflecting on competitive data science experience🔗 CONNECT WITH ALEXANDER GUSCHINLinkedin - https://www.linkedin.com/in/1aguschin/Website - https://www.aguschin.com/🔗 CONNECT WITH DataTalksClubJoin DataTalks.Club:⁠⁠⁠⁠https://datatalks.club/slack.html⁠⁠⁠⁠Our events:⁠⁠⁠⁠https://datatalks.club/events.html⁠⁠⁠⁠Datalike Substack -⁠⁠⁠⁠https://datalike.substack.com/⁠⁠⁠⁠LinkedIn:⁠⁠⁠⁠  / datatalks-club  ⁠

  --------  

  53:27
• Redefining AI Infrastructure: Open-Source, Chips, and the Future Beyond Kubernetes – Andrey Cheptsov

  In this podcast episode, we talked with Andrey Cheptsov about ​The future of AI infrastructure.About the Speaker:Andrey Cheptsov is the founder and CEO of dstack, an open-source alternative to Kubernetes and Slurm, built to simplify the orchestration of AI infrastructure. Before dstack, Andrey worked at JetBrains for over a decade helping different teams make the best developer tools.During the event, the guest, Andrey Cheptsov, founder and CEO of dstack, discussed the complexities of AI infrastructure. We explore topics like the challenges of using Kubernetes for AI workloads, the need to rethink container orchestration, and the future of hybrid and cloud-only infrastructures. Andrey also shares insights into the role of on-premise and bare-metal solutions, edge computing, and federated learning.00:00 Andrey's Career Journey: From JetBrains to DStack5:00 The Motivation Behind DStack7:00 Challenges in Machine Learning Infrastructure10:00 Transitioning from Cloud to On-Prem Solutions14:30 Reflections on OpenAI's Evolution17:30 Open Source vs Proprietary Models: A Balanced Perspective21:01 Monolithic vs. Decentralized AI businesses22:05 The role of privacy and control in AI for industries like banking and healthcare30:00 Challenges in training large AI models: GPUs and distributed systems37:03 DeepSpeed's efficient training approach vs. brute force methods39:00 Challenges for small and medium businesses: hosting and fine-tuning models47:01 Managing Kubernetes challenges for AI teams52:00 Hybrid vs. cloud-only infrastructure56:03 On-premise vs. bare-metal solutions58:05 Exploring edge computing and its challenges🔗 CONNECT WITH ANDREY CHEPTSOVTwitter -  / andrey_cheptsov  Linkedin -  / andrey-cheptsov  GitHub - https://github.com/dstackai/dstack/Website - https://dstack.ai/🔗 CONNECT WITH DataTalksClubJoin DataTalks.Club:⁠⁠⁠https://datatalks.club/slack.html⁠⁠⁠Our events:⁠⁠⁠https://datatalks.club/events.html⁠⁠⁠Datalike Substack -⁠⁠⁠https://datalike.substack.com/⁠⁠⁠LinkedIn:⁠⁠⁠  / datatalks-club  ⁠

  --------  

  56:55
• Linguistics and Fairness - Tamara Atanasoska

  In this podcast episode, we talked with Tamara Atanasoska about ​building fair AI systems.About the Speaker:​Tamara works on ML explainability, interpretability and fairness as Open Source Software Engineer at probable. She is a maintainer of fairlearn, contributor to scikit-learn and skops. Tamara has both computer science/ software engineering and a computational linguistics(NLP) background.During the event, the guest discussed their career journey from software engineering to open-source contributions, focusing on explainability in AI through Scikit-learn and Fairlearn. They explored fairness in AI, including challenges in credit loans, hiring, and decision-making, and emphasized the importance of tools, human judgment, and collaboration. The guest also shared their involvement with PyLadies and encouraged contributions to Fairlearn.00:00 Introduction to the event and the community01:51 Topic introduction: Linguistic fairness and socio-technical perspectives in AI02:37 Guest introduction: Tamara’s background and career03:18 Tamara’s career journey: Software engineering, music tech, and computational linguistics09:53 Tamara’s background in language and computer science14:52 Exploring fairness in AI and its impact on society21:20 Fairness in AI models26:21 Automating fairness analysis in models32:32 Balancing technical and domain expertise in decision-making37:13 The role of humans in the loop for fairness40:02 Joining Probable and working on open-source projects46:20 Scopes library and its integration with Hugging Face50:48 PyLadies and community involvement55:41 The ethos of Scikit-learn and Fairlearn🔗 CONNECT WITH TAMARA ATANASOSKALinkedin - https://www.linkedin.com/in/tamaraatanasoskaGitHub- https://github.com/TamaraAtanasoska🔗 CONNECT WITH DataTalksClubJoin DataTalks.Club:⁠⁠https://datatalks.club/slack.html⁠⁠Our events:⁠⁠https://datatalks.club/events.html⁠⁠Datalike Substack -⁠⁠https://datalike.substack.com/⁠⁠LinkedIn:⁠⁠  / datatalks-club  

  --------  

  53:11
• Career choices, transitions and promotions in and out of tech - Agita Jaunzeme

  In this podcast episode, we talked with Agita Jaunzeme about Career choices, transitions and promotions in and out of tech. About the Speaker: Agita has designed a career spanning DevOps/DataOps engineering, management, community building, education, and facilitation. She has worked on projects across corporate, startup, open source, and non-governmental sectors. Following her passion, she founded an NGO focusing on the inclusion of expats and locals in Porto. Embodying the values of innovation, automation, and continuous learning, Agita provides practical insights on promotions, career pivots, and aligning work with passion and purpose. During this event, discussed their career journey, starting with their transition from art school to programming and later into DevOps, eventually taking on leadership roles. They explored the challenges of burnout and the importance of volunteering, founding an NGO to support inclusion, gender equality, and sustainability. The conversation also covered key topics like mentorship, the differences between data engineering and data science, and the dynamics of managing volunteers versus employees. Additionally, the guest shared insights on community management, developer relations, and the importance of product vision and team collaboration. 0:00 Introduction and Welcome 1:28 Guest Introduction: Agita’s Background and Career Highlights 3:05 Transition to Tech: From Art School to Programming 5:40 Exploring DevOps and Growing into Leadership Roles 7:24 Burnout, Volunteering, and Founding an NGO 11:00 Volunteering and Mentorship Initiatives 14:00 Discovering Programming Skills and Early Career Challenges 15:50 Automating Work Processes and Earning a Promotion 19:00 Transitioning from DevOps to Volunteering and Project Management 24:00 Managing Volunteers vs. Employees and Building Organizational Skills 31:07 Personality traits in engineering vs. data roles 33:14 Differences in focus between data engineers and data scientists 36:24 Transitioning from volunteering to corporate work 37:38 The role and responsibilities of a community manager 39:06 Community management vs. developer relations activities 41:01 Product vision and team collaboration 43:35 Starting an NGO and legal processes 46:13 NGO goals: inclusion, gender equality, and sustainability 49:02 Community meetups and activities 51:57 Living off-grid in a forest and sustainability 55:02 Unemployment party and brainstorming session 59:03 Unemployment party: the process and structure 🔗 CONNECT WITH AGITA JAUNZEME Linkedin - /agita 🔗 CONNECT WITH DataTalksClub Join DataTalks.Club: ⁠https://datatalks.club/slack.html⁠ Our events: ⁠https://datatalks.club/events.html⁠ Datalike Substack - ⁠https://datalike.substack.com/⁠ LinkedIn: ⁠  / datatalks-club  

  --------  

  55:21

Altri podcast di Tecnologia

Podcast di tendenza in Tecnologia

Su DataTalks.Club