Welcome to Flyte Monthly!

Where you stay up to date with Flyte news, features, releases, events, and success stories.

 

🚀 What's new this month? 🚀

🎉Flyte v1.4.0 Release

Highlights:

• Suport for PodTemplate at the task-level
• Revamped auth system in flytekit
• New sandbox features, including support for:
  • Specifying extra configuration for Flyte
  • Specifying extra cluster resource templates for boostrapping new namespaces
  • Sandbox state (DB, buckets) is now persistent across restarts and upgrades

For full release notes, visit the release page on GitHub.
❗Note: dropped support for Python 3.7 (EOL'd in Dec 2022)

 

📣 New Governance model!

➡️ Check out the roles, responsibilities, requirements and values we're aiming to define for a sustainable, collaborative and healthy Flyte commmunity!

 

📅 Flyte contributors meeting comes to life!

With the goal of discussing RFCs, ideas and further topics around the contributor experience, there's a new meeting in the calendar:

• Every other Thursday at 1PM PT (starting March 16)
• Zoom bridge
• Import the calendar to avoid missing it!

It's a great opportunity to participate on discussions with members from Flyte's Technical Steering Committee or just to listen and learn!

 

🔔 Office Hours moved to on-demand format

In order to optimize time commitment from maintainers and be better prepared to address your questions, Office Hours can now be booked on demand using Calendly!

 💡What's coming for Flyte?💡

🎉Flyte v1.5.0 (scheduled for end of March)

Theme: improved flytekit authoring experience

Planned improvements:

• Partial Tasks #3403
• Mypy Typing Support for flytekit #2147
• Improved flytekit error messages #3404

➡️See the full roadmap and uptove/comment issues

 

❓Do you have an idea but not sure if it should be an RFC? Create an entry in the RFC Incubator to get the discussion started!

🧩 Latest integration 🧩

Flyte+Dask

Dask and Flyte have different approaches in how they create DAGs. While dask uses long-running workers, Flyte creates Kubernetes pods. Both methods differ in terms of bootup time, but Flyte adds a missing orchestration layer and a plugin ecosystem, providing multiple benefits including:

• Each Flyte task deploys a dask cluster using a Docker image tailored to the task providing consistency and customized dependency management
• Spot/Preemtible instances are natively supported
• The whole dask task can be cached
• Rapid setup

 

Shoutout

A big thank you to Bernhard Stadlbauer from Pachama who not only developed the integration with the support from Flyte's maintainer team, but also covered all the details in a blog post in the dask site!

🤝 Success Stories 🤝

Nokia: from PoC to Production

 

How to provide a unified platform for a team of about 100 Data Scientists to conduct all their experiments without asking them to write YAML? That was the overarching problem the Nokia Standards Unit was trying to solve.

 

After six months of researching open source solutions, Nokia adopted Flyte to orchestrate both short and long running trainings (from 3 hours to 5 days), integrating it successfully with their existing infrastructure.

 

Why Flyte?

• Much closer to the code (just add a decorator to your Python function)
• Rapid setup
• Good community support

 

➡️ Watch Sujit Samuel, Chief ML Engineer at Nokia, presenting the details of their journey with Flyte!

🎙️📖 Fresh content 📖🎙️

Blog posts

 

• Attach a Visual Debugger to ML-training Jobs on Kubernetes
  • Dr. Fabio Graetz covers the process to attach a visual debugger to VS Code for a deep learning training environment using Flyte, making debugging simpler and more efficient
• Who Let the DAGs out? From dbt into Flyte: Exporting and Running a DAG
  • Tim Leonard from Spotify, shares a comprehensive exploration on DAGs, dbt and Flyte!

⚙️ Jobs ⚙️

• Company: Union.ai
• Role: Senior Python Engineer
• Location: Seattle or fully remote
• See full job description and apply here

 

Looking for talent or in search of your next position? 

➡️ Head over to #jobs and let the community know!

✔ New adopters ✔ 

😎 Who's using Flyte?

 

• Kineo.ai
• Recogni
• Warner Bros

 

➡️Check out the full ADOPTERS list and add your project/organization!

🗓️ Coming Up 🗓️

Adopting Flyte at Shift Technologies

Tune in to learn from Sam Cox about how Shift Technologies uses Flyte!

📅 Tue, March 21, 9:00AM PST

 

➡️ Import the calendar of Flyte community events here

🙏  Thank you, Contributors! 🙏 

Special thanks go out to the following Flyte community members for being our February ✨Contributors of the Month✨!

Bernhard Stadlbauer

Sr Data Engineer at Pachama.

Nick Muller

Sr Go Developer at Blackshark.ai

Fabio Graetz, PhD

Sr ML Engineer at Recogni

As an integral part of our community, they have been providing valuable insights and answers to questions on Slack. We appreciate their consistent efforts to keep our community vibrant and engaged.

Flyte is an open-source workflow orchestration platform for 

building data, ML and analytics workflows with ease. 

Join our community on Slack

https://slack.flyte.org 

You've received it because you've subscribed to our newsletter.