Co-authors: Hangfei Lin, Jinghui Mo
We’re excited to announce today that Feathr is joining LF AI & Data, the Linux Foundation’s umbrella foundation supporting open source innovation in artificial intelligence (AI) and data. Feathr is a feature store that simplifies machine learning (ML) feature serving and improves developer productivity.
“We're excited to welcome Feathr to LF AI & Data and for it to be part of our technical project portfolio (41 projects and growing) with a community of over 17K developers,” said Dr. Ibrahim Haddad, Executive Director of LF AI & Data. “We aim to support Feathr to expand its user base, grow its community of developers, become a leader within its own category, and enable collaboration and integration opportunities with other projects. We look forward to the project’s continued growth and success as part of LF AI & Data.”
Launched internally at LinkedIn in 2017, Feathr makes feature serving easy, fast, and scalable, especially for real-time AI applications. We built it to help our AI teams store, transform, serve, and share features with low latency and high throughput.
Feathr powers many of our AI applications and manages thousands of features. It has reduced the engineering time required for adding and experimenting with new features from weeks to days. It’s also performed up to 50% faster than the custom feature processing pipelines that it replaced.
Why we are donating Feathr
In April 2022, Feathr was released under the Apache 2.0 license and we announced, in close conjunction with our Microsoft Azure partners, native integration and support for Feathr on Azure. Since being open sourced, Feathr has achieved substantial popularity among the machine learning operations (MLOps) community. It has been adopted by companies of various sizes across multiple industries and the community continues to grow rapidly. Most excitingly, more and more open-source enthusiasts are contributing code to Feathr.
It’s clear that many others experience the same pain points that Feathr aims to address. That’s why we are excited to share it with a broader audience and for Feathr to be adopted by a broader open-source community with help from LF AI & Data.
Donating to the LF AI & Data will help ensure that Feathr continues to grow and evolve across various dimensions, including visibility, user base, and contributor base. Also, the Feathr development team will have more opportunities to collaborate with other member companies and projects, such as achieving richer online store support via integration with Milvus and JanusGraph, and adopting open data lineage standard from OpenLineage. As a result, we hope Feathr helps AI engineers build and scale feature pipelines and feature applications in ways that push MLOps tech stacks and the industry forward for years to come.
Feathr’s Benefits
The Feathr feature store provides an abstraction layer between raw data and ML models. This abstraction layer standardizes and simplifies feature definition, transformation, serving, storage, and access from within ML workflows or applications. Feathr empowers AI engineers to focus on feature engineering while it takes care of data serialization format, connecting to various databases, performance optimization, and credential management. More specifically, Feathr helps:
- Define features once and use them in different scenarios, like model training and model serving
- Create training dataset with point-in-time correct semantics
- Connect to various offline data sources (data lakes, and data warehouses), and then transform source data into features
- Deliver feature data from offline system to online store for faster online serving
- Discover features or share features among colleagues or teams with ease
To learn more please visit Feathr’s GitHub page here and our April 2022 blog, Open sourcing Feathr – LinkedIn’s feature store for productive machine learning.
Acceptance into the LF AI & Data indicates an important recognition from the Linux Foundation. We believe a large, diverse, healthy, and self-sustained Feathr open-source community is important. We’re excited for the new chapter of Feathr and to welcome more people into the Feathr community.