Fate-FedAI@lists.lfaidata.foundation | FATE & KubeFATE v1.7.0 released! Flexible & decoupled FATE-Flow, new compute & storage engine support, tons of algorithm enhancements and MUCH MORE!

Major Features and Improvements in FATE 1.7.0

FATE-ARCH

Support EggRoll 2.4.0
Support Spark-Local Computing Engine
Support Hive Storage
Support LocalFS Storage for Spark-Local Computing Engine
Optimizing the API interface for Storage session and table
Simplified the API interface for Session, remove backend and workmode parameters
Heterogeneous Engine Support: Federation between Spark-Local and Spark-Cluster
Computing Engine, Storage Engine, Federation Engine are set in conf/service_conf.yaml when FATE is deployed

FederatedML

Optimized Hetero-SecureBoost: with gradient packing、cipher compressing, and sparse point statistics optimization, 4x+ faster
Homo-SecureBoost supports memory-based histogram computation for more efficient tree building, 5x+ faster
Optimized RSA Intersect with CRT optimization, 3x+ faster
New two-party Hetero Logistic Regression Algorithm: mixed protocol of HE and MPC, without a trusted third party
Support data with match-id, separating match id and sample id
New DH Intersect based on PH Key-exchange protocol
Intersect support cardinality estimation
Intersect adds optionally preprocessing step
RSA and DH Intersect support cache
New Feature Imputation module: can apply arbitrary imputation method to each column
New Label Transform module: transform categorical label values
Homo-LR, Homo-SecureBoost, Homo-NN now can convert models into sklearn、lightgbm、torch & tf-keras framework
Hetero Feature Binning supports multi-class task, higher efficiency with label packing
Hetero Feature Selection support multi-class iv filter
Secure Information Retrieval supports multi-column retrieval
Major training algorithms support warm-start and checkpoint : Homo & Hetero LR, Homo & Hetero-SecureBoost, Homo & Hetero NN
Optimized Pailler addition operation, several times faster, Hetero-SecureBoost with Paillier speed up 2x+

Fate-Client

Pipeline supports uploading match id functionality
Pipeline supports homo model conversion
Pipeline supports model push to FATE-Serving
Pipeline supports running jobs with specified FATE version

FATE-Test

Integrate FederatedML unittest
Support for uploading image data
Big data generation using storage interface, optimized generation logic
Support for historical data comparison
cache_deps and model_loader_deps support
Run DSL Testsuite with specified FATE version

Major Features and Improvements in FATE-Flow 1.7.0

Independent repository instead of all code in the main FATE repository
Component registry, which can hot load many different versions of component packages at the same time
Hot update of component parameters, component-specific reruns, automatic reruns
Model Checkpoint to support task hot start, model deployment and other
Data, Model and Cache can be reused between jobs
Reader component supports more data sources, such as MySQL, Hive
Realtime recording of dataset usage derivation routes
Multi-party permission control for datasets
Automatic push to reliable storage when model deployment, support Tencent Cloud COS, MySQL, Redis
REST API authentication

Major Features and Improvements in KubeFATE 1.7.0

KubeFATE v1.7.0 supports FATE v1.7.0 and brings the following updates:

Update the examples come with docker-compose from DSL v1 to DSL v2;
Update KubeFATE and FATE's Ingress deployment to support Kubernetes v1.22;
Update all examples of the Jupyter Notebook;
Fix some bugs.

The deliverables include:

The Docker Compose v1.7.0;
The FATE v1.7.0 chart;
The Exchange v1.7.0 chart.
The KubeFATE CLI and KubeFATE server image v1.4.2;

Note: The KubeFATE CLI and KubeFATE service do not need to upgrade. Import new charts following the guide: Manage charts in KubeFATE manually

Cheers!

wfangchi@... #63 Hi, We are very excited to introduce the recently released FATE 1.7.0 and KubeFATE 1.7.0! It is a major release with significant architectural improvements, lots of new features, enhancements and bug-fixes. Welcome to check this out! One of the major changes in FATE 1.7.0 is FATE-Flow, the multi-party federated task security scheduling platform for federated learning end-to-end pipeline, is now in an independent repository, which enables a more flexible model of working with all the different components. Release Notes: FATE 1.7.0: https://github.com/FederatedAI/FATE/releases/tag/v1.7.0 FATE-Flow 1.7.0: https://github.com/FederatedAI/FATE-Flow/releases/tag/v1.7.0 KubeFATE 1.7.0: https://github.com/FederatedAI/KubeFATE/releases/tag/v1.7.0 Major Features and Improvements in FATE 1.7.0 FATE-ARCH Support EggRoll 2.4.0 Support Spark-Local Computing Engine Support Hive Storage Support LocalFS Storage for Spark-Local Computing Engine Optimizing the API interface for Storage session and table Simplified the API interface for Session, remove backend and workmode parameters Heterogeneous Engine Support: Federation between Spark-Local and Spark-Cluster Computing Engine, Storage Engine, Federation Engine are set in conf/service_conf.yaml when FATE is deployed FederatedML Optimized Hetero-SecureBoost: with gradient packing、cipher compressing, and sparse point statistics optimization, 4x+ faster Homo-SecureBoost supports memory-based histogram computation for more efficient tree building, 5x+ faster Optimized RSA Intersect with CRT optimization, 3x+ faster New two-party Hetero Logistic Regression Algorithm: mixed protocol of HE and MPC, without a trusted third party Support data with match-id, separating match id and sample id New DH Intersect based on PH Key-exchange protocol Intersect support cardinality estimation Intersect adds optionally preprocessing step RSA and DH Intersect support cache New Feature Imputation module: can apply arbitrary imputation method to each column New Label Transform module: transform categorical label values Homo-LR, Homo-SecureBoost, Homo-NN now can convert models into sklearn、lightgbm、torch & tf-keras framework Hetero Feature Binning supports multi-class task, higher efficiency with label packing Hetero Feature Selection support multi-class iv filter Secure Information Retrieval supports multi-column retrieval Major training algorithms support warm-start and checkpoint : Homo & Hetero LR, Homo & Hetero-SecureBoost, Homo & Hetero NN Optimized Pailler addition operation, several times faster, Hetero-SecureBoost with Paillier speed up 2x+ Fate-Client Pipeline supports uploading match id functionality Pipeline supports homo model conversion Pipeline supports model push to FATE-Serving Pipeline supports running jobs with specified FATE version FATE-Test Integrate FederatedML unittest Support for uploading image data Big data generation using storage interface, optimized generation logic Support for historical data comparison cache_deps and model_loader_deps support Run DSL Testsuite with specified FATE version Major Features and Improvements in FATE-Flow 1.7.0 Independent repository instead of all code in the main FATE repository Component registry, which can hot load many different versions of component packages at the same time Hot update of component parameters, component-specific reruns, automatic reruns Model Checkpoint to support task hot start, model deployment and other Data, Model and Cache can be reused between jobs Reader component supports more data sources, such as MySQL, Hive Realtime recording of dataset usage derivation routes Multi-party permission control for datasets Automatic push to reliable storage when model deployment, support Tencent Cloud COS, MySQL, Redis REST API authentication Major Features and Improvements in KubeFATE 1.7.0 KubeFATE v1.7.0 supports FATE v1.7.0 and brings the following updates: Update the examples come with docker-compose from DSL v1 to DSL v2; Update KubeFATE and FATE's Ingress deployment to support Kubernetes v1.22; Update all examples of the Jupyter Notebook; Fix some bugs. The deliverables include: The Docker Compose v1.7.0; The FATE v1.7.0 chart; The Exchange v1.7.0 chart. The KubeFATE CLI and KubeFATE server image v1.4.2; Note: The KubeFATE CLI and KubeFATE service do not need to upgrade. Import new charts following the guide: Manage charts in KubeFATE manually Cheers!
More All Messages By This Member