Advancing provably private learning from federated data

A newly deployed federated learning system uses trusted execution environments to verify privacy guarantees externally.

2 min read

Advancing provably private learning from federated data

A new machine learning architecture aims to make privacy guarantees verifiable while improving training speed and accuracy. The system builds on federated learning — a technique where multiple decentralized devices collaborate to train a model under the coordination of a service provider without sharing raw user data.

Key facts

Paper Title
Toward provably private learning from federated data
Authors
Katharine Daly and 11 other authors
Application
Android Keyboard (Gboard)
Line graph showing noise multiplier versus differential privacy budget for old and new systems.
Privacy-utility curves comparing the previous federated learning system against the new TEE-based architecture., Source ↗

Trusted execution environments

To remove the need to trust the server operator, the updated system leverages trusted execution environments, which are secure hardware areas that offer confidentiality and integrity. Logic running inside these environments can be remotely attested by third parties to verify that computations run exactly as intended.

Diagram illustrating data flow and verification flow through trusted execution environments and transparency logs.
Technical architecture diagram of the confidential federated learning system featuring TEE processing and verification steps., Source ↗

The framework coordinates four primary operational concepts:

  • Data upload: Client devices locally encrypt training examples and pre-authorize an access policy that dictates allowable computations.
  • Key management: A consensus protocol cluster grants decryption keys only to server-side workloads matching the published access policy.
  • Workload execution: A root trusted execution environment executes a Python training program and delegates subtasks to worker environments.
  • Fault-tolerant recovery: Programs save an encrypted recovery state at the end of training rounds to recover from failures without leaking sensitive data.

Transparency and speedups

Access policies that define server workloads are published to a public transparency log called Rekor. External auditors can inspect these logs to track the full set of server-side workloads that devices might participate in, while binaries can be built reproducibly from open-source code.

By collecting device uploads before running server-side training, the framework avoids diurnal availability constraints where device connectivity fluctuates throughout the day. Furthermore, shifting compute bottlenecks from mobile devices to server clusters allows training times to drop significantly compared to older systems that took months to complete.

The architecture has already been adopted by Gboard to deploy English and Japanese next-word prediction models. Future work will explore integrating hardware accelerators with trusted execution environments to train larger models and support arbitrary Python workloads.

Source: Original announcement

Worth a read?

Report

Comments