mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-29 07:34:31 +00:00
42 lines
1.9 KiB
TeX
42 lines
1.9 KiB
TeX
\section{Related Work}
|
|
|
|
\label{sec:related}
|
|
|
|
\paragraph{Federated Learning.}
|
|
FedAvg~\cite{mcmahan2017communication} aggregates local model updates via
|
|
averaging. FedProx~\cite{li2020federated} adds a proximal term for
|
|
heterogeneous settings. These methods aggregate gradients/parameters, unlike
|
|
DSO which aggregates semantic priors.
|
|
|
|
\paragraph{Byzantine-Robust Aggregation.}
|
|
Krum~\cite{blanchard2017machine} selects the update closest to others.
|
|
Bulyan~\cite{mhamdi2018hidden} combines Krum with trimmed mean.
|
|
\cite{yin2018byzantine} analyze coordinate-wise median and trimmed mean.
|
|
Our weighted median extends these ideas to semantic prior aggregation.
|
|
|
|
\paragraph{Decentralized Machine Learning.}
|
|
Swarm Learning~\cite{warnat2021swarm} coordinates peer-to-peer training
|
|
without a central server. Gossip-based SGD~\cite{koloskova2020unified}
|
|
distributes gradient computation. DSO differs by operating on compressed
|
|
semantic priors rather than model parameters.
|
|
|
|
\paragraph{Blockchain for ML.}
|
|
FLock~\cite{flock2024} uses blockchain for federated learning coordination.
|
|
Bittensor~\cite{bittensor2023} creates a decentralized marketplace for ML
|
|
models. DSO uses blockchain only for integrity and incentive management,
|
|
keeping compute and storage off-chain.
|
|
|
|
\paragraph{Parameter-Efficient Federated Learning.}
|
|
FedLoRA~\cite{zhang2024fedlora} and FedPara~\cite{hyeon2022fedpara}
|
|
federate LoRA or low-rank parameter updates. These still require gradient
|
|
computation; DSO is entirely gradient-free.
|
|
|
|
\paragraph{Experience Sharing in Multi-Agent RL.}
|
|
\cite{christianos2021scaling} and \cite{baker2020emergent} study experience
|
|
sharing in multi-agent reinforcement learning. DSO adapts these ideas to the
|
|
LLM setting with compressed semantic priors instead of raw trajectories.
|
|
|
|
% ============================================================================
|
|
% 12. CONCLUSION
|
|
% ============================================================================
|