ResBM: Residual Bottleneck Models for Low-Bandwidth Pipeline Parallelism

Implement a prototype of Residual Bottleneck Models (ResBM) to prove that large models can be trained over low-bandwidth consumer connections. This could enable distributed LLM training on heterogeneous hardware.

Suggested repo: ResPipe

"Train massive models over high-latency networks."

Estimated effort: 100h