Why AI Training Clusters Are Pushing Peering Into the Data Center
Most of the time, a GPU node in a training cluster is busy talking to the switches right above it and swapping gradients with the rest of its compute pod over the fabric. But it may need to get outside the Data Center. It grabs dataset shards from an object