Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer | Paper Notes

Paper notes for Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer.