Pytorch - Tutorial(1)

• $\sum_{k=1}^{\infty} \epsilon_{k} = \infty$
• $\sum_{k=1}^{\infty} \epsilon_{k}^{2} < \infty$
• $\alpha \in [0,1)$
• $v \leftarrow \alpha v - \epsilon \nabla_{\theta}(\frac{1}{m} \sum_{i=1}^{m} L(f(x^{(i)};\theta), y^{(i)}))$