links

Author: vpj

docs/index.html

@@ -140,7 +140,8 @@ <h4>✨ <a href="optimizers/index.html">Optimizers</a></h4>
 <li><a href="optimizers/adam_warmup.html">Adam Optimizer with warmup</a> </li>
 <li><a href="optimizers/noam.html">Noam Optimizer</a> </li>
 <li><a href="optimizers/radam.html">Rectified Adam Optimizer</a> </li>
-<li><a href="optimizers/ada_belief.html">AdaBelief Optimizer</a></li></ul>
+<li><a href="optimizers/ada_belief.html">AdaBelief Optimizer</a> </li>
+<li><a href="optimizers/sophia.html">Sophia-G Optimizer</a></li></ul>
 <h4>✨ <a href="normalization/index.html">Normalization Layers</a></h4>
 <ul><li><a href="normalization/batch_norm/index.html">Batch Normalization</a> </li>
 <li><a href="normalization/layer_norm/index.html">Layer Normalization</a> </li>

docs/optimizers/index.html

@@ -77,7 +77,8 @@ <h2>Optimizer Implementations</h2>
 <li><a href="https://nn.labml.ai/optimizers/adam_warmup.html">Adam Optimizer with warmup</a> </li>
 <li><a href="https://nn.labml.ai/optimizers/noam.html">Noam Optimizer</a> </li>
 <li><a href="https://nn.labml.ai/optimizers/radam.html">Rectified Adam Optimizer</a> </li>
-<li><a href="https://nn.labml.ai/optimizers/ada_belief.html">AdaBelief Optimizer</a> </li></ul>
+<li><a href="https://nn.labml.ai/optimizers/ada_belief.html">AdaBelief Optimizer</a> </li>
+<li><a href="https://nn.labml.ai/optimizers/sophia.html">Sophia-G Optimizer</a> </li></ul>
 
         </div>
         <div class='code'>

docs/optimizers/readme.html

@@ -109,6 +109,7 @@
 * [Noam Optimizer](optimizers/noam.html)
 * [Rectified Adam Optimizer](optimizers/radam.html)
 * [AdaBelief Optimizer](optimizers/ada_belief.html)
+* [Sophia-G Optimizer](optimizers/sophia.html)
 
 #### ✨ [Normalization Layers](normalization/index.html)
 * [Batch Normalization](normalization/batch_norm/index.html)

labml_nn/__init__.py

@@ -15,6 +15,7 @@
 * [Noam Optimizer](noam.html)
 * [Rectified Adam Optimizer](radam.html)
 * [AdaBelief Optimizer](ada_belief.html)
+* [Sophia-G Optimizer](sophia.html)
 
 This [MNIST example](mnist_experiment.html) uses these optimizers.
 

labml_nn/optimizers/__init__.py

@@ -7,3 +7,4 @@
 * [Noam Optimizer](https://nn.labml.ai/optimizers/noam.html)
 * [Rectified Adam Optimizer](https://nn.labml.ai/optimizers/radam.html)
 * [AdaBelief Optimizer](https://nn.labml.ai/optimizers/ada_belief.html)
+* [Sophia-G Optimizer](https://nn.labml.ai/optimizers/sophia.html)

labml_nn/optimizers/readme.md

@@ -106,6 +106,7 @@ Solving games with incomplete information such as poker with CFR.
 * [Noam Optimizer](https://nn.labml.ai/optimizers/noam.html)
 * [Rectified Adam Optimizer](https://nn.labml.ai/optimizers/radam.html)
 * [AdaBelief Optimizer](https://nn.labml.ai/optimizers/ada_belief.html)
+* [Sophia-G Optimizer](https://nn.labml.ai/optimizers/sophia.html)
 
 #### ✨ [Normalization Layers](https://nn.labml.ai/normalization/index.html)
 * [Batch Normalization](https://nn.labml.ai/normalization/batch_norm/index.html)