Skip to content

Commit 196160b

Browse files
authored
Merge pull request karpathy#247 from gnobre/macbook-run-instructions
Macbook run instructions
2 parents 21f9bff + e30c8fd commit 196160b

1 file changed

Lines changed: 6 additions & 1 deletion

File tree

README.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,11 @@ $ python train.py config/train_shakespeare_char.py --device=cpu --compile=False
7676

7777
Here, since we are running on CPU instead of GPU we must set both `--device=cpu` and also turn off PyTorch 2.0 compile with `--compile=False`. Then when we evaluate we get a bit more noisy but faster estimate (`--eval_iters=20`, down from 200), our context size is only 64 characters instead of 256, and the batch size only 12 examples per iteration, not 64. We'll also use a much smaller Transformer (4 layers, 4 heads, 128 embedding size), and decrease the number of iterations to 2000 (and correspondingly usually decay the learning rate to around max_iters with `--lr_decay_iters`). Because our network is so small we also ease down on regularization (`--dropout=0.0`). This still runs in about ~3 minutes, but gets us a loss of only 1.88 and therefore also worse samples, but it's still good fun:
7878

79+
```
80+
$ python sample.py --out_dir=out-shakespeare-char --device=cpu
81+
```
82+
Generates samples like this:
83+
7984
```
8085
GLEORKEN VINGHARD III:
8186
Whell's the couse, the came light gacks,
@@ -86,7 +91,7 @@ No relving thee post mose the wear
8691

8792
Not bad for ~3 minutes on a CPU, for a hint of the right character gestalt. If you're willing to wait longer, feel free to tune the hyperparameters, increase the size of the network, the context length (`--block_size`), the length of training, etc.
8893

89-
Finally, on Apple Silicon Macbooks and with a recent PyTorch version make sure to add `--device mps` (short for "Metal Performance Shaders"); PyTorch then uses the on-chip GPU that can *significantly* accelerate training (2-3X) and allow you to use larger networks. See [Issue 28](https://github.com/karpathy/nanoGPT/issues/28) for more.
94+
Finally, on Apple Silicon Macbooks and with a recent PyTorch version make sure to add `--device=mps` (short for "Metal Performance Shaders"); PyTorch then uses the on-chip GPU that can *significantly* accelerate training (2-3X) and allow you to use larger networks. See [Issue 28](https://github.com/karpathy/nanoGPT/issues/28) for more.
9095

9196
## reproducing GPT-2
9297

0 commit comments

Comments
 (0)