Skip to content

Commit ae612e3

Browse files
committed
update readme
1 parent b06814e commit ae612e3

1 file changed

Lines changed: 4 additions & 1 deletion

File tree

Long_LLM/activation_beacon/new/README.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,9 @@ wget https://huggingface.co/datasets/namespace-Pt/projects/resolve/main/activati
2121

2222
cd /data
2323
tar -xzvf activation-beacon.tar.gz
24+
25+
# you must download the new longalpaca dataset that was organized into single-turn conversation
26+
wget https://huggingface.co/datasets/namespace-Pt/projects/resolve/main/longalpaca.json?download=true -O /data/activation-beacon/finetune/longalpaca.new.json
2427
```
2528

2629
**IMPORTANT NOTE**
@@ -36,7 +39,7 @@ cd new
3639
torchrun --nproc_per_node 8 -m main.train \
3740
--output_dir data/outputs/activation-beacon-llama2-chat-7b \
3841
--model_name_or_path meta-llama/Llama-2-7b-chat-hf \
39-
--train_data activation-beacon:pretrain/redpajama-sample.json activation-beacon:finetune/longalpaca.json \
42+
--train_data activation-beacon:pretrain/redpajama-sample.json activation-beacon:finetune/longalpaca.new.json \
4043
--max_length 8192 \
4144
--min_length 1200 \
4245
--max_train_num_per_data 200000 \

0 commit comments

Comments
 (0)