File tree Expand file tree Collapse file tree
Long_LLM/activation_beacon/new Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -21,6 +21,9 @@ wget https://huggingface.co/datasets/namespace-Pt/projects/resolve/main/activati
2121
2222cd /data
2323tar -xzvf activation-beacon.tar.gz
24+
25+ # you must download the new longalpaca dataset that was organized into single-turn conversation
26+ wget https://huggingface.co/datasets/namespace-Pt/projects/resolve/main/longalpaca.json? download=true -O /data/activation-beacon/finetune/longalpaca.new.json
2427```
2528
2629** IMPORTANT NOTE**
3639torchrun --nproc_per_node 8 -m main.train \
3740--output_dir data/outputs/activation-beacon-llama2-chat-7b \
3841--model_name_or_path meta-llama/Llama-2-7b-chat-hf \
39- --train_data activation-beacon:pretrain/redpajama-sample.json activation-beacon:finetune/longalpaca.json \
42+ --train_data activation-beacon:pretrain/redpajama-sample.json activation-beacon:finetune/longalpaca.new. json \
4043--max_length 8192 \
4144--min_length 1200 \
4245--max_train_num_per_data 200000 \
You can’t perform that action at this time.
0 commit comments