update doc

Author: namespace-Pt

Long_LLM/activation_beacon/new/README.md

@@ -1,6 +1,6 @@
 # Activation-Beacon
 
-[Activation Beacon](https://arxiv.org/abs/2401.03462) compresses the original KV into fewer yet more compact states (a.k.a. beacons) and hence enabling the LLM to perceive longer context given its fixed context window. It is known for the following features:
+[Activation Beacon](https://arxiv.org/abs/2401.03462) compresses the original KV into fewer yet more compact states (a.k.a. beacons) and hence enables the LLM to perceive longer context given its fixed context window. It is known for the following features:
 - **Effective**
   - there is little information loss given a compression ratio of 2, 4, and 8;
 - **Efficient**

Long_LLM/activation_beacon/new/docs/training.md

@@ -131,8 +131,8 @@ torchrun --nproc_per_node 8 $DDP -m main.train \
 --min_length 7200 \
 --group_by_stride strict \
 --enable_beacon \
---beacon_window 2048 \
---beacon_stride 2048 \
+--beacon_window 1024 \
+--beacon_stride 1024 \
 --beacon_attn full-coverage \
 --beacon_attend_prev True \
 --beacon_sink_size 1 \