Skip to content

Add ACLE intrinsics for non-temporal and unprivileged pair loads/stores #471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
CarolineConcatto wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add ACLE intrinsics for non-temporal and unprivileged pair loads/stores #471

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions neon_intrinsics/advsimd.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ See [Changelog for year 2026](#changelog-for-year-2026).

### Changelog for year 2026

* Added proposed support for FEAT_LSUI LDTNP/STTNP intrinsics.
* Added support for FEAT_F16F32DOT
* Added support for FEAT_F16F32MM and FEAT_F16MM
* Renamed vmmlaq_f16_f16 to vmmlaq_f16
Expand Down Expand Up @@ -3829,6 +3830,27 @@ The intrinsics in this section are guarded by the macro ``__ARM_NEON``.

### Load


#### Unprivileged pair with non-temporal hint


| Intrinsic | Argument preparation | AArch64 Instruction | Result | Supported architectures |
|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------|---------------------------------------------------------------------------------------------------------------------|---------------------------|
| <code>int8x16x2_t <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vldtnq_s8_x2" target="_blank">vldtnq_s8_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; int8_t const *ptr)</code> | `ptr -> Xn` | `LDTNP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| <code>int16x8x2_t <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vldtnq_s16_x2" target="_blank">vldtnq_s16_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; int16_t const *ptr)</code> | `ptr -> Xn` | `LDTNP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| <code>int32x4x2_t <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vldtnq_s32_x2" target="_blank">vldtnq_s32_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; int32_t const *ptr)</code> | `ptr -> Xn` | `LDTNP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| <code>int64x2x2_t <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vldtnq_s64_x2" target="_blank">vldtnq_s64_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; int64_t const *ptr)</code> | `ptr -> Xn` | `LDTNP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| <code>uint8x16x2_t <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vldtnq_u8_x2" target="_blank">vldtnq_u8_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; uint8_t const *ptr)</code> | `ptr -> Xn` | `LDTNP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| <code>uint16x8x2_t <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vldtnq_u16_x2" target="_blank">vldtnq_u16_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; uint16_t const *ptr)</code> | `ptr -> Xn` | `LDTNP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| <code>uint32x4x2_t <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vldtnq_u32_x2" target="_blank">vldtnq_u32_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; uint32_t const *ptr)</code> | `ptr -> Xn` | `LDTNP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| <code>uint64x2x2_t <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vldtnq_u64_x2" target="_blank">vldtnq_u64_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; uint64_t const *ptr)</code> | `ptr -> Xn` | `LDTNP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| <code>float16x8x2_t <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vldtnq_f16_x2" target="_blank">vldtnq_f16_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; float16_t const *ptr)</code> | `ptr -> Xn` | `LDTNP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| <code>bfloat16x8x2_t <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vldtnq_bf16_x2" target="_blank">vldtnq_bf16_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; bfloat16_t const *ptr)</code> | `ptr -> Xn` | `LDTNP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| <code>float32x4x2_t <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vldtnq_f32_x2" target="_blank">vldtnq_f32_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; float32_t const *ptr)</code> | `ptr -> Xn` | `LDTNP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| <code>float64x2x2_t <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vldtnq_f64_x2" target="_blank">vldtnq_f64_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; float64_t const *ptr)</code> | `ptr -> Xn` | `LDTNP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| <code>mfloat8x16x2_t <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vldtnq_mf8_x2" target="_blank">vldtnq_mf8_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; mfloat8_t const *ptr)</code> | `ptr -> Xn` | `LDTNP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |


#### Stride

| Intrinsic | Argument preparation | AArch64 Instruction | Result | Supported architectures |
Expand Down Expand Up @@ -4308,6 +4330,24 @@ The intrinsics in this section are guarded by the macro ``__ARM_NEON``.

### Store

#### Unprivileged pair with non-temporal hint

| Intrinsic | Argument preparation | AArch64 Instruction | Result | Supported architectures |
|-----------|----------------------|---------------------|--------|-------------------------|
| <code>void <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vsttnq_s8_x2" target="_blank">vsttnq_s8_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; int8_t *ptr,<br>&nbsp;&nbsp;&nbsp;&nbsp; int8x16x2_t val)</code> | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTNP Qt1,Qt2,[Xn]` | | `A64` |
| <code>void <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vsttnq_s16_x2" target="_blank">vsttnq_s16_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; int16_t *ptr,<br>&nbsp;&nbsp;&nbsp;&nbsp; int16x8x2_t val)</code> | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTNP Qt1,Qt2,[Xn]` | | `A64` |
| <code>void <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vsttnq_s32_x2" target="_blank">vsttnq_s32_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; int32_t *ptr,<br>&nbsp;&nbsp;&nbsp;&nbsp; int32x4x2_t val)</code> | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTNP Qt1,Qt2,[Xn]` | | `A64` |
| <code>void <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vsttnq_s64_x2" target="_blank">vsttnq_s64_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; int64_t *ptr,<br>&nbsp;&nbsp;&nbsp;&nbsp; int64x2x2_t val)</code> | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTNP Qt1,Qt2,[Xn]` | | `A64` |
| <code>void <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vsttnq_u8_x2" target="_blank">vsttnq_u8_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; uint8_t *ptr,<br>&nbsp;&nbsp;&nbsp;&nbsp; uint8x16x2_t val)</code> | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTNP Qt1,Qt2,[Xn]` | | `A64` |
| <code>void <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vsttnq_u16_x2" target="_blank">vsttnq_u16_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; uint16_t *ptr,<br>&nbsp;&nbsp;&nbsp;&nbsp; uint16x8x2_t val)</code> | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTNP Qt1,Qt2,[Xn]` | | `A64` |
| <code>void <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vsttnq_u32_x2" target="_blank">vsttnq_u32_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; uint32_t *ptr,<br>&nbsp;&nbsp;&nbsp;&nbsp; uint32x4x2_t val)</code> | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTNP Qt1,Qt2,[Xn]` | | `A64` |
| <code>void <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vsttnq_u64_x2" target="_blank">vsttnq_u64_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; uint64_t *ptr,<br>&nbsp;&nbsp;&nbsp;&nbsp; uint64x2x2_t val)</code> | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTNP Qt1,Qt2,[Xn]` | | `A64` |
| <code>void <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vsttnq_f16_x2" target="_blank">vsttnq_f16_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; float16_t *ptr,<br>&nbsp;&nbsp;&nbsp;&nbsp; float16x8x2_t val)</code> | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTNP Qt1,Qt2,[Xn]` | | `A64` |
| <code>void <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vsttnq_bf16_x2" target="_blank">vsttnq_bf16_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; bfloat16_t *ptr,<br>&nbsp;&nbsp;&nbsp;&nbsp; bfloat16x8x2_t val)</code> | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTNP Qt1,Qt2,[Xn]` | | `A64` |
| <code>void <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vsttnq_f32_x2" target="_blank">vsttnq_f32_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; float32_t *ptr,<br>&nbsp;&nbsp;&nbsp;&nbsp; float32x4x2_t val)</code> | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTNP Qt1,Qt2,[Xn]` | | `A64` |
| <code>void <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vsttnq_f64_x2" target="_blank">vsttnq_f64_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; float64_t *ptr,<br>&nbsp;&nbsp;&nbsp;&nbsp; float64x2x2_t val)</code> | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTNP Qt1,Qt2,[Xn]` | | `A64` |
| <code>void <a href="https://developer.arm.com/architectures/instruction-sets/intrinsics/vsttnq_mf8_x2" target="_blank">vsttnq_mf8_x2</a>(<br>&nbsp;&nbsp;&nbsp;&nbsp; mfloat8_t *ptr,<br>&nbsp;&nbsp;&nbsp;&nbsp; mfloat8x16x2_t val)</code> | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTNP Qt1,Qt2,[Xn]` | | `A64` |

#### Stride

| Intrinsic | Argument preparation | AArch64 Instruction | Result | Supported architectures |
Expand Down
1 change: 1 addition & 0 deletions neon_intrinsics/advsimd.template.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ See [Changelog for year 2026](#changelog-for-year-2026).

### Changelog for year 2026

* Added proposed support for FEAT_LSUI LDTNP/STTNP intrinsics.
* Added support for FEAT_F16F32DOT
* Added support for FEAT_F16F32MM and FEAT_F16MM
* Renamed vmmlaq_f16_f16 to vmmlaq_f16
Expand Down
27 changes: 27 additions & 0 deletions tools/intrinsic_db/advsimd.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4863,3 +4863,30 @@ float32x4_t vmmlaq_f32_f16(float32x4_t r, float16x8_t a, float16x8_t b) r -> Vd.

<SECTION> Non-widening half-precision matrix multiply instruction. Requires the +f16mm architecture extension.
float16x8_t vmmlaq_f16(float16x8_t r, float16x8_t a, float16x8_t b) r -> Vd.8H;a -> Vn.8H;b -> Vm.8H FMMLA Vd.8H, Vn.8H, Vm.8H Vd.8H -> result A64
<SECTION> Unprivileged pair with non-temporal hint
int8x16x2_t vldtnq_s8_x2(int8_t const *ptr) ptr -> Xn LDTNP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
int16x8x2_t vldtnq_s16_x2(int16_t const *ptr) ptr -> Xn LDTNP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
int32x4x2_t vldtnq_s32_x2(int32_t const *ptr) ptr -> Xn LDTNP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
int64x2x2_t vldtnq_s64_x2(int64_t const *ptr) ptr -> Xn LDTNP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
uint8x16x2_t vldtnq_u8_x2(uint8_t const *ptr) ptr -> Xn LDTNP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
uint16x8x2_t vldtnq_u16_x2(uint16_t const *ptr) ptr -> Xn LDTNP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
uint32x4x2_t vldtnq_u32_x2(uint32_t const *ptr) ptr -> Xn LDTNP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
uint64x2x2_t vldtnq_u64_x2(uint64_t const *ptr) ptr -> Xn LDTNP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
float16x8x2_t vldtnq_f16_x2(float16_t const *ptr) ptr -> Xn LDTNP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
bfloat16x8x2_t vldtnq_bf16_x2(bfloat16_t const *ptr) ptr -> Xn LDTNP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
float32x4x2_t vldtnq_f32_x2(float32_t const *ptr) ptr -> Xn LDTNP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
float64x2x2_t vldtnq_f64_x2(float64_t const *ptr) ptr -> Xn LDTNP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
mfloat8x16x2_t vldtnq_mf8_x2(mfloat8_t const *ptr) ptr -> Xn LDTNP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
void vsttnq_s8_x2(int8_t *ptr, int8x16x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTNP Qt1,Qt2,[Xn] A64
void vsttnq_s16_x2(int16_t *ptr, int16x8x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTNP Qt1,Qt2,[Xn] A64
void vsttnq_s32_x2(int32_t *ptr, int32x4x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTNP Qt1,Qt2,[Xn] A64
void vsttnq_s64_x2(int64_t *ptr, int64x2x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTNP Qt1,Qt2,[Xn] A64
void vsttnq_u8_x2(uint8_t *ptr, uint8x16x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTNP Qt1,Qt2,[Xn] A64
void vsttnq_u16_x2(uint16_t *ptr, uint16x8x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTNP Qt1,Qt2,[Xn] A64
void vsttnq_u32_x2(uint32_t *ptr, uint32x4x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTNP Qt1,Qt2,[Xn] A64
void vsttnq_u64_x2(uint64_t *ptr, uint64x2x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTNP Qt1,Qt2,[Xn] A64
void vsttnq_f16_x2(float16_t *ptr, float16x8x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTNP Qt1,Qt2,[Xn] A64
void vsttnq_bf16_x2(bfloat16_t *ptr, bfloat16x8x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTNP Qt1,Qt2,[Xn] A64
void vsttnq_f32_x2(float32_t *ptr, float32x4x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTNP Qt1,Qt2,[Xn] A64
void vsttnq_f64_x2(float64_t *ptr, float64x2x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTNP Qt1,Qt2,[Xn] A64
void vsttnq_mf8_x2(mfloat8_t *ptr, mfloat8x16x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTNP Qt1,Qt2,[Xn] A64
26 changes: 26 additions & 0 deletions tools/intrinsic_db/advsimd_classification.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4745,3 +4745,29 @@ vdot_laneq_f32_f16 Vector arithmetic|Dot product
vdotq_lane_f32_f16 Vector arithmetic|Dot product
vmmlaq_f32_f16 Vector arithmetic|Matrix multiply
vmmlaq_f16 Vector arithmetic|Matrix multiply
vldtnq_s8_x2 Load|Stride
vldtnq_s16_x2 Load|Stride
vldtnq_s32_x2 Load|Stride
vldtnq_s64_x2 Load|Stride
vldtnq_u8_x2 Load|Stride
vldtnq_u16_x2 Load|Stride
vldtnq_u32_x2 Load|Stride
vldtnq_u64_x2 Load|Stride
vldtnq_f16_x2 Load|Stride
vldtnq_bf16_x2 Load|Stride
vldtnq_f32_x2 Load|Stride
vldtnq_f64_x2 Load|Stride
vldtnq_mf8_x2 Load|Stride
vsttnq_s8_x2 Store|Stride
vsttnq_s16_x2 Store|Stride
vsttnq_s32_x2 Store|Stride
vsttnq_s64_x2 Store|Stride
vsttnq_u8_x2 Store|Stride
vsttnq_u16_x2 Store|Stride
vsttnq_u32_x2 Store|Stride
vsttnq_u64_x2 Store|Stride
vsttnq_f16_x2 Store|Stride
vsttnq_bf16_x2 Store|Stride
vsttnq_f32_x2 Store|Stride
vsttnq_f64_x2 Store|Stride
vsttnq_mf8_x2 Store|Stride
Loading