Code

Causal attention#448

Code

Draft

nicolasvasilache wants to merge 12 commits intoiree-org:mainiree-org/iree-turbine:mainfrom nicolasvasilache:causal_attentionnicolasvasilache/iree-turbine:causal_attentionCopy head branch name to clipboard

+2,212-83

Commits

Commits on Jan 30, 2025

• hack iota / self_index

  Show description for eba3427
  

  ftynse

  authored and

  nicolasvasilache

  committed
  eba3427

  Copy full SHA for eba3427

• self_index fixes and stress test

  Show description for 1e45ec3
  nicolasvasilache

  committed
  1e45ec3

  Copy full SHA for 1e45ec3

• Add extend attention

  Show description for 1f308bb
  

  harsh-nod

  authored and

  nicolasvasilache

  committed
  1f308bb

  Copy full SHA for 1f308bb

Commits on Jan 31, 2025

• Initial support for T5 RPE in vanilla attention

  Show description for 8b2cd37
  nicolasvasilache

  committed
  8b2cd37

  Copy full SHA for 8b2cd37

• Add support for tkw.minimum

  Show description for 9bba5b1
  nicolasvasilache

  committed
  9bba5b1

  Copy full SHA for 9bba5b1

Commits on Feb 1, 2025

• Finish T5 RPE on vanilla attention

  Show description for 48aa3ec
  nicolasvasilache

  committed
  48aa3ec

  Copy full SHA for 48aa3ec

Commits on Feb 3, 2025

• Rework T5 RPE test

  Show description for acfa521
  nicolasvasilache

  committed
  acfa521

  Copy full SHA for acfa521

• Rename files

  Show description for 6be5ef0
  nicolasvasilache

  committed
  6be5ef0

  Copy full SHA for 6be5ef0

• add signed integer comparisons, select and pow ops

  Show description for 5296cf2
  

  ftynse

  authored and

  nicolasvasilache

  committed
  5296cf2

  Copy full SHA for 5296cf2

• Add simple triangular test that will be useful for causal attention

  Show description for 7bfc1c6
  nicolasvasilache

  committed
  7bfc1c6

  Copy full SHA for 7bfc1c6

• Add causal attention

  Show description for a2e0c64
  nicolasvasilache

  committed
  a2e0c64

  Copy full SHA for a2e0c64

• Debug causal attention

  nicolasvasilache

  committed
  953ee25

  Copy full SHA for 953ee25