Long Context

Slides
Video Lecture

References

  1. Long Context Compression with Activation BeaconPeitian Zhang, Zheng Liu, Shitao Xiao, Ninglu Shao, Qiwei Ye, Zhicheng Dou2024
  2. RoFormer: Enhanced Transformer with Rotary Position EmbeddingJianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, Yunfeng Liu2021
  3. Extending Context Window of Large Language Models via Positional InterpolationShouyuan Chen, Sherman Wong, Liangjian Chen, Yuandong Tian2023