Emulating ALiBi with RoPE

LLMs usually use some sort of positional encoding for helping the model understand where different tokens — or more precisely, KV cache entries — are. Two of the main strategies for doing this are RoP...