Upper and lower memory capacity bounds of transformers for next-token prediction
L. Madden, C. Fox, C. Thrampoulidis. "Next-token Prediction Capacity: General Upper Bounds and a Lower Bound for Transformers". IEEE Transactions on Information Theory, 2025