Papers I readed recently about LLM application

How much do LLM memorize? key definition unintended memorization: memorize a specific dataset generalization (intended memorization): contains about the true data-generation process calculation method: by information entropy and mutual information double desent appear on the changing points from unintended memorization into generalization GPT-models store 3.6bits data per parameters value of…