FinPerMA Benchmarks LLM Agent Personalized Memory in Finance

As Large Language Model (LLM) agents are increasingly deployed as personalized assistants, particularly in critical sectors like financial advising, their capacity to maintain and adapt individualized user models over extended periods is a key concern. Existing benchmarks for personalized memory often focus on factual recall or rely on loosely constrained, model-generated scenarios, leaving the crucial aspect of event-driven preference adaptation largely unexplored. To address this, FinPerMA has been developed as an event-grounded benchmark. It assesses personalized memory by testing LLM agents against pre-defined, longitudinal investor trajectories, incorporating deterministic, theory-informed impact rules and controlled LLM narration. A "Post-Shock" checkpoint specifically evaluates whether an agent successfully integrates significant financial events into its persistent user model. Evaluations across 2,994 questions from 276 personas, involving seven frontier LLMs and various memory configurations, show that current models are far from proficient. No full-context configuration achieved more than approximately 47% overall accuracy, or 39% on multiple-choice questions. Analysis indicates that summary-based memory often retains facts but loses critical preference signals needed for true personalization, sometimes making simple retrieval more effective, especially after market shocks.