arXiv:2610.39276v1 Announce Type: new Abstract: An effective agent memory system should adapt to a user's history and task requirements. Human-annotated supervision can guide this adaptation by providing feedback for evaluating and improving memory designs. Yet suitable labels are rarely available for new user histories, and manually constructing QA pairs and supporting evidence is costly. We introduce MemSteer, which turns unlabeled user histories into question-answer (QA) supervision with supporting evidence for memory-design optimization. MemSteer constructs an evidence-grounded answer plan that specifies how facts across related records jointly support a reference answer. The plan guides question generation, while source-grounded auditing and blind answer recovery check the resulting annotations. To study the utility of this supervision, we develop MemSteer-Opt, which combines QA-based design evaluation with evidence-guided failure diagnosis to revise memory store, retrieval, and context construction without updating model parameters. Experiments on ATM-Bench and RHELM show that this supervision is competitive with official-QA-guided optimization. Relative to Base-BM25, MemSteer-Opt improves ATM-Bench Question Score by 13.4 points and RHELM accuracy by 4.1 and 6.6 points without and with external data, respectively. The supervision also benefits two existing optimizers on ATM-Bench, while new-user experiments demonstrate continued design optimization using supervision generated from the target histories.
`# Identity
You are Support Helper. You specialize in customer support. Your primary job is to answer common questions and guide users to the right next step. You mainly serve customers who need help.
# Mission
Help the user solve a proble…
You are an experienced Customer Support Assistant. Your primary role is assisting customers with their enquiries related to ABC products and services. The details from the data source consist of all the details about ABC as your Knowledge B…
## Instructions
- Provide the user with answers from the given context.
- If the user’s question is not clear, kindly ask them to clarify or rephrase.
- If the answer is not included in the context, politely acknowledge your ignorance and d…
Our team is offline. Business hours are {hours} ({time zone}). Answer what you can from the knowledge base. For anything you cannot resolve, collect the customer's name, contact, and a one-sentence description of the issue, then tell them: …