Recently, ICDE 2023 (IEEE International Conference on Data Engineering, CCF-A Conference), one of the premier conferences in the database field, was successfully held in Anaheim, CA. The research paper “Incremental Learning for Multi-Interest Sequential Recommendation” from the Data Driven Software Technology Laboratory (DDST) in the Department of Computer Science and Engineering won the best paper award of ICDE 2023.

The winning paper, entitled "Incremental Learning for Multi-Interest Sequential Recommendation", proposes an effective incremental learning framework named IMSR for multi-interest sequential recommendation. IMSR augments the traditional fine-tuning strategy with the existing-interests retainer (EIR), new-interests detector (NID), and projection-based interests trimmer (PIT) to adaptively expand the model to accommodate user’s new interests and prevent it from forgetting user’s existing interests. Extensive experiments on real-world datasets verify the effectiveness of IMSR on incremental multi-interest sequential recommendation, compared with various state-of-the-art approaches.

Motivation

Multi-interest Sequential Recommendation (MSR) explores multiple latent interests of users from the sequences of historical interactions for item recommendation. This technology has received extensive attention from academia and industry due to its excellent recommendation performance. In practice, recommendation systems will continuously collect a user's latest interaction sequence, whose existing interests may drift slightly and new interests will occur. How to use the latest interaction data to update the multi-interest sequence recommendation model is an important research topic. At present, the widely used model update methods mainly include two types: full retraining and incremental learning. The former captures all the interests contained in user's historical interaction sequence, thus incurring high training cost. The latter uses only incremental data to fine-tune the model. While incremental learning is more cost-effective, it suffers from forgetting the user’s existing interests that appear in the historical interaction data, and it is impossible to dynamically expand the model to capture the user's new interests. Therefore, it is of great importance to develop an incremental learning framework for multi-interest sequence recommendation, which has the ability to retain user's existing interests and dynamically capture user's new interests.

Figure 1 Incremental learning for multi-interest sequential recommendation

Research Outcomes

This paper proposes an incremental learning framework named IMSR for multi-interest sequence recommendation model, which can retain the user's existing interests and dynamically expand the model to capture the user's new interests. The IMSR framework consists of three important components: existing interest retainer (EIR), new interest detector (NID), and projection-based interest trimmer (PIT). The EIR utilizes knowledge distillation to constrain the updates of user’s existing interests, ensuring that representations of existing interests do not drift from their original locations during incremental learning of the model. The NID is responsible for determining whether a user has generated new interests based on the correlation between the newly interacted items and the existing interests. If so, the model will be expanded dynamically to encode the identified new interests. The PIT computes the projection of new interests on existing interests’ hyperplane to prune trivial new interests and remove redundant interests. With the above three components, IMSR not only retains user's existing interests, but also dynamically and adaptively creates new interests for each user that evolve from new interactions. Experimental results on multiple multi-interest sequence recommendation datasets show that IMSR outperforms the state-of-the-art model update methods by achieving better recommendation performance.

Figure 2 The overview of the proposed IMSR framework

Author Information

The first author of the paper, Zhikai Wang, is a fourth-year doctoral student in the Department of Computer Science and Engineering at Shanghai Jiao Tong University. His research focuses on dynamic adaptive learning for recommender systems. At present, he has published several academic papers in top international conferences such as IJCAI, and ICDE. He obtained the bachelor degree from Shanghai Jiao Tong University and owned the Shengshen Scholarship. Home page: https://cloudcatcher888.github.io.

The corresponding author of the paper, Yanyan Shen, is a tenured associate professor in the Department of Computer Science and Engineering at Shanghai Jiao Tong University. Her broad research interests include: big data analysis and processing, machine learning, data mining, etc. She has published more than 90 high-quality research papers in the related fields, and has won the VLDB 2022 Best Research Paper, DASFAA 2020 Best Paper Runner-up, and APWeb-WAIM 2018 Best Student Paper. She serves as associate editors for many premier international conferences and journals including IEEE TKDE, VLDB 2023-2024.

Conference Information

IEEE International Conference on Data Engineering (ICDE) is the flagship conference held by the Institute of Electrical and Electronics Engineers (IEEE). It is one of the three top international conferences in the field of databases, which enjoys a high reputation internationally and has extensive academic influence.

Paper Link

https://cloudcatcher888.github.io/files/icde.pdf