目录

AAAI'2024

Federated Contextual Cascading Bandits with Asynchronous Communication and Heterogeneous Users

General Commerce Intelligence: Glocally Federated NLP-Based Engine for Privacy-Preserving and Sustainable Personalized Services of Multi-Merchants

No Prejudice! Fair Federated Graph Neural Networks for Personalized Recommendation

ICLR'2024

Federated Recommendation with Additive Personalization

NeurIPS'23

Large-Scale Distributed Learning via Private On-Device LSH

Wyze Rule: Federated Rule Dataset for Rule Recommendation Benchmarking

ICML'2023

Vertical Federated Graph Neural Network for Recommender System

SIGIR'2023

Fine-Grained Preference-Aware Personalized Federated POI Recommendation with Data Sparsity

Manipulating Federated Recommender Systems: Poisoning with Synthetic Users and Its Countermeasures

AAAI'2023

Untargeted Attack against Federated Recommendation Systems via Poisonous Item Embeddings and the Defense

Win-Win: A Privacy-Preserving Federated Framework for Dual-Target Cross-Domain Recommendation

KDD'2023

UA-FedRec: Untargeted Attack on Federated News Recommendation

Privacy Matters: Vertical Federated Linear Contextual Bandits for Privacy Protected Recommendation

PrivateRec: Differentially Private Model Training and Online Serving for Federated News Recommendation

---

AAAI'2024

Federated Contextual Cascading Bandits with Asynchronous Communication and Heterogeneous Users

Authors: Hantao Yang; Xutong Liu; Zhiyong Wang; Hong Xie; John C. S. Lui; Defu Lian; Enhong Chen

Url: https://ojs.aaai.org/index.php/AAAI/article/view/30045

Abstract: We study the problem of federated contextual combinatorial cascading bandits, where agents collaborate under the coordination of a central server to provide tailored recommendations to users. Existing works consider either a synchronous framework, necessitating full agent participation and global synchronization, or assume user homogeneity with identical behaviors. We overcome these limitations by considering (1) federated agents operating in an asynchronous communication paradigm, where no mandatory synchronization is required and all agents communicate independently with the server, (2) heterogeneous user behaviors, where users can be stratified into latent user clusters, each exhibiting distinct preferences. For this setting, we propose a UCB-type algorithm with delicate communication protocols. Through theoretical analysis, we give sub-linear regret bounds on par with those achieved in the synchronous framework, while incurring only logarithmic communication costs. Empirical evaluation on synthetic and real-world datasets validates our algorithm's superior performance in terms of regrets and communication costs.

abstractTranslation: 我们研究联合上下文组合级联老虎机的问题，其中代理在中央服务器的协调下进行协作，为用户提供量身定制的建议。现有的工作要么考虑同步框架，需要完全代理参与和全局同步，要么假设用户具有相同行为的同质性。我们通过考虑（1）在异步通信范式中运行的联合代理来克服这些限制，其中不需要强制同步并且所有代理独立地与服务器通信，（2）异构用户行为，其中用户可以分层为潜在用户集群，每个人都表现出不同的偏好。对于这种设置，我们提出了一种具有精细通信协议的 UCB 型算法。通过理论分析，我们给出了与同步框架中实现的亚线性后悔界限相同的结果，同时仅产生对数通信成本。对合成数据集和真实数据集的实证评估验证了我们的算法在遗憾和通信成本方面的卓越性能。

Notes:

PAGE PDF

General Commerce Intelligence: Glocally Federated NLP-Based Engine for Privacy-Preserving and Sustainable Personalized Services of Multi-Merchants

Authors: Kyoung Jun Lee; Baek Jeong; Suhyeon Kim; Dam Kim; Dongju Park

Url: https://ojs.aaai.org/index.php/AAAI/article/view/30309

Abstract: One of the most crucial capabilities in the commercial sector is a personalized prediction of a customer's next purchase. We present a novel method of creating a commerce intelligence engine that caters to multiple merchants intended for the UB Platform, managed by e-payment company Harex InfoTech. To cultivate this intelligence, we utilized payment receipt data and created a Natural Language Processing (NLP)-based commerce model using a Transformer to accommodate multinational and merchant trade. Our model, called General Commerce Intelligence (GCI), provides a range of services for merchants, including product recommendations, product brainstorming, product bundling, event promotions, collaborative marketing, target marketing, and demand fore-casting etc. To bolster user privacy and foster sustainable business collaboration, especially among micro-, small-, and medium-sized enterprises (MSMEs), the GCI model was trained through federated learning, especially with glocalization. This study delves into the structure, development, and assessment of GCI, showcasing its transformative capacity to implement User Centric AI and re-shape the global commerce landscape to benefit MSMEs.

abstractTranslation: 商业领域最重要的功能之一是对客户下次购买的个性化预测。我们提出了一种创建商业智能引擎的新颖方法，该引擎可满足 UB 平台的多个商家的需求，由电子支付公司 Harex InfoTech 管理。为了培养这种智能，我们利用付款收据数据，并使用 Transformer 创建基于自然语言处理 (NLP) 的商业模型，以适应跨国和商业贸易。我们的模型称为通用商业智能（GCI），为商家提供一系列服务，包括产品推荐、产品头脑风暴、产品捆绑、活动促销、协作营销、目标营销和需求预测等。为了促进可持续的业务合作，特别是在微型、小型和中型企业 (MSME) 之间，GCI 模型通过联合学习（尤其是全球本地化）进行培训。本研究深入探讨了 GCI 的结构、开发和评估，展示了其实施以用户为中心的人工智能和重塑全球商业格局以使中小微企业受益的变革能力。

Notes:

PAGE

No Prejudice! Fair Federated Graph Neural Networks for Personalized Recommendation

Authors: Nimesh Agrawal; Anuj Kumar Sirohi; Sandeep Kumar; Jayadeva

Url: https://ojs.aaai.org/index.php/AAAI/article/view/28950

Abstract: Ensuring fairness in Recommendation Systems (RSs) across demographic groups is critical due to the increased integration of RSs in applications such as personalized healthcare, finance, and e-commerce. Graph-based RSs play a crucial role in capturing intricate higher-order interactions among entities. However, integrating these graph models into the Federated Learning (FL) paradigm with fairness constraints poses formidable challenges as this requires access to the entire interaction graph and sensitive user information (such as gender, age, etc.) at the central server. This paper addresses the pervasive issue of inherent bias within RSs for different demographic groups without compromising the privacy of sensitive user attributes in FL environment with the graph-based model. To address the group bias, we propose F2PGNN (Fair Federated Personalized Graph Neural Network), a novel framework that leverages the power of Personalized Graph Neural Network (GNN) coupled with fairness considerations. Additionally, we use differential privacy techniques to fortify privacy protection. Experimental evaluation on three publicly available datasets showcases the efficacy of F2PGNN in mitigating group unfairness by 47% ∼ 99% compared to the state-of-the-art while preserving privacy and maintaining the utility. The results validate the significance of our framework in achieving equitable and personalized recommendations using GNN within the FL landscape. Source code is at: https://github.com/nimeshagrawal/F2PGNN-AAAI24

abstractTranslation: 由于推荐系统 (RS) 在个性化医疗、金融和电子商务等应用中的集成度不断提高，确保推荐系统 (RS) 跨人口群体的公平性至关重要。基于图的 RS 在捕获实体之间复杂的高阶交互方面发挥着至关重要的作用。然而，将这些图模型集成到具有公平性约束的联邦学习（FL）范例中会带来巨大的挑战，因为这需要访问整个交互图和中央服务器上的敏感用户信息（例如性别、年龄等）。本文通过基于图的模型解决了 RS 中针对不同人口群体的固有偏见的普遍问题，同时又不损害 FL 环境中敏感用户属性的隐私。为了解决群体偏见，我们提出了 F2PGNN（公平联邦个性化图神经网络），这是一种新颖的框架，利用个性化图神经网络（GNN）的力量并结合公平性考虑。此外，我们还使用差分隐私技术来加强隐私保护。对三个公开数据集的实验评估表明，与最先进的技术相比，F2PGNN 在减轻群体不公平性方面的功效提高了 47% ~ 99%，同时保护了隐私并保持了实用性。结果验证了我们的框架在 FL 领域使用 GNN 实现公平和个性化推荐的重要性。源代码位于：https://github.com/nimeshagrawal/F2PGNN-AAAI24

Notes:

PAGE PDF CODE

ICLR'2024

Federated Recommendation with Additive Personalization

Authors:

Conference : The Twelfth International Conference on Learning Representations

Url: https://openreview.net/forum?id=xkXdE81mOK

Abstract: Building recommendation systems via federated learning (FL) is a new emerging challenge for next-generation Internet service. Existing FL models share item embedding across clients while keeping the user embedding private and local on the client side. However, identical item embedding cannot capture users' individual differences in perceiving the same item and may lead to poor personalization. Moreover, dense item embedding in FL results in expensive communication costs and latency. To address these challenges, we propose Federated Recommendation withAdditive Personalization (FedRAP), which learns a global view of items via FL and a personalized view locally on each user. FedRAP encourages a sparse global view to save FL's communication cost and enforces the two views to be complementary via two regularizers. We propose an effective curriculum to learn the local and global views progressively with increasing regularization weights. To produce recommendations for a user, FedRAP adds the two views together to obtain a personalized item embedding. FedRAP achieves the best performance in FL setting on multiple benchmarks. It outperforms recent federated recommendation methods and several ablation study baselines. Our code is available at https://github.com/mtics/FedRAP.

abstractTranslation: 通过联邦学习（FL）构建推荐系统是下一代互联网服务面临的新挑战。现有的 FL 模型在客户端之间共享项目嵌入，同时在客户端保持用户嵌入的私密性和本地性。然而，相同的项目嵌入无法捕获用户在感知同一项目时的个体差异，并且可能导致个性化较差。此外，FL 中的密集项目嵌入会导致昂贵的通信成本和延迟。为了应对这些挑战，我们提出了具有附加个性化的联邦推荐（FedRAP），它通过 FL 学习项目的全局视图以及每个用户的本地个性化视图。 FedRAP 鼓励稀疏的全局视图以节省 FL 的通信成本，并通过两个正则化器强制这两种视图互补。我们提出了一种有效的课程，通过增加正则化权重来逐步学习本地和全球观点。为了为用户生成推荐，FedRAP 将两个视图添加在一起以获得个性化的项目嵌入。 FedRAP 在多个基准测试中实现了 FL 设置的最佳性能。它优于最近的联邦推荐方法和几个消融研究基线。我们的代码可在 https://github.com/mtics/FedRAP 获取。

Notes:

PDF CODE

NeurIPS'23

Large-Scale Distributed Learning via Private On-Device LSH

Authors: Tahseen Rabbani; Marco Bornstein; Furong Huang

Conference : Thirty-seventh Conference on Neural Information Processing Systems

Url: https://openreview.net/forum?id=dpdbbN7AKr

Abstract: Locality-sensitive hashing (LSH) based frameworks have been used efficiently to select weight vectors in a dense hidden layer with high cosine similarity to an input, enabling dynamic pruning. While this type of scheme has been shown to improve computational training efficiency, existing algorithms require repeated randomized projection of the full layer weight, which is impractical for computational- and memory-constrained devices. In a distributed setting, deferring LSH analysis to a centralized host is (i) slow if the device cluster is large and (ii) requires access to input data which is forbidden in a federated context. Using a new family of hash functions, we develop the first private, personalized, and memory-efficient on-device LSH framework. Our framework enables privacy and personalization by allowing each device to generate hash tables, without the help of a central host, using device-specific hashing hyper-parameters (e.g., number of hash tables or hash length). Hash tables are generated with a compressed set of the full weights, and can be serially generated and discarded if the process is memory-intensive. This allows devices to avoid maintaining (i) the fully-sized model and (ii) large amounts of hash tables in local memory for LSH analysis. We prove several statistical and sensitivity properties of our hash functions, and experimentally demonstrate that our framework is competitive in training large scale recommender networks compared to other LSH frameworks which assume unrestricted on-device capacity.

abstractTranslation: 基于局部敏感哈希 (LSH) 的框架已被有效地用于在与输入具有高余弦相似度的密集隐藏层中选择权重向量，从而实现动态修剪。虽然这种类型的方案已被证明可以提高计算训练效率，但现有算法需要对全层权重进行重复随机投影，这对于计算和内存受限的设备来说是不切实际的。在分布式设置中，将 LSH 分析推迟到集中式主机会 (i) 如果设备集群很大，则速度很慢；(ii) 需要访问在联合上下文中被禁止的输入数据。使用一系列新的哈希函数，我们开发了第一个私有、个性化且内存高效的设备上 LSH 框架。我们的框架通过允许每个设备生成哈希表来实现隐私和个性化，而无需中央主机的帮助，使用设备特定的哈希超参数（例如哈希表的数量或哈希长度）。哈希表是使用完整权重的压缩集生成的，如果该过程是内存密集型的，则可以连续生成和丢弃。这允许设备避免在本地内存中维护 (i) 全尺寸模型和 (ii) 大量哈希表以进行 LSH 分析。我们证明了哈希函数的几个统计和敏感性属性，并通过实验证明，与假设设备上容量不受限制的其他 LSH 框架相比，我们的框架在训练大规模推荐网络方面具有竞争力。

Notes:

SUPP PDF

Wyze Rule: Federated Rule Dataset for Rule Recommendation Benchmarking

Authors: Mohammad Mahdi Kamani; Yuhang Yao; Hanjia Lyu; Zhongwei Cheng; Lin Chen; Liangju Li; Carlee Joe-Wong; Jiebo Luo

Conference : Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track

Url: https://openreview.net/forum?id=qynH28Y4xE

Abstract: In the rapidly evolving landscape of smart home automation, the potential of IoT devices is vast. In this realm, rules are the main tool utilized for this automation, which are predefined conditions or triggers that establish connections between devices, enabling seamless automation of specific processes. However, one significant challenge researchers face is the lack of comprehensive datasets to explore and advance the field of smart home rule recommendations. These datasets are essential for developing and evaluating intelligent algorithms that can effectively recommend rules for automating processes while preserving the privacy of the users, as it involves personal information about users' daily lives. To bridge this gap, we present the Wyze Rule Dataset, a large-scale dataset designed specifically for smart home rule recommendation research. Wyze Rule encompasses over 1 million rules gathered from a diverse user base of 300,000 individuals from Wyze Labs, offering an extensive and varied collection of real-world data. With a focus on federated learning, our dataset is tailored to address the unique challenges of a cross-device federated learning setting in the recommendation domain, featuring a large-scale number of clients with widely heterogeneous data. To establish a benchmark for comparison and evaluation, we have meticulously implemented multiple baselines in both centralized and federated settings. Researchers can leverage these baselines to gauge the performance and effectiveness of their rule recommendation systems, driving advancements in the domain. The Wyze Rule Dataset is publicly accessible through HuggingFace's dataset API.

abstractTranslation: 在快速发展的智能家居自动化领域，物联网设备潜力巨大。在这个领域，规则是用于这种自动化的主要工具，它们是在设备之间建立连接的预定义条件或触发器，从而实现特定流程的无缝自动化。然而，研究人员面临的一个重大挑战是缺乏全面的数据集来探索和推进智能家居规则建议领域。这些数据集对于开发和评估智能算法至关重要，这些算法可以有效地推荐自动化流程的规则，同时保护用户的隐私，因为它涉及有关用户日常生活的个人信息。为了弥补这一差距，我们提出了 Wyze 规则数据集，这是一个专为智能家居规则推荐研究而设计的大型数据集。 Wyze Rule 包含从 Wyze Labs 300,000 名不同用户群收集的超过 100 万条规则，提供广泛且多样化的真实世界数据集合。我们的数据集专注于联邦学习，旨在解决推荐领域跨设备联邦学习设置的独特挑战，其特点是拥有大量具有广泛异构数据的客户端。为了建立比较和评估的基准，我们在集中式和联邦式设置中精心实施了多个基线。研究人员可以利用这些基线来衡量其规则推荐系统的性能和有效性，从而推动该领域的进步。 Wyze 规则数据集可通过 HuggingFace 的数据集 API 公开访问。

Notes:

SUPP Dataset

ICML'2023

Vertical Federated Graph Neural Network for Recommender System

Authors: Peihua Mai; Yan Pang

Conference : International Conference on Machine Learning

Url: https://proceedings.mlr.press/v202/mai23b.html

Abstract: Conventional recommender systems are required to train the recommendation model using a centralized database. However, due to data privacy concerns, this is often impractical when multi-parties are involved in recommender system training. Federated learning appears as an excellent solution to the data isolation and privacy problem. Recently, Graph neural network (GNN) is becoming a promising approach for federated recommender systems. However, a key challenge is to conduct embedding propagation while preserving the privacy of the graph structure. Few studies have been conducted on the federated GNN-based recommender system. Our study proposes the first vertical federated GNN-based recommender system, called VerFedGNN. We design a framework to transmit: (i) the summation of neighbor embeddings using random projection, and (ii) gradients of public parameter perturbed by ternary quantization mechanism. Empirical studies show that VerFedGNN has competitive prediction accuracy with existing privacy preserving GNN frameworks while enhanced privacy protection for users’ interaction information.

ISSN: 2640-3498 abstractTranslation: 传统的推荐系统需要使用集中式数据库来训练推荐模型。然而，由于数据隐私问题，当多方参与推荐系统训练时，这通常是不切实际的。联邦学习似乎是数据隔离和隐私问题的绝佳解决方案。最近，图神经网络（GNN）正在成为联邦推荐系统的一种有前景的方法。然而，一个关键的挑战是在保护图结构隐私的同时进行嵌入传播。关于基于 GNN 的联邦推荐系统的研究很少。我们的研究提出了第一个基于 GNN 的纵向联邦推荐系统，称为 VerFedGNN。我们设计了一个框架来传输：（i）使用随机投影的邻居嵌入的总和，以及（ii）由三元量化机制扰动的公共参数的梯度。实证研究表明，VerFedGNN 与现有的隐私保护 GNN 框架相比，具有竞争性的预测精度，同时增强了对用户交互信息的隐私保护。

Notes:

[PUB] PDF [CODE]

SIGIR'2023

Fine-Grained Preference-Aware Personalized Federated POI Recommendation with Data Sparsity

Authors: Xiao Zhang; Ziming Ye; Jianfeng Lu; Fuzhen Zhuang; Yanwei Zheng; Dongxiao Yu

Conference : SIGIR '23: The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Url: https://dl.acm.org/doi/10.1145/3539618.3591688

Abstract: With the raised privacy concerns and rigorous data regulations, federated learning has become a hot collaborative learning paradigm for the recommendation model without sharing the highly sensitive POI data. However, the time-sensitive, heterogeneous, and limited POI records seriously restrict the development of federated POI recommendation. To this end, in this paper, we design the fine-grained preference-aware personalized federated POI recommendation framework, namely PrefFedPOI, under extremely sparse historical trajectories to address the above challenges. In details, PrefFedPOI extracts the fine-grained preference of current time slot by combining historical recent preferences and periodic preferences within each local client. Due to the extreme lack of POI data in some time slots, a data amount aware selective strategy is designed for model parameters uploading. Moreover, a performance enhanced clustering mechanism with reinforcement learning is proposed to capture the preference relatedness among all clients to encourage the positive knowledge sharing. Furthermore, a clustering teacher network is designed for improving efficiency by clustering guidance. Extensive experiments are conducted on two diverse real-world datasets to demonstrate the effectiveness of proposed PrefFedPOI comparing with state-of-the-arts. In particular, personalized PrefFedPOI can achieve 7% accuracy improvement on average among data-sparsity clients.

abstractTranslation: 随着隐私问题的提出和严格的数据监管，联邦学习已成为推荐模型的热门协作学习范式，无需共享高度敏感的 POI 数据。然而，时间敏感、异构、有限的POI记录严重限制了联邦POI推荐的发展。为此，本文在极其稀疏的历史轨迹下设计了细粒度的偏好感知个性化联邦 POI 推荐框架，即 PrefFedPOI，以应对上述挑战。具体来说，PrefFedPOI通过结合每个本地客户端内的历史近期偏好和周期性偏好来提取当前时段的细粒度偏好。由于某些时段POI数据极度缺乏，针对模型参数上传设计了数据量感知选择性策略。此外，提出了一种带有强化学习的性能增强聚类机制，以捕获所有客户之间的偏好相关性，以鼓励积极的知识共享。此外，还设计了聚类教师网络，通过聚类指导来提高效率。在两个不同的现实世界数据集上进行了广泛的实验，以证明所提出的 PrefFedPOI 与最先进技术相比的有效性。特别是，个性化的 PrefFedPOI 在数据稀疏的客户端中平均可以实现 7% 的准确率提升。

Manipulating Federated Recommender Systems: Poisoning with Synthetic Users and Its Countermeasures

Authors: Wei Yuan; Quoc Viet Hung Nguyen; Tieke He; Liang Chen; Hongzhi Yin

Conference : SIGIR '23: The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Url: https://dl.acm.org/doi/10.1145/3539618.3591722

Abstract: Federated Recommender Systems (FedRecs) are considered privacypreserving techniques to collaboratively learn a recommendation model without sharing user data. Since all participants can directly influence the systems by uploading gradients, FedRecs are vulnerable to poisoning attacks of malicious clients. However, most existing poisoning attacks on FedRecs are either based on some prior knowledge or with less effectiveness. To reveal the real vulnerability of FedRecs, in this paper, we present a new poisoning attack method to manipulate target items’ ranks and exposure rates effectively in the top- recommendation without relying on any prior knowledge. Specifically, our attack manipulates target items’ exposure rate by a group of synthetic malicious users who upload poisoned gradients considering target items’ alternative products. We conduct extensive experiments with two widely used FedRecs (Fed-NCF and Fed-LightGCN) on two real-world recommendation datasets. The experimental results show that our attack can significantly improve the exposure rate of unpopular target items with extremely fewer malicious users and fewer global epochs than state-of-the-art attacks. In addition to disclosing the security hole, we design a novel countermeasure for poisoning attacks on FedRecs. Specifically, we propose a hierarchical gradient clipping with sparsified updating to defend against existing poisoning attacks. The empirical results demonstrate that the proposed defending mechanism improves the robustness of FedRecs.

abstractTranslation: 联邦推荐系统（FedRecs）被认为是隐私保护技术，可以在不共享用户数据的情况下协作学习推荐模型。由于所有参与者都可以通过上传梯度直接影响系统，因此 FedRecs 很容易受到恶意客户端的投毒攻击。然而，大多数现有的针对 FedRecs 的中毒攻击要么基于一些先验知识，要么效率较低。为了揭示 FedRecs 的真正漏洞，在本文中，我们提出了一种新的中毒攻击方法，可以在不依赖任何先验知识的情况下有效地操纵 Top-K 推荐中目标项目的排名和暴露率。具体来说，我们的攻击通过一群合成恶意用户操纵目标项目的暴露率，这些用户考虑目标项目的替代产品上传中毒梯度。我们使用两个广泛使用的 FedRec（Fed-NCF 和 Fed-LightGCN）在两个真实世界的推荐数据集上进行了广泛的实验。实验结果表明，与最先进的攻击相比，我们的攻击可以显着提高不受欢迎的目标项目的曝光率，并且恶意用户数和全局时期数极少。除了披露安全漏洞之外，我们还设计了一种针对 FedRecs 中毒攻击的新颖对策。具体来说，我们提出了一种具有稀疏更新的分层梯度裁剪，以防御现有的中毒攻击。实证结果表明，所提出的防御机制提高了 FedRecs 的稳健性。

Notes:

PDF

AAAI'2023

Untargeted Attack against Federated Recommendation Systems via Poisonous Item Embeddings and the Defense

Authors: Yang Yu; Qi Liu; Likang Wu; Runlong Yu; Sanshi Lei Yu; Zaixi Zhang

Journal : Proceedings of the AAAI Conference on Artificial Intelligence

Url: https://ojs.aaai.org/index.php/AAAI/article/view/25611

Abstract: Federated recommendation (FedRec) can train personalized recommenders without collecting user data, but the decentralized nature makes it susceptible to poisoning attacks. Most previous studies focus on the targeted attack to promote certain items, while the untargeted attack that aims to degrade the overall performance of the FedRec system remains less explored. In fact, untargeted attacks can disrupt the user experience and bring severe financial loss to the service provider. However, existing untargeted attack methods are either inapplicable or ineffective against FedRec systems. In this paper, we delve into the untargeted attack and its defense for FedRec systems. (i) We propose ClusterAttack, a novel untargeted attack method. It uploads poisonous gradients that converge the item embeddings into several dense clusters, which make the recommender generate similar scores for these items in the same cluster and perturb the ranking order. (ii) We propose a uniformity-based defense mechanism (UNION) to protect FedRec systems from such attacks. We design a contrastive learning task that regularizes the item embeddings toward a uniform distribution. Then the server filters out these malicious gradients by estimating the uniformity of updated item embeddings. Experiments on two public datasets show that ClusterAttack can effectively degrade the performance of FedRec systems while circumventing many defense methods, and UNION can improve the resistance of the system against various untargeted attacks, including our ClusterAttack.

abstractTranslation: 联邦推荐（FedRec）可以在不收集用户数据的情况下训练个性化推荐器，但去中心化的性质使其容易受到中毒攻击。之前的大多数研究都集中在促进某些项目的有针对性的攻击上，而旨在降低 FedRec 系统整体性能的非针对性攻击仍然很少被探讨。事实上，无针对性的攻击可能会破坏用户体验并给服务提供商带来严重的经济损失。然而，现有的非针对性攻击方法对于 FedRec 系统要么不适用，要么无效。在本文中，我们深入研究了 FedRec 系统的非针对性攻击及其防御。 (i)我们提出ClusterAttack，一种新颖的非目标攻击方法。它上传有毒梯度，将项目嵌入收敛到几个密集的集群中，这使得推荐器为同一集群中的这些项目生成相似的分数，并扰乱排名顺序。 (ii) 我们提出了一种基于一致性的防御机制（UNION）来保护 FedRec 系统免受此类攻击。我们设计了一个对比学习任务，将项目嵌入规范化为均匀分布。然后，服务器通过估计更新项目嵌入的均匀性来过滤掉这些恶意梯度。在两个公共数据集上的实验表明，ClusterAttack可以有效降低FedRec系统的性能，同时规避许多防御方法，而UNION可以提高系统对各种非针对性攻击的抵抗力，包括我们的ClusterAttack。

Notes:

PDF code

Win-Win: A Privacy-Preserving Federated Framework for Dual-Target Cross-Domain Recommendation

Authors: Gaode Chen; Xinghua Zhang; Yijun Su; Yantong Lai; Ji Xiang; Junbo Zhang; Yu Zheng

Journal : Proceedings of the AAAI Conference on Artificial Intelligence

Url: https://ojs.aaai.org/index.php/AAAI/article/view/25531

Abstract: Cross-domain recommendation (CDR) aims to alleviate the data sparsity by transferring knowledge from an informative source domain to the target domain, which inevitably proposes stern challenges to data privacy and transferability during the transfer process. A small amount of recent CDR works have investigated privacy protection, while they still suffer from satisfying practical requirements (e.g., limited privacy-preserving ability) and preventing the potential risk of negative transfer. To address the above challenging problems, we propose a novel and unified privacy-preserving federated framework for dual-target CDR, namely P2FCDR. We design P2FCDR as peer-to-peer federated network architecture to ensure the local data storage and privacy protection of business partners. Specifically, for the special knowledge transfer process in CDR under federated settings, we initialize an optimizable orthogonal mapping matrix to learn the embedding transformation across domains and adopt the local differential privacy technique on the transformed embedding before exchanging across domains, which provides more reliable privacy protection. Furthermore, we exploit the similarity between in-domain and cross-domain embedding, and develop a gated selecting vector to refine the information fusion for more accurate dual transfer. Extensive experiments on three real-world datasets demonstrate that P2FCDR significantly outperforms the state-of-the-art methods and effectively protects data privacy.

abstractTranslation: 跨域推荐（CDR）旨在通过将知识从信息丰富的源域转移到目标域来缓解数据稀疏性，这不可避免地对转移过程中的数据隐私和可转移性提出了严峻的挑战。近期的少量CDR工作对隐私保护进行了研究，但仍存在满足实际需求（例如有限的隐私保护能力）和防止潜在的负转移风险的问题。为了解决上述具有挑战性的问题，我们提出了一种新颖且统一的双目标 CDR 隐私保护联邦框架，即 P2FCDR。我们将P2FCDR设计为点对点联盟网络架构，以确保业务合作伙伴的本地数据存储和隐私保护。具体来说，针对联邦环境下CDR中的特殊知识转移过程，我们初始化一个可优化的正交映射矩阵来学习跨域的嵌入变换，并在跨域交换之前对变换后的嵌入采用局部差分隐私技术，从而提供更可靠的隐私保护。此外，我们利用域内嵌入和跨域嵌入之间的相似性，并开发门控选择向量来细化信息融合，以实现更准确的双重传输。对三个真实世界数据集的大量实验表明，P2FCDR 显着优于最先进的方法，并有效保护数据隐私。

KDD'2023

UA-FedRec: Untargeted Attack on Federated News Recommendation

Authors: Jingwei Yi; Fangzhao Wu; Bin Zhu; Jing Yao; Zhulin Tao; Guangzhong Sun; Xing Xie

Conference : Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Url: https://dl.acm.org/doi/10.1145/3580305.3599923

Abstract: News recommendation is essential for personalized news distribution. Federated news recommendation, which enables collaborative model learning from multiple clients without sharing their raw data, is a promising approach for preserving users' privacy. However, the security of federated news recommendation is still unclear. In this paper, we study this problem by proposing an untargeted attack on federated news recommendation called UA-FedRec. By exploiting the prior knowledge of news recommendation and federated learning, UA-FedRec can effectively degrade the model performance with a small percentage of malicious clients. First, the effectiveness of news recommendation highly depends on user modeling and news modeling. We design a news similarity perturbation method to make representations of similar news farther and those of dissimilar news closer to interrupt news modeling, and propose a user model perturbation method to make malicious user updates in opposite directions of benign updates to interrupt user modeling. Second, updates from different clients are typically aggregated with a weighted average based on their sample sizes. We propose a quantity perturbation method to enlarge sample sizes of malicious clients in a reasonable range to amplify the impact of malicious updates. Extensive experiments on two real-world datasets show that UA-FedRec can effectively degrade the accuracy of existing federated news recommendation methods, even when defense is applied. Our study reveals a critical security issue in existing federated news recommendation systems and calls for research efforts to address the issue. Our code is available at https://github.com/yjw1029/UA-FedRec.

abstractTranslation: 新闻推荐对于个性化新闻发布至关重要。联邦新闻推荐可以在不共享原始数据的情况下从多个客户端进行协作模型学习，是保护用户隐私的一种有前景的方法。然而，联邦新闻推荐的安全性仍不清楚。在本文中，我们通过提出一种名为 UA-FedRec 的联邦新闻推荐无目标攻击来研究这个问题。通过利用新闻推荐和联邦学习的先验知识，UA-FedRec 可以在少量恶意客户端的情况下有效降低模型性能。首先，新闻推荐的有效性高度依赖于用户建模和新闻建模。我们设计了一种新闻相似性扰动方法，使相似新闻的表示更加接近中断新闻建模，并提出了一种用户模型扰动方法，使恶意用户更新与良性更新相反，从而中断用户建模。其次，来自不同客户的更新通常根据其样本大小进行加权平均值聚合。我们提出了一种数量扰动方法，将恶意客户端的样本量扩大在合理范围内，以放大恶意更新的影响。对两个真实世界数据集的大量实验表明，即使应用防御，UA-FedRec 也可以有效降低现有联邦新闻推荐方法的准确性。我们的研究揭示了现有联邦新闻推荐系统中的一个关键安全问题，并呼吁开展研究工作来解决该问题。我们的代码可在 https://github.com/yjw1029/UA-FedRec 获取。

Notes:

PDF CODE

Privacy Matters: Vertical Federated Linear Contextual Bandits for Privacy Protected Recommendation

Authors: Zeyu Cao; Zhipeng Liang; Bingzhe Wu; Shu Zhang; Hangyu Li; Ouyang Wen; Yu Rong; Peilin Zhao

Conference : KDD '23: The 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Url: https://dl.acm.org/doi/10.1145/3580305.3599475

Abstract: Recent awareness of privacy protection and compliance requirement resulted in a controversial view of recommendation system due to personal data usage. Therefore, privacy-protected recommendation emerges as a novel research direction. In this paper, we first formulate this problem as a vertical federated learning problem, i.e., features are vertically distributed over different departments. We study a contextual bandit learning problem for recommendation in the vertical federated setting. To this end, we carefully design a customized encryption scheme named orthogonal matrix-based mask mechanism (O3M). O3M mechanism, a tailored component for contextual bandits by carefully exploiting their shared structure, can ensure privacy protection while avoiding expensive conventional cryptographic techniques. We further apply the mechanism to two commonly-used bandit algorithms, LinUCB and LinTS, and instantiate two practical protocols for online recommendation. The proposed protocols can perfectly recover the service quality of centralized bandit algorithms while achieving a satisfactory runtime efficiency, which is theoretically proved and analysed in this paper. By conducting extensive experiments on both synthetic and real-world datasets, we show the superiority of the proposed method in terms of privacy protection and recommendation performance.

abstractTranslation: 最近人们对隐私保护和合规要求的认识导致了由于个人数据的使用而对推荐系统产生了争议。因此，隐私保护推荐成为一个新颖的研究方向。在本文中，我们首先将这个问题表述为纵向联邦学习问题，即特征纵向分布在不同的部门。我们研究了纵向联邦环境中推荐的上下文强盗学习问题。为此，我们精心设计了一种定制的加密方案，称为基于正交矩阵的掩码机制（O3M）。 O3M机制是通过仔细利用其共享结构为上下文强盗量身定制的组件，可以确保隐私保护，同时避免昂贵的传统加密技术。我们进一步将该机制应用于两种常用的老虎机算法LinUCB和LinTS，并实例化了两个实用的在线推荐协议。所提出的协议能够完美恢复集中式强盗算法的服务质量，同时达到令人满意的运行效率，这在本文中得到了理论证明和分析。通过对合成数据集和真实数据集进行广泛的实验，我们展示了所提出的方法在隐私保护和推荐性能方面的优越性。

PrivateRec: Differentially Private Model Training and Online Serving for Federated News Recommendation

Authors: Ruixuan Liu; Yang Cao; Yanlin Wang; Lingjuan Lyu; Yun Chen; Hong Chen

Conference : Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Url: https://dl.acm.org/doi/10.1145/3580305.3599889

Abstract: Federated recommendation can potentially alleviate the privacy concerns in collecting sensitive and personal data for training personalized recommendation systems. However, it suffers from a low recommendation quality when a local serving is inapplicable due to the local resource limitation and the data privacy of querying clients is required in online serving. Furthermore, a theoretically private solution in both the training and serving of federated recommendation is essential but still lacking. Naively applying differential privacy (DP) to the two stages in federated recommendation would fail to achieve a satisfactory trade-off between privacy and utility due to the high-dimensional characteristics of model gradients and hidden representations. In this work, we propose a federated news recommendation method for achieving better utility in model training and online serving under a DP guarantee. We first clarify the DP definition over behavior data for each round in the pipeline of federated recommendation systems. Next, we propose a privacy-preserving online serving mechanism under this definition based on the idea of decomposing user embeddings with public basic vectors and perturbing the lower-dimensional combination coefficients. We apply a random behavior padding mechanism to reduce the required noise intensity for better utility. Besides, we design a federated recommendation model training method, which can generate effective and public basic vectors for serving while providing DP for training participants. We avoid the dimension-dependent noise for large models via label permutation and differentially private attention modules. Experiments on real-world news recommendation datasets validate that our method achieves superior utility under a DP guarantee in both training and serving of federated news recommendations.

abstractTranslation: 联邦推荐可以潜在地缓解收集敏感数据和个人数据以训练个性化推荐系统时的隐私问题。然而，当由于本地资源限制而不适用本地服务并且在线服务需要查询客户端的数据隐私时，其推荐质量较低。此外，在联邦推荐的训练和服务中理论上的私有解决方案是必要的，但仍然缺乏。由于模型梯度和隐藏表示的高维特性，将差分隐私（DP）简单地应用于联邦推荐的两个阶段将无法在隐私和效用之间实现令人满意的权衡。在这项工作中，我们提出了一种联邦新闻推荐方法，以在 DP 保证下在模型训练和在线服务中实现更好的效用。我们首先澄清联邦推荐系统流程中每轮行为数据的 DP 定义。接下来，我们基于用公共基本向量分解用户嵌入并扰动低维组合系数的思想，在此定义下提出了一种保护隐私的在线服务机制。我们应用随机行为填充机制来降低所需的噪声强度，以获得更好的实用性。此外，我们设计了一种联邦推荐模型训练方法，可以生成有效的、公共的服务基本向量，同时为训练参与者提供DP。我们通过标签排列和差异私人注意模块避免大型模型的维度相关噪声。对现实世界新闻推荐数据集的实验验证了我们的方法在 DP 保证下在联邦新闻推荐的训练和服务方面实现了卓越的实用性。

Notes:

PDF NEWS