Dong 2023 - MINE Loss

Revisiting Recommendation Loss Functions through Contrastive Learning

This paper compares several recommendation loss functions like BPR, CCL and introduce two new losses: InfoNCE+ and MINE+.

Setup

Let $\mathcal{U},\mathcal{I}$ denote the user and item sets. We denote that:

• Each user $u\in \mathcal{U}$ has interactions with ${\mathcal{I}}_u^{+}$ items, and has no interactions with the remaining $\mathcal{I}\setminus {\mathcal{I}}_u^{+}$ set of items.
• On the item side, ${\mathcal{U}}_i^{+}$ denotes all users who interacted with item $i$
• We can also denote $r_{ui}=1$ if there was an interaction and $r_{ui}=0$ otherwise

Let the latent embeddings $v_u,v_i$ represent user $u$ and item $i$ respectively. The similarity measure between them is then denoted ${\hat{y}}_{ui}=<v_u,v_i>$.

BPR Loss

The most widely used loss is Bayesian Personalized Ranking:

$$\begin{array}{rl}{\mathcal{L}}_{BPR}& ={\mathbb{E}}_u{\mathbb{E}}_{i\sim p_u^{+}}\sum _{j\sim p_i}^N-\log \sigma \left({\hat{y}}_{ui}-{\hat{y}}_{uj}\right)\\ & ={\mathbb{E}}_u{\mathbb{E}}_{i\sim p_u^{+}}\sum _{j\sim p_i}^N-\log \left(1+exp({\hat{y}}_{uj}-{\hat{y}}_{ui})\right)\end{array}$$

Note that for each user, we take the expectation over the set of items relevant to him. We then sample $N$ negatives from the overall item distribution (usually uniformly at random).

Softmax Loss

One common approach is to model $P(I=i|U=u)$ as an extreme classification problem where $\mathcal{I}$ is a very large set. The probability may then be modeled as a softmax: $$P(I=i|U=u)=\frac{e^{v_u^T{v}_i}}{{\sum }_{j\in \mathcal{I}}{e}^{v_u^T{v}_j}}$$

In practice, it is infeasible to compute over the large item set, so we sample negative candidates for the denominator. The sampling is then corrected via importance weighting.

• Covington 2016 for such an approach.
• Jean 2014 for importance weighting correction

Using this approach, the loss may be formulated as: $${\mathcal{L}}_{soft}=-{\mathbb{E}}_u\log \sum _{i\in {\mathcal{I}}_u^{+}}\frac{e^{{\hat{y}}_{ui}}}{e^{{\hat{y}}_{ui}}+{\sum }_{j\sim p_u^{-}}^N{e}^{{\hat{y}}_{uj}}}$$

Note that $p_u^{-}$ is a negative sampling distribution for each user $u$, and is typically implemented as $p$ which is based on item popularity.

Contrastive Learning Loss (InfoNCE)

InfoNCE loss looks very similar to the sampled softmax loss, although the motivation is different. The key idea is to pull similar points closer and push dissimilar points apart. InfoNCE loss is the most famous contrastive learning loss:

$$\begin{array}{rl}{\mathcal{L}}_{info}& =-{\mathbb{E}}_u\sum _{i\in {\mathcal{I}}_u^{+}}\log \frac{e^{{\hat{y}}_{ui}}}{e^{{\hat{y}}_{ui}}+{\sum }_{j\sim p_i^{-}}^N{e}^{{\hat{y}}_{uj}}}\\ & =-{\mathbb{E}}_u\sum _{i\in {\mathcal{I}}_u^{+}}\left[{\hat{y}}_{ui}-\log \left(e^{{\hat{y}}_{ui}}+\sum _{j\sim p_i^{-}}^N{e}^{{\hat{y}}_{uj}}\right)\right]\end{array}$$

Note that the only difference from the sampled softmax loss is that the $\log$ is inside the sum rather than outside. The InfoNCE loss has been shown to maximize the mutual information between user $u$ and item $i$ and minimize mutual information between unrelated pairs.

Empirical Exploration of InfoNCE+

The authors propose an InfoNCE+, which is just adding some hyperparameters to InfoNCE and performing some empirical tuning of these hyperparameters. The InfoNCE+ proposes adding $ϵ$ and $\lambda$:

$$\begin{array}{rl}{\mathcal{L}}_{info+}& =-{\mathbb{E}}_u\sum _{i\in {\mathcal{I}}_u^{+}}\left({\hat{y}}_{ui}-\lambda \cdot \log \mathcal{N}\right)\\ \mathcal{N}& =ϵ\cdot e^{{\hat{y}}_{ui}}+\sum _{j\sim p_u^{-}}^N{e}^{{\hat{y}}_{uj}}\end{array}$$

Empirically, the authors find that setting $ϵ=0$ and $\lambda =1.1$ usually works best (tbh, the empirical evidence is not super convincing).

Theoretical Support for Removing Positive term from Denominator

As we can see, setting $ϵ=0$ effectively removes the positive term $e^{{\hat{y}}_{ui}}$ from the denominator of the loss. This makes intuitive sense as it would constrain ${\hat{y}}_{ui}$ from increasing which is what we want.

This has theoretical backing as well, as explored in Decoupled Contrastive Learning - Yeh 2022. The DCL paper also shows that removing the positive term from the denominator leads to more stable training and less hyperparameter sensitivity.

The DCL loss is thus: $${\mathcal{L}}_{DCL}=-{\mathbb{E}}_u\sum _{i\in {\mathcal{I}}_u^{+}}\left({\hat{y}}_{ui}-\log \sum _{j\sim p_u^{-}}^N{e}^{{\hat{y}}_{uj}}\right)$$

The authors also show that this "decoupling" is also justified from the Mutual Information Neural Estimator perspective. Specifically, the MINE paper shows that we can estimate the true mutual information between each user $u$ and item $i$ by the following optimization problem:

$$\widehat{\mathbb{I}(u,i)}={\text{sup}}_{(v_u;v_i)}{\mathbb{E}}_{p_{u,i}}({\hat{y}}_{ui})-\log {\mathbb{E}}_{p_u\otimes p_i}(e^{{\hat{y}}_{ui}})$$

Intuitively, we want to maximize the above equation over the similarity function parametrized by the embeddings $v_i,v_u$.

• The first term takes an expectation of similarity scores over the joint user, item distribution where an interaction occurs (i.e. positive pairs).
• The second term takes an expectation of exponentiated similarity scores over the product measure of marginal user and item distributions (i.e. assuming independence between user and item distribution).

MINE Loss

The authors then say that a "simple" adaptation of the MINE problem to the recommendation setting is formalized as the MINE loss:

$${\mathcal{L}}_{mine}=-{\mathbb{E}}_u{\mathbb{E}}_{i\sim p_i^{+}}\left[{\hat{y}}_{ui}-\log {\mathbb{E}}_{j\sim p_i}\left(e^{{\hat{y}}_{uj}}\right)\right]$$

Not too sure how this is derived from the above.

They also add a hyperparameter $\lambda$ to control the relative weightage of the positive and negative samples. This results in what they term as MINE+:

$${\mathcal{L}}_{mine+}=-{\mathbb{E}}_u{\mathbb{E}}_{i\sim p_i^{+}}\left[{\hat{y}}_{ui}-\lambda \log {\mathbb{E}}_{j\sim p_i}\left(e^{{\hat{y}}_{uj}}\right)\right]$$

Based on some ablation studies, they find that $\lambda =1.1-1.2$ usually works best.

The paper also offers some lower bound analysis and de-biasing of InfoNCE which I will not delve into for now.