Are you ready for the Machine Unlearning challenge ?

1 minute read

(This article is a transcript in English of one of my french LinkedIn posts)

Have you heard about the Machine Unlearning challenge?

It's an ethics-oriented challenge, launched by researchers from Google DeepMind. The challenge consists of making the model forget part of its training data. In other words, it involves removing the influence of a subset of the training data from the weights of an already trained model.

Machine unlearning is currently a very active area of research because the question of ethics arises more and more, and it raises new technical challenges with it.

Indeed, when you train a model with a dataset, the learning materializes from the model's point of view by changing its weights. The problem is that if tomorrow you are asked to make the model forget some data (under the right to be forgotten in particular), it's very complicated. There is no obvious mapping between the training data and its materialization as weights.

In addition, there is an issue with maintaining satisfactory performance in production after "forgetting the data", because performance will inevitably be impacted.

You have to find the best compromise between effective forgetting and preserving the model's performance.

For the challenge, you have at your disposal an image classification model (ResNet18) and the CIFAR10 dataset.

You will have to submit an algorithm that takes 3 inputs:

  • The ResNet18 model pre-trained on the entire initial dataset
  • The set of images to forget "forget"
  • The set of images to retain "retain"
    and outputs:

An updated ResNet18 model that has "forgotten" the data from the "forget" set.

How is this evaluated?

Submissions are evaluated using:

Tools similar to MIA - (Membership Inference Attack)
Statistical tests to quantify the difference between the distribution of the "unlearned" model and the original model.
To help you, there is a starter kit with a notebook that goes into detail on the objectives. As an example and for inspiration, the notebook also proposes a simple but suboptimal solution, which consists of fine-tuning the model on the "retain set" for 5 epochs, which has the consequence of sufficiently affecting the weights to blur the "forget set".

Challenge link

The winners will be announced in October 2023.

Is anyone up for the challenge? 🤓

You May Also Enjoy