Large Model "Right to be Forgotten": How to technically implement it when customers request to "delete my training data"?

Amid tightening global data privacy regulations, the user's "Right to be Forgotten" has evolved from a mere legal provision into a critical technical challenge for LLM engineering teams. Unlike clear deletion commands in traditional relational databases, LLM training is essentially deep lossy compression and full-parameter probabilistic reconstruction. Once sensitive data is "learned," backpropagation transforms it into elusive weight changes across hundreds of billions of parameters, making precise physical data removal a near-paradox in technical logic. Facing retraining costs involving millions of dollars in compute, relying solely on starting from scratch cannot meet high-frequency compliance demands, compelling the industry to explore the frontier of "Machine Unlearning." This article analyzes the underlying logic of this complex engineering challenge, revealing how to achieve targeted "erasure" of specific knowledge and model behavior correction via Gradient Ascent and its variants, without compromising the model's general language capabilities and logical reasoning foundation. This requires not only sophisticated loss function design but also finding a narrow balance between maximizing specific data removal and avoiding "Catastrophic Forgetting." Mastering this technology is essential for enterprises to efficiently address compliance challenges like GDPR and is key to building secure, controllable, and continuously iterative Generative AI systems, marking a shift in LLM governance from crude black-box training to an era of precise parameter surgery.

Why is "Deleting Data" a Complex Engineering Challenge in LLMs?

In traditional software engineering, responding to user "Right to be Forgotten" requests is usually a standardized database operation. Whether it is DELETE FROM table WHERE user_id = 'xyz' in SQL, or key-value removal in NoSQL, data deletion is clear and verifiable at both logical and physical levels. However, when this requirement extends to the training data of Large Language Models (LLMs), engineers face an essentially completely different technical paradox.

Database Logic vs. Neural Network Weights

The core of an LLM is not the storage of information, but the "lossy compression" and probabilistic modeling of information. When a model "learns" a piece of sensitive data (such as a company's private code or a user's personal address) during the pre-training or fine-tuning stage, this data does not exist as an independent record in a specific sector. Instead, through the Backpropagation algorithm, it is "scattered" and diffused into minute numerical changes across hundreds of billions of parameters (Weights).

This is like trying to "delete" a drop of ink mixed into a finished latte. You cannot find this drop of ink through simple retrieval because it has integrated with the physical structure of the whole cup of coffee. On a technical level, Machine Unlearning requires us to precisely locate and reverse these weight updates, which is several orders of magnitude more complex than simple index deletion.

The Cost Trap of "Exact Unlearning"

Theoretically, the only method that can strictly guarantee mathematically that "data has been completely deleted" is Exact Unlearning, which means removing the target data and then retraining the model from scratch (Retraining from Scratch).

However, in engineering practice, this method is almost unfeasible. Training a competitive LLM is extremely expensive. According to industry estimates, training a model of the scale of GPT-4 may involve over $100 million in compute costs. Even for smaller open-source models, training costs are often in the range of tens to hundreds of thousands of dollars. If the model were retrained for every GDPR deletion request received, corporate computing resources would quickly be depleted.

Therefore, research focus in academia and industry has been forced to shift to Approximate Unlearning. This path does not pursue a physical "reset," but adjusts model weights through algorithmic means so that its output behaves "as if it had never seen that data."

Core Engineering Contradiction: Eliminating Influence vs. Maintaining Capability

For AI engineers, the biggest challenge in implementing "unlearning" lies not in destroying memories, but in not damaging the model's general capabilities while destroying specific memories.

This is known as the inverse problem of "Catastrophic Forgetting." If we crudely maximize the loss function (Gradient Ascent) for the target data, the model is likely to destroy its language understanding or logical reasoning capabilities while erasing sensitive information, leading to consequences akin to a "model lobotomy."

Relevant review research points out that LLM unlearning tasks must find an extremely narrow balance point among the following three conflicting goals:

1. Forget Quality: Can the target data truly no longer be induced to generate?
2. Model Utility: Does the model's general question-answering capability decline?
3. Efficiency: The algorithm's running time must be far less than the time for retraining.

Therefore, when business departments relay customer "delete data" requests, what the technical team delivers is often not a simple "delete key," but a complex parameter optimization process aimed at erasing the mathematical projection of specific data points in the neural network at the minimum cost.

Landscape of Mainstream Machine Unlearning Technology Routes

In current large model engineering practices, machine unlearning technology is not a single standard component, but rather differentiates into three mainstream technology routes based on the trade-off between "unlearning quality" and "computational cost." To precisely eliminate the influence of specific data while preserving the model's general capabilities, engineers usually need to choose between Data-driven, Optimization-based, and Model Editing approaches.

The following table summarizes the core logic and engineering pros and cons of these three technology routes:

Why is the LLM Industry Currently Focusing on "Optimization Algorithms"?

Although data-driven methods like SISA provide the strictest privacy guarantees, in large model scenarios with hundreds of billions of parameters (70B+), their storage and training costs are unbearable for engineering teams. Therefore, Optimization-based methods are currently the mainstream choice for LLM unlearning technology.

These methods (especially Gradient Ascent and its variants) attempt to find a balance point between "forgetting specific data" and "retaining general capabilities." For example, IBM research points out that while simple gradient ascent can eliminate data influence like "rewinding," it is also very likely to accidentally harm other capabilities of the model. Therefore, modern optimization algorithms usually introduce a "Retain Set" for constraints, or use technologies such as Task Vectors, to more precisely strip away specific knowledge by calculating the weight difference before and after fine-tuning, without the need for expensive full-scale computation.

Practical Guide: Implementation of Unlearning Algorithms Based on Gradient Ascent

In engineering practice, Retraining from Scratch is often unfeasible due to excessive costs. Therefore, gradient-based optimization algorithms have become the mainstream means of "approximate unlearning." This section will delve into the code logic, analyzing how to use Gradient Ascent and its variants to targetedly erase memories of specific data while retaining the model's general capabilities.

1. Dataset Construction: Forget Set and Retain Set

The first step in implementing machine unlearning is not adjusting parameters, but constructing the correct data pairs. Unlike traditional fine-tuning, we need two key datasets:

• Forget Set ($D_f$): Contains specific knowledge that needs to be deleted (such as the copyrighted book Harry Potter, or specific user PII data).
• Retain Set ($D_r$): General corpus used to maintain the model's language capabilities and logical reasoning abilities (such as Wikipedia samples, general dialogue data).

In Microsoft Research's Who's Harry Potter? experiment, researchers not only used the original text as the forget set but also constructed "Generic Prediction Labels." For example, when the model inputs "Who is Harry Potter?", the goal is no longer to predict "wizard," but to guide the model to predict it as "generic" content (such as "actor" or a common noun), thereby severing the association between the specific entity and the knowledge.

2. Core Algorithm Logic: Reverse Weight Update

Standard model training is Gradient Descent (minimizing loss), while the unlearning process is Gradient Ascent (maximizing the loss of specific data). However, pure gradient ascent leads to model parameter divergence, causing "Catastrophic Forgetting," where the model turns into gibberish.

Therefore, the common practice in the industry is to adopt a Combined Loss Function. While maximizing the forget set loss, we must minimize the retain set loss or constrain parameter drift.

Below is a simplified implementation logic based on PyTorch, demonstrating how to achieve this balance through weighted loss in the training loop:

import torch
from torch.nn import functional as F

def computeunlearningloss(model, batchforget, batchretain, method="gakl"):
    """
    Calculate unlearning loss function
    batchforget: Forget set data containing inputids and labels
    batchretain: Retain set data containing inputids and labels
    """

# 1. Forward pass: Calculate Loss for Forget Set
    outputsforget = model(inputids=batchforget['inputids'], labels=batchforget['labels'])
    lossforget = outputsforget.loss  # Usually CrossEntropyLoss

# 2. Forward pass: Calculate Loss for Retain Set (used to maintain general capabilities)
    outputsretain = model(inputids=batchretain['inputids'], labels=batchretain['labels'])
    lossretain = outputsretain.loss

# 3. Define optimization objective
    if method == "gradientascent":
        # Naive Gradient Ascent: Extremely unstable, prone to destroying the model
        totalloss = -lossforget

elif method == "gaplusdescent":
        # Gradient Ascent + Gradient Descent: Reinforce general data while forgetting specific data
        # alpha is a hyperparameter controlling the unlearning intensity
        alpha = 0.1
        totalloss = -lossforget + alpha * lossretain

elif method == "gakl":
        # Recommended: Gradient Ascent + KL Divergence Constraint
        # Constrain model distribution not to deviate too far from the original model (Ref Model)
        with torch.nograd():
            refoutputs = refmodel(batchretain['inputids'])

probscurrent = F.logsoftmax(outputsretain.logits, dim=-1)
        probsref = F.softmax(refoutputs.logits, dim=-1)

losskl = F.kldiv(probscurrent, probsref, reduction='batchmean')
        totalloss = -lossforget + losskl

return totalloss

# Pseudo-code training loop
optimizer.zerograd()
loss = computeunlearningloss(model, batchforget, batchretain, method="gakl")
loss.backward()
optimizer.step()

3. Key Engineering Details and Guide to Avoiding Pitfalls

When actually deploying open-source solutions like Unlearning_LLM or GNM, engineers need to pay attention to the following points:

1. Learning Rate is extremely sensitive: The learning rate for the unlearning process usually needs to be smaller than that for fine-tuning (e.g., $1e^{-5}$ or lower). An excessively large learning rate will instantly destroy the model's syntactic structure.
2. Gradient Accumulation: Since the forget set is usually small, to obtain stable gradient estimates, it is recommended to use a larger number of gradient accumulation steps. For example, when processing Llama-7b level models, although the Batch Size is small, the accumulation steps can be set to 16 or 32.
3. Early Stopping: Unlearning is not like training where lower Loss is always better. Two metrics need to be monitored: Forget Accuracy (should decrease) and Retain Accuracy (should remain stable). Once the performance of the retain set begins to decline significantly, training should be stopped immediately.
4. Token Alignment Issue: As mentioned in the Who's Harry Potter? paper, different Tokenizers treat the same word differently (e.g., whether there is a space before "Harry"), which may lead to failure in locating the unlearning target. When pre-processing data, it is essential to ensure that the Target Token strictly matches the Token in the model's vocabulary.

Through the above process, we are actually performing a "minimally invasive surgery" within the model's parameter space: excising the activation weights of specific neural circuits while using the retain set as "sutures" to ensure that the surrounding knowledge structure does not collapse.

Data Preparation: Building Forget Sets and Retain Sets

Before implementing the "unlearning" operation for large models, the most critical engineering step is not adjusting the learning rate, but building high-quality datasets. Many engineers make a typical mistake when trying for the first time: using only the target deletion data (Forget Set) for Gradient Ascent. This "purely destructive" approach easily leads to Catastrophic Forgetting—the model not only forgets the specified information but also destroys basic language capabilities, resulting in gibberish output or logical collapse.

To protect the model's general capabilities while "precisely excising" memories, we need to prepare two types of data:

1. Building the Forget Set

This is the specific data that needs to be erased from the model weights. Depending on the actual business scenario, the Forget Set usually takes two forms:

• Raw Corpus Segments: For copyright compliance scenarios (such as "deleting all content related to Harry Potter"), directly use the raw text chunks corresponding to the model training.
• Specific QA Pairs: For privacy or misinformation scenarios. For example, in the TOFU (Task of Fictitious Unlearning) benchmark, researchers built a synthetic QA dataset regarding fictitious authors, aiming to let the model forget biographical details of specific individuals without affecting cognition of other entities or general facts.

2. Building the Retain Set

The function of the Retain Set is to act as an "anchor." By introducing KL divergence constraints or standard language modeling loss into the loss function, it forces the model to maintain parameter stability in non-target regions. The following strategies are usually adopted to build the Retain Set:

• General Distribution Sampling: Select general corpora consistent with the pre-training data distribution (such as subsets of Wikipedia or BookCorpus). This effectively prevents the model from destroying basic grammar and logical capabilities when updating weights.
• Neighboring Adversarial Samples: This is a more advanced engineering technique. If the goal is to make the model forget "Harry Potter," the Retain Set should not just be news corpora but should also include The Lord of the Rings or general fantasy novels. This forces the model to learn to distinguish between "specific Harry Potter knowledge" and "general fantasy literature concepts" during gradient updates, thereby achieving more precise unlearning.

Example Walkthrough: Taking the "Harry Potter" Removal Experiment as an Example

In relevant research from NeurIPS 2024, to verify the effectiveness of unlearning algorithms, the research team adopted the following data configuration:

• Forget Set ($D_{fgt}$): Directly used the raw text of Harry Potter and the Sorcerer's Stone. The goal was to make the model behave as if it had never read the book when asked about relevant plots.
• Retain Set ($D_{nor}$): Used the BookCorpus dataset as a general benchmark. Since BookCorpus is also book text, its format is similar to the Forget Set, which minimizes interference caused by data format differences, ensuring the model only forgets content knowledge rather than causing parameter drift due to text style changes.

In practical engineering implementation, the size of the Retain Set usually does not need to match the entire pre-training data; selecting a scale of 10% to 100% of the Forget Set is generally sufficient to achieve a significant regularization effect, thereby striking a balance between computational cost and model stability.

Core Code Logic: Reverse Gradient Update and KL Divergence Constraints

After constructing the Forget Set and Retain Set, the core of the technical implementation lies in designing a loss function that can simultaneously satisfy the dual goals of "forgetting" and "protecting." Unlike conventional Fine-tuning, which aims to minimize prediction error, Machine Unlearning typically employs a Gradient Ascent strategy to maximize the model's entropy on specific data, while utilizing KL Divergence (Kullback-Leibler Divergence) to constrain the model's distribution on general knowledge from drifting drastically.

Loss Function Design Principles

To utilize standard optimizers (like AdamW) for backpropagation in frameworks such as PyTorch, we need to construct a scalar loss value total_loss that achieves the following logic during minimization:

1. Forget Term (Gradient Ascent): We want the model's predictions on the Forget Set to become "poor" (i.e., the Loss generated by this data increases). In code, this typically manifests as minimizing the negative CrossEntropy Loss.
2. Retain Term (KL Divergence): We want the model's performance on the Retain Set to remain as consistent as possible with the original model (Reference Model) before unlearning.

The formula can be abstracted as:
$$ \mathcal{L}{total} = -\mathcal{L}{forget} + \beta \cdot \mathcal{L}_{retain} $$

Where $\beta$ is a hyperparameter used to balance forgetting intensity and model stability.

PyTorch Implementation Code Example

The following code demonstrates a typical unlearning training loop (Unlearning Loop). To prevent Model Collapse, we introduce a frozen parameter ref_model as an anchor.

import torch
import torch.nn.functional as F
from torch.optim import AdamW

def computeunlearningloss(model, refmodel, batchforget, batchretain, beta=0.1):
    """
    Calculate the hybrid loss function for machine unlearning
    model: The model currently undergoing unlearning training (requiresgrad=True)
    refmodel: Copy of the original model, parameters frozen (requiresgrad=False)
    batchforget: Contains inputids and labels for data to be forgotten
    batchretain: Contains inputids for data to be retained
    beta: Weight coefficient for the KL divergence term
    """

# 1. Calculate loss for the Forget Set (we want to maximize this loss)
    # Use standard Causal Language Modeling Loss (CrossEntropy)
    outputsforget = model(
        inputids=batchforget['inputids'], 
        labels=batchforget['labels']
    )
    lossforget = outputsforget.loss  # This is a positive scalar

# 2. Calculate KL divergence constraint for the Retain Set
    # Goal: Make the current model's output distribution on general data as close as possible to the original model
    with torch.nograd():
        refoutputs = refmodel(inputids=batchretain['inputids'])
        reflogits = refoutputs.logits

outputsretain = model(inputids=batchretain['inputids'])
    currentlogits = outputsretain.logits

# Use logsoftmax and softmax to calculate KL Div
    # Note: F.kldiv expects logtarget=False by default (target is probability distribution, not log probability)
    # But for numerical stability, usually input values after logsoftmax
    lossretain = F.kldiv(
        F.logsoftmax(currentlogits, dim=-1),
        F.softmax(reflogits, dim=-1),
        reduction='batchmean'
    )

# 3. Combine total loss
    # The key here is the negative sign: minimizing (-lossforget) is equivalent to maximizing lossforget (gradient ascent)
    totalloss = -lossforget + beta * lossretain

return totalloss

# Training loop pseudo-code
# model = ... (load target model)
# refmodel = ... (load original model and execute refmodel.eval())
# optimizer = AdamW(model.parameters(), lr=1e-5) # Note: Learning rate is usually smaller than fine-tuning

# for step, (batchf, batchr) in enumerate(dataloader):
#     optimizer.zerograd()
#     loss = computeunlearningloss(model, refmodel, batchf, batchr)
#     loss.backward()
#     optimizer.step()

Key Details and Engineering Pitfalls

1. The Anchoring Role of KL Divergence:
   The existence of ref_model in the code is crucial. Without this constraint (i.e., $\beta=0$), pure gradient ascent would cause model parameters to diverge rapidly, causing the model to not only forget the target data but also lose language generation capabilities and start outputting gibberish. In academia, this is known as "catastrophic forgetting" or "model collapse." By calculating the KL divergence on the Retain Set, we force the model to retain its cognition of general language patterns when modifying parameters.
2. Selection of Learning Rate:
   When performing "destructive" updates (such as gradient ascent), model parameters are extremely sensitive. Engineering practice suggests using a smaller learning rate than conventional SFT (Supervised Fine-tuning) (e.g., 1e-5 or 5e-6). An excessively large learning rate will instantly destroy the Transformer's attention mechanism weights, leading to irreversible damage to the model.
3. Gradient Clipping:
   Since we are maximizing the loss, the norm of the gradients can become very large. Adding torch.nn.utils.clipgradnorm(model.parameters(), maxnorm=1.0) before optimizer.step() is a standard operation to prevent numerical instability.

Through this mechanism of "anchoring in the retain zone and pushing away in the forget zone," we can mathematically achieve the erasure of specific data traces without retraining the entire large model. As pointed out by studies such as LLM unlearning via loss adjustment, this loss-adjustment-based method is far superior to retraining in terms of efficiency and is the mainstream path for implementing the "Right to be Forgotten" in the current industry.

Evaluation Metrics: How to Prove the Model Has Truly "Forgotten"?

In traditional database operations, after the DELETE command is executed, data physically disappears. But in neural networks, data is stored distributively in the form of weights, making the verification of "unlearning" a highly complex statistical problem. Relying solely on the model no longer outputting specific sentences is not enough; we need to introduce a Dual-Metric Framework to verify Efficacy while ensuring the model's Utility remains uncompromised.

1. Efficacy Metrics: Verifying the Depth of "Unlearning" (Efficacy)

The core of efficacy evaluation lies in proving that when the model faces the deleted data (Forget Set), its performance degrades to an "untrained" or "random guessing" state.

• Verbatim Memorization Elimination:
  This is the most intuitive test. For text fragments in the training data (such as copyrighted books), we input a prefix and observe the content the model continues to write. If the model can verbatim restore the subsequent text, unlearning has failed.
• • Metric: Use ROUGE-L or BLEU scores to calculate the overlap between the model-generated content and the original training data. Ideally, this score should drop significantly after unlearning.
  • Case: In NeurIPS 2024 research on LLM unlearning, researchers used "Harry Potter" as a test set. The original model could accurately continue the text when receiving the prompt "Harry lit a lamp..."; whereas the model processed by Gradient Ascent outputted "I can't assist it" or irrelevant gibberish, proving the elimination of verbatim memory.
• Knowledge Memorization Elimination:
  The model might no longer recite the original text, but still retains the knowledge points within it (e.g., knowing "Harry Potter went to school at Hogwarts").
• • Test Method: Construct a QA set targeting the Forget Set.
  • Metric: Accuracy. In an ideal unlearning state, the model's accuracy in answering related questions should approach random levels, or it should refuse to answer.
• Membership Inference Attack (MIA):
  This is the "gold standard" from a security compliance perspective. Attackers try to judge whether a specific piece of data existed in the training set through the model's output distribution (Logits).
• • Evaluation: Calculate the Attack Success Rate. If the attacker cannot distinguish between the Forget Set and never-before-seen data (Holdout Set) with a probability higher than random guessing (50%), it can be considered that privacy-level unlearning has been achieved. The OpenUnlearning Framework currently integrates various MIA attack algorithms such as LOSS, ZLib, and MinK to quantify this privacy leakage risk.

2. Limitations and General Capabilities: Preventing "Catastrophic Forgetting" (Locality & Utility)

The most common side effect of unlearning algorithms is "model brain damage"—deleting 1% of data leads to the collapse of the entire model's linguistic or logical capabilities. Therefore, the following metrics must be monitored:

• Retain Set Stability (Retain Set Accuracy):
  The Retain Set usually contains data with a similar distribution to the Forget Set but needs to be preserved (e.g., deleted "Harry Potter", but need to keep "Lord of the Rings").
• • Metric: Calculate the model's Perplexity or ROUGE score on the Retain Set. This metric should maintain minor fluctuations constrained by KL Divergence before and after unlearning, and should not show a significant drop.
• General Benchmarks:
  To ensure the model hasn't lost basic reasoning and language capabilities, regression testing must be performed on standard leaderboards.
• • Metric: MMLU (Massive Multitask Language Understanding) or C-Eval. If an unlearning algorithm causes the MMLU score to plummet by more than 10%, the algorithm is unusable in engineering.
  • Reference Data: According to ICLR 2025 TOFU benchmark research, excellent unlearning algorithms (such as methods combining KL constraints) can control the loss of "Model Utility" within an extremely low range (e.g., 0% - 5%), while crude gradient ascent methods may lead to utility loss as high as 17% or more.

3. Comprehensive Evaluation Benchmarks (Benchmarks)

To standardize this process, academia and industry are forming consensus test sets:

• TOFU (Task of Fictitious Unlearning): This is a specially designed synthetic dataset containing biographies and QA of fictitious authors. It allows engineers to precisely measure "how many fictitious facts were deleted" and "how much real-world knowledge was retained," quantifying algorithm quality by calculating the Forget Quality Gap.
• MUSE (Machine Unlearning Six-Way Evaluation): Provides finer-grained classification, distinguishing between "verbatim memory" and "knowledge memory," and introduces test scenarios for news data to help developers verify unlearning effects on non-fiction data.

In summary, a successful unlearning operation must present a specific shape on a radar chart: scoring extremely low on the Forget Set axis (approaching random), while almost overlapping with the original model on the Retain Set and General Task axes.

Current Limitations and Engineering Recommendations

When transforming "Machine Unlearning" from academic papers into deployed features in a production environment, engineers must face a core contradiction: the trade-off between efficiency and certainty. Although Retraining from Scratch is considered the gold standard for achieving "unlearning," mathematically guaranteeing that data no longer exists in the model distribution, its high computational and time costs are often unacceptable in actual business scenarios.

1. Technical Limitations: Probabilistic Erasure and "Knowledge Residue"

Current Approximate Unlearning algorithms are essentially probabilistic. Unlike the traditional database DELETE operation, executing Gradient Ascent or preference optimization in neural networks does not guarantee that traces of the data are physically cleared.

• Lack of Mathematical Guarantees: Most efficient algorithms (such as GA, NPO) can only ensure that the model's output under specific prompts no longer contains the target information, but cannot prove that the model parameters $\theta_{unlearned}$ are completely identical in distribution to the model parameters $\theta_{retrained}$ that have never seen the data. This discrepancy means attackers might recover the "deleted" information through carefully designed Adversarial Attacks or Membership Inference Attacks (MIA).
• Knowledge Trace: This is the thorniest engineering problem. Even if the model can no longer recite a specific copyrighted article, the syntactic structures, logical associations, or domain-specific vocabulary distributions acquired during training may still remain. For example, deleting the entity concept of "Harry Potter" is relatively easy, but completely stripping away the corpus's implicit contribution to the model's construction of a "magical worldview" is extremely difficult.

2. Practical Recommendations for Engineers

Addressing the above limitations, when designing enterprise-level large model data compliance systems, the following engineering strategies are recommended:

Prioritize Parameter-Efficient Fine-Tuning (PEFT) for Unlearning

Do not directly modify the full weights of the base model. It is recommended to build the unlearning process based on LoRA (Low-Rank Adaptation).

• Advantages: By training a negative LoRA Adapter for a specific unlearning request (or a batch of requests), targeted unlearning can be achieved without destroying the general capabilities of the base model. If the unlearning effect leads to model capability collapse, one simply needs to unload the Adapter without rolling back the entire model.
• Resources: Academia has proven that Parameter-Efficient Unlearning can run efficiently on a single GPU, significantly reducing implementation costs.

Establish Strict Version Control and "Pre-Unlearning" Snapshots

"Catastrophic Forgetting" is the biggest side effect of unlearning algorithms—while attempting to forget a piece of data, the model often destroys its general reasoning ability or linguistic fluency.

• Engineering Standards: Before executing any backpropagation operations, the current model weights must be version-frozen.
• Evaluation Pipeline: Establish an automated test set that not only detects whether the target data has been forgotten (e.g., perplexity increase) but also monitors the decline in general capabilities through benchmarks like MMLU. Only when the loss of general capability is within a threshold (e.g., <2%) is the new version allowed to be released.

Manage Expectations: Distinguish "Physical Deletion" from "Unmemorization"

When communicating with legal departments or clients, the boundaries of technical terms must be clarified.

• Unlearning vs. Unmemorization: For copyright compliance scenarios, the goal is usually to prevent the model from "verbatim regurgitation" of training data. In this case, methods similar to Microsoft Obliviate can be used, focusing on eliminating Verbatim Memorization rather than promising that the model is completely unaffected by the data.
• Compliance Language: It is recommended to define the Service Level Agreement (SLA) as "best-effort probabilistic shielding" rather than "absolute physical deletion" to avoid legal risks caused by technical infeasibility.