Using customer data in AI without losing trust or legal cover

The two regimes that apply

Any commerce company using customer data with AI is operating under two overlapping regimes: the GDPR, which governs the processing of personal data, and the EU AI Act, which governs how AI systems are built and deployed. GDPR asks whether you may process the data. The AI Act asks how transparent, safe, and governed the system doing the processing is. You need answers to both.

Rule 1: start from a lawful basis, not from a use case

The most common failure in AI projects is picking a use case first and hunting for a legal basis after the fact. Reverse the order. For every dataset you plan to use, identify the lawful basis you already have: contract, legitimate interest, or consent, and check that the AI use case is compatible with the purpose the customer originally saw.

Reusing checkout data to train a shared vendor model is a new purpose. Personalising the storefront for a returning customer is usually compatible with the original purpose. If in doubt, get written advice.

Rule 2: minimise before you personalise

The best defence against a data incident is not having the data in the first place. Before you send a customer record to a model:

Strip fields the model does not need. Name, email, and address rarely improve a recommender.
Pseudonymise where you can. Use a stable but non-identifying customer key.
Aggregate where you can. Segment-level signals beat individual-level signals for many use cases.
Send prompts, not databases. A well-designed prompt gives the model exactly what it needs and no more.

Rule 3: automated decisions with legal or significant effects need extra care

Under GDPR Article 22, customers have the right not to be subject to a decision based solely on automated processing that has legal or significant effects on them. In commerce, this is most likely to bite in fraud scoring, credit decisions, account suspensions, and price discrimination.

If the model's output is applied automatically and materially affects the customer, you need a human review route, a way for the customer to contest the decision, and a clear explanation of the logic involved.

Rule 4: the surprise test

If you had to explain the specific use of the customer's data on the storefront the next time they logged in, would they be surprised? If yes, either change the use or explain it upfront. Most reputational damage from AI comes not from illegal uses but from uses that were technically legal and completely unexpected.

Rule 5: contract carefully with model vendors

Get a data processing agreement (DPA) before any personal data touches the vendor's system.
Opt out of training on your data by default. If a vendor cannot let you opt out, treat that as a serious risk.
Know every subprocessor. OpenAI, Anthropic, Google, and Azure all have different processing regions and different sub-processor lists.
Require notification before subprocessor changes and reserve the right to object.
Match retention. Your vendor's log retention should not exceed what you have told your customer.

Rule 6: keep humans in the loop where it counts

Human oversight is not window dressing. It works when the person reviewing the AI's output has the authority, the information, and the time to overturn it. If any of those three is missing, the "human in the loop" is a signature machine.

Rule 7: write it down while you still remember it

For every AI system that touches customer data, keep a one-page record: purpose, lawful basis, data categories, retention, vendor and subprocessors, risk classification, human oversight route, and the date of last review. This is the document a regulator will ask for, and it is the document your next colleague will thank you for.

Back to the Knowledge Hub

Using customer data in AI without losing trust