PIPEDA-compliant AI: a privacy checklist and AI use policy for employees

Someone on your team is probably already using an AI tool for work, whether or not you have approved one. That is fine, and it is often useful. The risk is not the technology. It is what goes into it, where that information ends up, and who is accountable for what comes out.

This checklist is for Canadian small businesses that handle customer information. It covers which law applies, how to sort your data into tiers, what to ask any AI vendor, what to tell customers, what to write in a staff policy, and what to do if something goes wrong. It is general information, not legal advice, and following any checklist does not make a business compliant by itself. For sensitive data or regulated work, get advice from a privacy professional.

Two cases that show the risks

The first is about what goes in. In April 2023, according to a company memo reported by Bloomberg, Samsung engineers accidentally leaked internal source code by uploading it to ChatGPT. In May 2023, Samsung temporarily restricted generative AI tools on company-owned devices and on its internal networks. The memo's worry was that data sent to these tools is stored on external servers, where it is hard to retrieve or delete and could be disclosed to other users. The leak was accidental. Nobody needs bad intentions for confidential information to end up in a tool the business does not control.

The second is about what comes out. In Moffatt v. Air Canada, decided in February 2024, a British Columbia tribunal held the airline responsible for wrong information its website chatbot gave a customer about bereavement fares. The chatbot suggested he could apply for the reduced fare after traveling, which the airline's policy did not allow. Air Canada argued it could not be held liable for what its chatbot said, which the tribunal read as suggesting the chatbot was a separate legal entity, and pointed out that the correct policy appeared on another page of its website. The tribunal rejected both arguments. It found the airline had not taken reasonable care to make sure its chatbot was accurate and ordered it to pay $812.02: $650.88 in damages for negligent misrepresentation, plus interest and tribunal fees. The lesson for small businesses is simple: a tool that speaks to your customers speaks for you.

Which law applies

The federal Personal Information Protection and Electronic Documents Act, known as PIPEDA, covers private-sector organizations across Canada that collect, use, or disclose personal information in the course of commercial activity. Alberta, British Columbia, and Quebec have their own private-sector laws that the federal government has declared substantially similar, and organizations subject to them are generally exempt from PIPEDA for activities within those provinces. PIPEDA still applies to personal information that crosses provincial or national borders, and to federally regulated businesses such as banks, telecommunications companies, and transportation companies. Health information has separate rules in several provinces, and four of those laws (in Ontario, New Brunswick, Nova Scotia, and Newfoundland and Labrador) have also been declared substantially similar.

PIPEDA is built on ten fair information principles: accountability, identifying purposes, consent, limiting collection, limiting use, disclosure, and retention, accuracy, safeguards, openness, individual access, and challenging compliance. They apply to AI tools the same way they apply to a filing cabinet. If you collect personal information for one purpose, you generally cannot start using it for a new purpose without further consent, and you remain responsible for it when a vendor handles it on your behalf.

In December 2023, the federal, provincial, and territorial privacy commissioners published nine principles for generative AI: legal authority and consent, appropriate purposes, necessity and proportionality, openness, accountability, individual access, limiting collection, use, and disclosure, accuracy, and safeguards. They explain how existing law applies to generative AI. They are not new legislation.

Quebec deserves its own note. Law 25 modernized the province's private-sector privacy law with tougher enforcement. For a business, the Commission d'accès à l'information can impose administrative monetary penalties of up to $10 million or 2 percent of worldwide turnover, whichever is greater. Separately, a court can impose fines of $15,000 to $25 million or 4 percent of worldwide turnover, whichever is greater, in penal proceedings the Commission can bring, and fines double for a repeat offence. Obligations include naming a person in charge of protecting personal information (by default the head of the business, who can delegate in writing) and publishing their title and contact details. Before personal information is communicated outside Quebec, which includes other provinces, the business must carry out a privacy impact assessment, and the same applies when it hands the information to a vendor outside Quebec to handle on its behalf. Businesses must also tell people when a decision about them is based exclusively on automated processing. If you have US customers, state laws may also apply, and they differ widely.

What "PIPEDA-compliant AI" means, and what it does not

You will see AI tools described as "PIPEDA compliant." PIPEDA is a law that places obligations on organizations that handle personal information, not a certificate a product can hold. When a vendor says its tool is compliant, that is a statement about the vendor's own practices. It does not settle whether your use of the tool is compliant.

The reason is the accountability principle. Under clause 4.1.3 of Schedule 1 to the Act, an organization is responsible for personal information in its possession or custody, including information transferred to a third party for processing, and it must use contractual or other means to provide a comparable level of protection while the third party processes it. If an AI vendor processes your customers' information, you stay responsible for it, and the contract is where you protect it.

In practice, when a vendor claims compliance, ask it to say in writing what that means: where the data is stored, how long it is kept, whether your inputs train its models, who can see them, and what happens if there is a breach. The vendor checklist in step 3 turns those questions into a list.

Step 1: Find out what personal information you hold

Before you decide what AI can touch, write down what you have. A one-page inventory is enough.

The "Unknown" entries are the point of the exercise. Ask your team how they use AI today. People rarely hide it. They just have not been asked.

Step 2: Sort your data into three tiers

Minimizing data does most of the work. A tool that drafts a polite reply rarely needs the customer's full name, address, and phone number. Strip out what the task does not need before you paste anything in.

Step 3: Vet every tool before it touches amber data

Consumer and business plans of the same product often have different data terms, so read the terms for the plan you actually use, as they stand today. Terms change. Ask these questions and keep the answers.

1. Does the vendor use our inputs and outputs to train its models? Can we opt out, and is the opt-out the default on our plan?
2. How long is our data kept, and can we delete it on request?
3. Where is it stored and processed? If outside Canada, is that disclosed in the contract and in our privacy policy?
4. Which other companies (subprocessors) can see the data?
5. Who at the vendor can access our data, and under what circumstances?
6. Does the tool support access controls, so only approved staff can use it, and activity logs?
7. Will the vendor tell us promptly if there is a security incident affecting our data, and is that written into the contract?
8. What happens to our data if we cancel?
9. Does the vendor have a written data processing agreement we can sign?

If a vendor cannot answer these in plain language, treat the tool as red-tier until it can.

Step 4: Tell people, and keep to the purpose

Two of the ten principles do the most work here: identifying purposes and consent. Customers should be able to understand from your privacy policy what you do with their information, including that you use service providers and, where it matters, AI tools. Update the policy if it does not say so.

• Do not use information collected for one purpose, such as booking an appointment, to train or feed a tool for an unrelated purpose without fresh consent.
• If a chatbot or assistant talks to customers, say clearly that it is automated and give a way to reach a person.
• If you record or transcribe calls or meetings with an AI note-taker, tell participants before it starts and give them the chance to object.
• In Quebec, tell people when a decision about them is based exclusively on automated processing, explain it if they ask, and let them submit their comments to someone who can review the decision.

Step 5: Review what comes out

The Air Canada case is the reason this step exists. Anything an AI tool says to a customer is your statement. Set rules for review and write them down.

• A person reviews every AI-drafted message that quotes a price, makes a promise, gives legal or medical information, or responds to a complaint.
• Chatbots and assistants answer only from approved sources, such as your own policies and price list, and hand off to a person when unsure.
• Log outputs so you can see what was said and fix recurring errors.
• A disclaimer helps, but it does not remove responsibility for what the tool says.

Step 6: Put a short policy in writing

A policy that fits on one page will be read. Adapt this one to your business and review it every six months.

1. Approved tools
We use only these AI tools for work: [list]. Do not use other AI tools, or personal accounts, for work tasks.

2. What you can enter
Green information: any approved tool.
Amber information (customer names, contact details, quotes, job notes): approved business-plan tools only, and only what the task needs.
Red information (health details, payment or bank details, ID numbers, passwords, anything under NDA): never, unless [privacy lead] approves in writing.

3. Review
A person checks any AI-drafted message that quotes a price, makes a promise, gives professional advice, or responds to a complaint before it goes to a customer.

4. Transparency
If a customer is talking to an automated assistant, say so and offer a person. Tell people before recording or transcribing calls.

5. Problems
If you think customer information went into the wrong tool, or an AI tool said something wrong to a customer, tell [privacy lead] the same day. You will not be blamed for reporting.

6. Ownership
[Name] is responsible for this policy, the approved tool list, and the privacy inventory. Reviewed every six months.

Step 7: Know what to do if something goes wrong

Under PIPEDA, a breach of security safeguards means the loss of, unauthorized access to, or unauthorized disclosure of personal information resulting from a breach of your safeguards or a failure to establish them. If a breach creates a real risk of significant harm to an individual, you must report it to the Office of the Privacy Commissioner of Canada and notify the affected individuals as soon as feasible, and notify any other organization that may be able to reduce the harm. Significant harm includes bodily harm, humiliation, damage to reputation or relationships, loss of employment or business opportunities, financial loss, identity theft, negative effects on a credit record, and damage to or loss of property. Whether or not a breach meets that threshold, you must keep a record of every breach of security safeguards for 24 months after the day you determine it occurred.

If someone has pasted customer information into a tool that is not approved, treat it as a possible incident and assess it the way you would any other: what information, how sensitive, who could see it, whether it can be deleted, and whether harm is realistic. If you are unsure whether it counts as a reportable breach, get advice quickly rather than waiting.

1. Contain it: stop further use, change passwords, ask the vendor to delete the data.
2. Record it: date, what happened, what information was involved, who might be affected.
3. Assess the risk of significant harm, with advice where needed.
4. Report and notify if the threshold is met.
5. Fix the cause, whether that is the policy, the tool, or training, and write down what you changed.

A 30-day rollout

1. Week 1: ask staff which AI tools they use and for what. Write the data inventory.
2. Week 2: choose the approved tool list and run each tool through the vendor checklist. Cancel or restrict anything that fails.
3. Week 3: update the privacy policy, write the staff policy, and name who is responsible.
4. Week 4: brief the team, set the review rules, and put the incident steps where people can find them.

This is the privacy step of a full AI audit, which also scores which tasks are worth handing to AI at all.

Sources

1. Personal Information Protection and Electronic Documents Act, S.C. 2000, c. 5, sections 2, 10.1 to 10.3 and Schedule 1, Justice Laws Website (Government of Canada)
2. Breach of Security Safeguards Regulations, SOR/2018-64, Justice Laws Website (Government of Canada)
3. Office of the Privacy Commissioner of Canada, "Provincial legislation deemed substantially similar to PIPEDA"
4. Office of the Privacy Commissioner of Canada, "PIPEDA in brief" (application and the ten fair information principles)
5. Office of the Privacy Commissioner of Canada and provincial and territorial counterparts, "Principles for responsible, trustworthy and privacy-protective generative AI technologies," December 2023
6. Office of the Privacy Commissioner of Canada, "What you need to know about mandatory reporting of breaches of security safeguards"
7. Act respecting the protection of personal information in the private sector, CQLR c P-39.1, sections 3.1, 12.1, 17, 90.12 and 91, Légis Québec
8. Osler, "Law 25: A new enforcement scheme for the protection of personal information in the private sector in Quebec"
9. TechCrunch, "Samsung bans use of generative AI tools like ChatGPT after April internal data leak," May 2, 2023
10. CSO Online, "Samsung bans staff AI use over data leak concerns," May 2, 2023 (reporting on the memo first described by Bloomberg)
11. Civil Resolution Tribunal of British Columbia, Moffatt v. Air Canada, 2024 BCCRT 149, February 14, 2024
12. McCarthy Tétrault, "Moffatt v. Air Canada: A misrepresentation by an AI chatbot," February 2024