Data Boundaries for AI
Knowing what AI should access before convenience becomes exposure
One question often reveals more about AI risk than any policy, framework, or technical control:
Should this data be allowed into the system?
At first glance, it sounds like a simple yes-or-no question. In reality, it rarely is.
The challenge is not usually whether information can be entered into an AI system. The harder question is what happens after the data gets there:
• What purpose will it be used for?
• What other information can it be combined with?
• How long will it be retained?
• Who else can access it?
• Could it appear somewhere unexpected later?
These questions matter because AI is changing how information moves through organizations.
Revisiting A Recent AI Blueprint Article
In previous AI Blueprint articles, we explored AI risk tiering and why not all AI use cases should be treated the same. A writing assistant carries different risks than an AI system that influences hiring decisions, determines creditworthiness, supports healthcare decisions, or automates business processes.
Risk tiering helps determine how much governance and oversight a use case requires. What it does not tell you is whether the information powering that use case belongs in the system in the first place. That is where data boundaries come in.
Put simply:
Risk tiering helps determine how much governance and oversight an AI use case needs.
Data boundaries determine what information that system should be allowed to access, use, retain, or act upon.
Both are essential.
This distinction matters more today because AI is no longer confined to standalone chatbots. AI is increasingly embedded in email, document libraries, customer platforms, HR systems, collaboration tools, and business applications. Many systems can now retrieve information from multiple sources and take action on a user's behalf.
A chatbot that produces an inaccurate answer can be frustrating. An AI system that retrieves the wrong information, updates a business system, and shares the result before anyone notices creates an entirely different category of risk.
The question is no longer simply: What might the model say?
The more important questions are:
What can the model access?
What information can it combine?
What actions can it take?
As AI systems become more connected to organizational data and business processes, responsible AI is no longer only about model behaviour. It is about defining the boundaries of what an AI system is allowed to reach, retain, influence, and do.
Why "Sensitive" Is the Wrong Start
Most teams reach first for their classification labels (e.g., public, internal, confidential, restricted). That works in a traditional setting where data moves along known pipes to known places. It does not carry over cleanly to AI.
An AI Acceptable Use Policy that bans confidential material from unapproved tools is a sensible floor. It is also not enough, because the same information can be perfectly fine in one place and clearly wrong in another.
For example:
Customer health information may be entirely appropriate inside an approved clinical documentation system but completely inappropriate inside a general-purpose customer service chatbot.
A financial forecast may belong in a secured planning tool, but not in an employee's personal AI account.
The information has not changed. The context has.
That is why the real test is rarely sensitivity alone. The more important question is whether the information is necessary and appropriate for the specific purpose, system, user, and moment in which it is being used.
Organizations Should Think About AI Data Boundaries Across Four Dimensions
| Boundary | The Questions It Answers | Where It Typically Fails |
|---|---|---|
| Context | Is this information necessary and appropriate for this use case? | Information is accessible but not required to perform the task or is inconsistent with the purpose for which it was collected. |
| Knowledge | What may the AI retrieve, combine, or remember? | Information appears outside the intended workflow or carries into unrelated tasks and contexts. |
| Decision | What may the AI recommend, influence, or do? | A support tool gradually begins shaping important decisions without reassessment. |
| Trust | Where does organizational control over the information end? | Information flows into vendor systems, plugins, or subprocessors without sufficient oversight. |
A policy can warn people not to paste sensitive data into public tools. A boundary goes further: for each deployment, it defines what data is permitted or prohibited, which sources the system may use, who may access it, how long information is retained, what actions the system may take, when human approval is required, and how activity is monitored and remediated.
When Access Becomes Exposure
Data boundaries rarely fail because someone deliberately ignores policy. More often, they fail through ordinary convenience, one reasonable-looking shortcut at a time.
An employee pastes a customer email into an AI tool to improve the wording before sending a response. A manager asks an assistant to summarize documents before a meeting. A project team uploads reports to generate a presentation outline. In each case, the task is completed faster and the result appears harmless. While the immediate benefit is visible, the movement of information behind the scenes is not.
The challenge is that AI systems process information differently from traditional business applications. They do not simply store or display information. They can retrieve information, summarize it, connect it to other information, and increasingly use it to support decisions or trigger actions.
As organizations connect AI to business systems, the risk shifts from what a model might say to what it can access behind the scenes.
What Users Hand Over
When most people think about data entering AI systems, they picture prompts. A prompt feels like a conversation, but it is also a mechanism for transferring information.
Through a single prompt, an employee might provide customer details, employee records, legal strategy, source code, financial forecasts, or commercially sensitive plans. The challenge is that sensitive information often exists in the context rather than in obvious identifiers. Consider the following prompt:
"Summarize the attached termination rationale and identify any weaknesses before tomorrow's meeting with counsel."
There may be no credit card number, health card number, or government identifier in the request. Yet the prompt may contain employee information, legal privilege, employment relations issues, and details of an active dispute all at the same time.
This is one reason traditional data loss prevention tools can struggle in AI environments. They are often designed to detect specific patterns or identifiers. AI prompts are different. The sensitivity frequently comes from the context rather than any single piece of data. For example, a name in one prompt, a date in another, and a financial figure in a third may together reveal confidential information that none of the individual prompts would expose on their own.
This is ultimately a purpose-limitation problem. Information that was appropriate in one context can become inappropriate when shared with a different system for a different purpose.
Organizations therefore need clear boundaries around prompts, including:
What information may be entered into AI systems
What information must be removed or redacted first
Whether prompts and responses are retained
Who can access prompt history
Whether memory features are enabled
The same principles already exist in privacy law. Organizations are expected to use personal information only for appropriate and necessary purposes. AI does not change that obligation. It simply creates new ways for information to move.
What the System Fetches
The second pathway is often less visible and considerably more powerful. Modern AI assistants are increasingly connected to organizational knowledge sources, including document repositories, SharePoint sites, customer systems, knowledge bases, and business applications. Before generating a response, they can search those sources and retrieve information that appears relevant to the request.
This capability is commonly referred to as Retrieval-Augmented Generation (RAG). In simple terms, it allows an AI system to answer questions using your organization's information rather than relying solely on what the model learned during training. While the responses are more accurate, more relevant, and more useful, there are governance challenges and risks.
AI Systems Are Designed to Retrieve What Is Relevant
They are not inherently capable of determining what is appropriate.
A knowledge assistant asked about a contract may surface HR records, legal memoranda, or privileged communications because the system considers those documents relevant to the request. Technically, the retrieval may be accurate. From a governance perspective, it may be entirely inappropriate.
Rather, a well-run knowledge assistant needs a deliberately chosen source set, retrieval that respects the user's role, extra filtering on the most sensitive material, and, for custom agents, a way to refresh or drop indexed content when the source changes, along with a record of which sources shaped an important answer, and testing for over-retrieval. Skip that, and the assistant slowly turns into an over-permissioned search engine that nobody decided to build.
Underneath all of it sits one principle: relevance is not authorization. A document can match a question perfectly and still be one the user was never meant to see. In practice that means every query carries the user's own identity, group membership and permission, so the assistant can surface only what the individual could already open, rather than the assistant reaching into one broad account that can read everything. Where the platform allows it, that filtering should honour sensitivity labels as well as permissions, because permissions decide who may open a document while labels decide what may be done with its contents. Scope matters just as much as identity. Point the assistant at a deliberately chosen set of sources rather than everything available, with sensitive domains such as HR and legal kept more tightly permissioned so a general question cannot pull them in by accident.
Whether the platform is Microsoft 365 Copilot, a custom AI agent, or an enterprise AI account, the same question applies: What information should this system be allowed to access, retrieve, and act upon?
Applying Data Boundaries in the Tools You Already Use
The principles behind data boundaries are technology-neutral. The tools may differ, but the underlying governance questions remain the same.
Microsoft 365 Copilot
Many organizations assume that introducing Microsoft 365 Copilot creates entirely new information risks. In reality, Copilot generally operates within the permissions and controls that already exist within Microsoft 365. If a user can access a file, Copilot can use that file to answer a question. If a user cannot access it, Copilot cannot bypass those permissions.
The challenge is that AI changes how people find information.
Before Copilot, employees often needed to know where a document was stored or what it was called. Information that was technically accessible was not always easy to discover. Copilot changes that experience. A user can simply ask a question and surface information that previously required significant effort to locate. This is why one of the most important Copilot controls is not an AI control at all. It is permission hygiene.
Before rolling out Copilot broadly, organizations should review:
Overshared SharePoint sites
Legacy permissions
External sharing arrangements
Sensitive repositories
Labeling Practices
In many cases, AI does not expose a new problem. It simply reveals an existing one.
Data Loss Prevention (DLP) controls add another important layer. Permissions determine who can access information. DLP helps control how sensitive information is used, shared, or transferred once accessed.
Cleaning up permissions decides who can reach a document. DLP decides how sensitive content moves once reached. Copilot makes weakly secured content easier to find, so the two have to run together.
Enterprise AI Accounts
Most organizations now use commercial AI platforms in some form, whether through ChatGPT Enterprise, Claude for Business, Microsoft Copilot Chat, or similar services. Business accounts typically provide stronger safeguards than personal accounts, including administrative controls, identity management, retention settings, audit capabilities, and contractual protections. Those safeguards are important, but they should not be confused with governance.
Organizations still need to decide:
What information employees are permitted to enter
Which data sources may be connected
Whether memory features are enabled
How long information is retained
Which third parties may process the information
Under privacy laws, accountability for personal information does not disappear simply because a vendor processes it. Organizations remain responsible for understanding how information is handled, where it is processed, and whether appropriate safeguards exist. A licence is not a boundary. The boundary comes from the contract, the admin settings, identity, the sources you connect, retention, the apps you switch on, and how staff actually behave, all working together. A trustworthy provider cannot save you from an over-wired integration or a sloppy internal workflow.
Retention Is A Three-Part Responsibility
Retention is worth pausing on, because it is where good intentions quietly lapse. There is no single setting for it, and it sits in three places. First, the vendor. Enterprise plans from the major vendors let an administrator configure how long prompts and outputs are kept, and in some cases opt for zero retention, but the default is often to keep data indefinitely until someone changes it, and personal accounts offer no such control at all. Second, your own platform. In Microsoft 365, for example, Copilot prompts and responses can be governed with the same retention and deletion policies you already use for email and documents, including shorter timelines for AI interactions specifically. Third, the connected sources. Information the assistant retrieves keeps the retention rules of the system it came from, not the AI. The point to hold onto is simple: for any AI system touching personal or confidential data, you should be able to say who set the retention period, where, and how deletion is proven.
Copilot Studio and Custom Agents
The governance challenge grows once organizations move beyond document retrieval and begin building custom agents. A custom agent is no longer limited to finding information. It can connect to business systems, retrieve records, trigger workflows, update data, and interact with multiple applications. Risks emerge gradually because teams focus on what can be connected rather than what should be connected. A business needs to ask, what information and capabilities does this agent actually need to perform its approved purpose? Every additional connector expands the agent's reach. Every connected system increases the information available to it.
AI Agents Need Limits
The boundary becomes more consequential when an AI system moves from generating content to taking action.
A chatbot that drafts an email or summarizes a document is primarily supporting a person. An AI agent goes further. It can retrieve information, choose tools, make decisions along the way, and interact with multiple systems to complete a task. That shift changes the nature of the risk.
An agent should have access only to the information, tools, memory, and authority required to perform its approved function. Nothing more. In practice, that discipline comes down to five boundaries.
| Boundary | What It Means |
|---|---|
| Bound the data | Define what information the agent can receive, retrieve, and use. |
| Bound the tools | Allow only the connectors, systems, and functions required for the task using least-privilege access. |
| Bound the action | Clearly separate actions the agent can perform independently from those requiring approval. |
| Bound the memory | Control what information is retained and prevent information from carrying across users, customers, or matters. |
| Prove it | Log what the agent accessed, what decisions it made, and what actions it performed so accountability remains clear. |
Logging: Prove What the Agent Did Without Keeping the Data
Of the five, proving it (logging) is the one most often left vague, so it is worth being concrete. A useful log records the decision, not the data: who or what triggered the action, what the agent reached for, which tool it called, whether policy allowed or blocked it, and the outcome, each with a timestamp. It does not need a copy of the sensitive content itself, and it should not keep one. This record should sit apart from the agent, which in most organizations is the security tooling already in place. In a Microsoft environment, Copilot and connected AI-app interactions can be captured and governed through Purview, and agent activity can flow into the same audit and monitoring tools that already watch everything else; other platforms feed their logs into a SIEM the same way. Because the log exists for accountability rather than operation, it carries its own retention, separate from anything the agent remembers, and is disposed of on that schedule. Kept this way, you can reconstruct exactly what an agent did without building a second store of everything it touched.
Organizations should also think carefully before allowing an agent to:
Send confidential information outside the organization
Modify important records
Move money
Change permissions or access rights
Communicate decisions that affect an individual
Trigger actions that cannot easily be reversed
Human review remains an important control for these situations. That review should be meaningful. Someone should understand what the agent is proposing, what information informed the recommendation, and the consequences of proceeding.
A Proportionate Place to Start
You do not need the full stack on day one. Start with what fits the size of the organization and grow into the rest. Consider organization's size, complexity, and AI maturity.
| Organization Type | What To Put In Place | Typical Tools |
|---|---|---|
| Small to Medium Organizations | Use approved enterprise AI tools, clean up permissions and sharing, apply sensitivity labels, and enable available DLP controls. | Native platform controls (e.g., Microsoft Purview, or the Google and AWS equivalents) |
| Mid-sized organization | Add the ability to discover, classify, and understand where sensitive information resides before connecting AI to it. | Data discovery and classification tools (e.g., Purview, Cyera, Securiti, Varonis, BigID) |
| Organizations building AI solutions or deploying agents | Add controls that inspect interactions with models, govern agent actions, and identify unsanctioned AI usage. | AI governance, security, and monitoring tools (e.g., Emerging native platform controls + gateway policies, Lakera, Prompt Security; Netskope, Zscaler for shadow use) |
No single tool exists today that does all of this. Each runs deep in one lane and dry in the rest. The work is to layer a few tools that each do their part well, and to know what each one is not doing for you.
For Smaller Organizations
Most of the protection comes from a handful of basics you very likely already own. Use approved enterprise tools so company data never runs through personal accounts. Clean up sharing and permissions before connecting anything to AI and apply sensitivity labels to the material that matters. Turn on the DLP (data-loss controls) in the platform. In Microsoft, Purview can flag oversharing, stop sensitive content being pulled into a prompt, and block paste-and-upload into consumer AI sites. Google and AWS have their own versions of the same idea.
For Mid-Sized Organizations
Usually, the data is spread across several clouds and SaaS platforms. It is worth looking at a dedicated tool to discover and map that data before you connect it to anything, so you are not wiring an assistant into libraries you have never really inventoried. Vendors such as Cyera, Varonis, and BigID play in this space. It is a genuine spend, so bring it in when the sprawl justifies it, not before.
When You Start Building with AI
Rather than simply using it, or when agents begin to act rather than advise, two more layers earn their place. The first sits between users and the model and inspects what goes in and what comes back, catching injections, stripping out sensitive data, and blocking leaks. Lakera, Prompt Security, and similar tools do this, though it is worth being honest that this is one layer and never the whole answer, since a model still cannot reliably tell instructions from data. The second is agent governance: gating tool calls, enforcing least privilege, and logging each step. Alongside all of it, keep an eye on the AI nobody sanctioned. Your existing web and cloud security tools, the Netskope and Zscaler kind, can show you who is using unapproved tools, so you can steer people back to the safe path rather than pretend the shadow use is not there.
The goal is not to install every control on the list. It is to close the gaps whose absence would be hardest to explain after something has gone wrong.
What Leadership Signs Off
Before an AI system is given access to sensitive information or allowed to take action on behalf of the organization, leadership should insist on six things:
There is a named owner who is accountable for the system.
The information the system may and may not access is clearly defined.
Vendor processing, retention practices, and connected services are understood.
The system's authority has clear limits.
Human intervention exists at consequential or irreversible points.
Controls for higher-risk systems are tested, not assumed.
These are governance decisions that shape how AI will operate in practice. Done well, these controls do not slow adoption down. They create the conditions for responsible adoption by ensuring the organization understands what its AI systems can access, what authority they have, and where human judgment remains essential.
Organizations that define those boundaries early are better positioned to adopt AI confidently, meet their privacy obligations, and scale adoption without losing control of the information that matters most.
Frequently Asked Questions About AI and Data Boundaries
-
Data boundaries define what information an AI system is allowed to access, retrieve, retain, and act on for a specific use case. Unlike classification labels such as public, confidential, or restricted, data boundaries are context-specific. The same information can be appropriate in one AI deployment and wrong in another.
-
Risk tiering determines how much governance and oversight an AI use case needs. Data boundaries determine what information that system is allowed to touch. A use case can be correctly risk-tiered and still fail if the underlying data access was never scoped. The two controls work together; neither substitutes for the other.
-
An AI assistant primarily helps users create, summarize, search, or analyze information while a person remains responsible for taking action. An AI agent can go further by retrieving information, interacting with systems, triggering workflows, or completing tasks on a user's behalf. As a result, AI agents typically require additional governance controls around permissions, approvals, monitoring, logging, and authority. The more an AI system can act rather than advise, the more important clearly defined data and action boundaries become.
-
Not directly. Copilot generally operates within existing Microsoft 365 permissions. If a user can already access a file, Copilot can use it; if they can't, Copilot can't bypass that. The real risk is that Copilot makes previously hard-to-find but technically accessible content easy to surface, which is why permission hygiene matters more than any AI-specific setting.
-
RAG lets an AI system answer questions using an organization's own documents and data sources instead of relying only on its training data. It improves accuracy, but retrieval systems surface what's relevant to a query, not necessarily what's appropriate for the requester to see. Relevance is not authorization.
-
Retention sits in three places. The AI vendor's settings, which may default to indefinite retention. The organization's own platform policies, such as Microsoft 365 retention rules extended to Copilot interactions. And the connected source systems, which keep their own retention rules regardless of what the AI does with the data.
-
Five boundaries: bound the data it can access, bound the tools and connectors it can use, bound which actions it can take independently versus needing approval, bound what it retains in memory, and log its decisions for accountability. Agents should also require human review before sending confidential data externally, modifying records, or making irreversible changes.
-
No. Business-tier accounts add safeguards such as admin controls, retention settings, and audit logs, but a license isn't a governance program. Organizations still need to define permitted use cases, data boundaries, connected data sources, approval requirements, retention practices, and accountability for higher-risk deployments.
-
Not necessarily. Sensitivity labels can help apply protections such as encryption, access restrictions, and usage controls, but they are only one part of AI governance. Organizations should combine labels with permissions, data loss prevention controls, retention policies, and clearly defined data boundaries to ensure AI systems access only information appropriate for their intended purpose.
-
Start with what you likely already have. Use approved enterprise AI accounts instead of personal ones, clean up sharing and permissions before connecting anything to AI, apply sensitivity labels, and turn on native DLP controls like Microsoft Purview or the Google and AWS equivalents. Add dedicated data discovery tools only once data sprawl across clouds justifies the cost.
