How should a mid-size business govern AI tools?

Key facts

• Governance needs a named owner and a small group, not a large committee.
• An approved tool list and a request process prevent most shadow AI.
• Fixing data access, especially SharePoint oversharing, is part of AI governance.
• Identity controls such as Entra ID single sign-on keep AI access tied to employment.
• Frameworks such as the NIST AI Risk Management Framework can guide structure without heavy process.

Who owns AI governance

In a mid-size business, AI governance works best as a small working group that meets monthly at first and quarterly once things settle. Typical members are an executive sponsor, IT or security lead, legal or compliance, HR and one or two business leaders who use AI heavily.

Give one person clear ownership. They maintain the approved tool list, run the request process, coordinate policy updates and report to leadership. Without a named owner, governance documents get written once and then ignored.

The core governance artifacts

Technical controls that back up the policy

• Identity: connect approved AI tools to Entra ID with single sign-on, apply Conditional Access and MFA, and provision access by group so leavers lose it automatically.
• Data access: fix SharePoint and Teams oversharing, because Microsoft 365 Copilot and the Claude Microsoft 365 connector both surface what users can already reach.
• Classification: use Microsoft Purview sensitivity labels to mark sensitive content and drive protections.
• Data loss prevention: configure Purview DLP policies, including AI-related controls where available, to limit sensitive data in prompts and outputs.
• Discovery: use Defender for Cloud Apps or network controls, where licensed, to find unapproved AI services in use.
• Connectors and consent: require admin approval for AI connectors and third-party app consents in Entra ID.
• Logging: keep audit logs for approved tools and decide how long AI interactions are retained.

A risk-based review for new uses

Not every AI use needs the same scrutiny. Sort requests into tiers so low-risk uses move fast and high-risk uses get proper review.

Low risk: drafting internal emails, summarizing public material, brainstorming. Approve through the standard tool list.

Medium risk: working with confidential client documents, internal financial data or customer communications. Require an approved company account, data classification checks and human review.

High risk: uses that affect decisions about people, such as hiring or performance, regulated data, automated customer-facing responses or agents that take actions in other systems. Require a documented review, a named business owner, testing and periodic reassessment.

The NIST AI Risk Management Framework offers a useful structure, with its govern, map, measure and manage functions, and can be applied in a light form suited to a mid-size business.

Keeping governance practical

The biggest governance failure in mid-size businesses is not too little process but process so slow that people go around it. If a tool request takes three months, staff will use personal accounts. Set target turnaround times for requests, keep the policy short and publish the approved tool list where everyone can find it.

Measure what matters: number of approved tools, share of AI use on company accounts, open requests and their age, data incidents involving AI and training completion. Report these to leadership each quarter, along with any changes in regulations, client requirements or vendor terms that affect your tools.

Where to start

In the first 30 days, name an owner, publish an interim one-page policy and list the tools already in use. In the next 60 days, connect approved tools to single sign-on, start oversharing cleanup in SharePoint, set up a request process and finish the full acceptable use policy. After that, move to a quarterly cycle of tool review, risk review and reporting.

How Accred Consulting approaches it

Accred Consulting helps mid-size businesses set up AI governance that fits their size. Discovery maps the AI tools already in use, where sensitive data lives and how identity and sharing are configured in Microsoft 365. Accred then drafts the policy and approved tool list for your leadership and counsel to approve, and configures the supporting controls: Entra ID single sign-on, SharePoint oversharing fixes, sensitivity labels, DLP and connector approval. Accred deploys both Microsoft 365 Copilot and Claude. Work has a fixed price, US-based specialists and documentation at handover, with optional Support retainers for quarterly reviews.

Track record: 475+ client projects, 2,000+ migration workstreams and 210,000+ users migrated by a US-based team. Company facts · Case studies

Related questions