AI Governance for a Mid-Size Company: A Practical Guide

Cover card with the title AI Governance for a Mid-Size Company: A Practical Guide, in white text on a dark background with a cyan glow

AI governance is the set of rules, owners and checks that decide how a company uses AI: which tools staff may use, what data may go in, who approves new uses, and what happens when something goes wrong. A mid-size company can start with six parts and a few pages of writing.

This guide shows what a working program contains, how to write the employee usage policy, and how to start in four weeks. If you would rather have a senior person build it with you, see the consulting page or my fractional Chief AI Officer service.

What is AI governance, and how is it different from compliance?

AI governance is the system a company uses to manage how AI is chosen, used and watched. Compliance is narrower. It means meeting the laws, contracts and standards that apply to you.

The two overlap, but they are not the same job. A company can follow every rule it knows about and still have no idea which AI tools its staff use. Governance fills that gap. It gives the company a list of tools, a named owner and a way to respond when an answer is wrong or data leaks.

Why does a mid-size company need AI governance now?

Staff are already using AI, and most companies have not written a full set of rules. Four sourced numbers show the gap:

What the research found Number Source
Organizations with a formal, comprehensive AI policy 38% (30% have a limited one, 25% have none) ISACA, May 2026
AI users who bring their own AI tools to work 78% Microsoft and LinkedIn, May 2024
Breached organizations with no AI governance policy, or one still in development 63% IBM, July 2025
Extra breach cost at companies with high levels of shadow AI $670,000 on average IBM, July 2025

Shadow AI means staff using AI tools the company has not reviewed. The IBM study covered 600 organizations and breaches between March 2024 and February 2025. Among organizations that reported a breach of an AI model or application, 97% said they lacked proper AI access controls. IBM’s cost figure is an average across its sample, so your own number will differ.

Incident readiness is thin as well. ISACA’s 2026 poll of more than 3,400 digital trust professionals found that 56% do not know how long it would take to halt an AI system after a security incident. Another 39% do not know whether they have a documented process for shutting one down.

For a mid-size company the pressure comes from three sides: customers who ask about AI in security reviews, employees who want clear permission, and leaders who want AI results without a surprise. If you are weighing whether to bring in senior help, see when to hire a fractional Chief AI Officer.

What risks should AI governance cover?

AI governance should cover the ways AI can harm the business, its customers or its staff. For a full list of the risks and the numbers behind them, see AI risks for business. The usual list is wrong or invented answers, leaked data, bias in decisions about people, weak security around AI systems, and uses nobody can explain afterward.

Two habits keep those risks visible. The first is transparency: staff and customers should know when AI is involved in a piece of work or a decision. The second is monitoring. Someone should review how AI systems perform after launch, record errors and complaints, and have human oversight over any use that affects a person’s job, money or access to a service. Littler, a US employment law firm, points out that AI should not make a final decision about a person without independent human judgment, and that bias is a leading concern in hiring.

What does an AI governance framework include?

A working AI governance framework has six parts. Each can start as a one-page document, and the table shows what a first version looks like.

Part What it answers A first version
1. Owner Who is accountable for AI at the company? One named senior person and a small review group from IT, security, legal, HR and the business
2. AI inventory Which AI tools and AI features are in use? A shared spreadsheet: tool, team, data it touches, who approved it
3. Risk tiers How careful do we need to be with each use? Three tiers. Low (public information), medium (internal data), high (customer, employee or financial data, or a decision about a person)
4. Usage policy What may staff do and not do? A short policy that covers tools, data, human review and training (checklist below)
5. Vendor review Can we trust this tool with our data? A short questionnaire: where data is stored, whether it trains the model, who can access it, how it is deleted
6. Incident plan and review What happens when something goes wrong, and how do we keep the rules current? A named contact, a way to switch a tool off, and a scheduled review of the tool list and the policy

Order matters. Start with the owner and the inventory, because every other part depends on them. The risk tiers decide how much review each tool gets, so a low-risk writing helper does not wait in the same line as a tool that reads customer contracts.

The inventory is also where hidden AI shows up. Littler, a US employment law firm, notes that AI is built into common software, so employers should check which of their platforms use it. A tool list that only names the chatbots will miss those.

Which AI governance framework should you start from?

Most mid-size companies do not need to invent one. The NIST AI Risk Management Framework is a common starting point. NIST released version 1.0 on January 26, 2023, and a Generative AI Profile on July 26, 2024.

Three facts make it a good base for a company without a large risk team:

  • It is voluntary, so adopting it is a choice, not a certification you pay for.
  • It is meant for any sector and any size of organization.
  • It groups the work into four functions: Govern, Map, Measure and Manage.

The table below shows how the four functions line up with the six parts above. This is a plain-English reading, not NIST’s own wording, so check the framework itself before you quote it to a customer or auditor.

NIST function Where it shows up in the six parts
Govern Owner, usage policy, review schedule
Map AI inventory, risk tiers
Measure Vendor review, checks on output quality, incident log
Manage Incident plan, fixes, retiring tools that fail the review

NIST says it is revising version 1.0 as part of the White House AI Action Plan, so confirm the current version when you start.

How do you set up AI governance in four weeks?

Four weeks is enough for a first version at a company with a few hundred people. This schedule is a suggested plan, not a research finding, and a larger or more regulated company will need longer.

  1. Week 1: name the owner and count the tools. Pick one accountable person. Ask every team lead which AI tools they use and send a short anonymous survey to staff.
  2. Week 2: tier the risk and write the usage policy. Sort each tool into low, medium or high. Draft the employee policy from the checklist in the next section. Have legal and HR read it.
  3. Week 3: review the vendors that touch sensitive data. Start with the high tier. Approve, restrict or retire each tool, and publish the approved list where staff can find it.
  4. Week 4: train the team and set the review dates. Run a short session on the policy with real examples. My AI workshops can run that session live for your team. Put a quarterly tool-list review and an annual policy review on the calendar, plus a review after any incident.

Do not skip week 4. A policy nobody has been trained on tends to sit in a folder. ISACA’s 2026 poll found that only 33% of organizations train all employees on AI, up from 22% the year before. Staff cannot follow rules they have never seen explained.

What should an AI usage policy for employees include?

An AI usage policy for employees is a short document that says which tools staff may use, what data may go in, and who is responsible for the output. The list below follows the elements Littler published in March 2025 for employer AI policies, with plain-English notes.

  1. Purpose. One or two sentences on what the policy is for, such as allowing useful AI use while protecting data.
  2. Who it covers. Employees, contractors and anyone else who has access to the company’s AI tools.
  3. Approved tools. A named list. Littler recommends that any AI system not explicitly approved be treated as prohibited, with a simple way to request a new tool.
  4. Data rules. Give examples. No customer records, employee data, contracts, source code or financial data in a public AI tool. Say what may go into approved tools and what may not.
  5. Allowed, limited and prohibited uses. Split by team where it helps. Drafting a public blog post is a different risk than screening job candidates.
  6. Human review. A person checks AI output before it reaches a customer, a report or a decision. Littler says AI should not make a final business or employment decision without independent human judgment. Teach staff to check names, numbers, quotes and sources, because AI tools state wrong answers with confidence.
  7. Vendor limits. Some vendors restrict how their tools may be used. Fold those limits into your rules.
  8. Training. Say who must complete it and how often it is refreshed.
  9. Who to ask and what happens next. Name a contact for questions and describe the steps that follow a breach of the policy. A company can make the first steps coaching, not punishment.
  10. Links to other policies. Point to your privacy, security, anti-discrimination and harassment policies so the rules agree.

Keep it to two or three pages. A policy staff can read in five minutes gets followed more often than a twenty-page one. Re-read it every time a major tool changes its terms.

Here is how I handle human review. I always give Claude permission to read, because that does no harm, but I never give it blanket permission to edit or delete. When it wants to write, edit or delete, it asks and I approve. There are horror stories of an AI that deleted someone’s files.

Shared team tools need the same kind of rule. In my advanced skills video, I keep each skill as a plain folder in Dropbox, so I can give a teammate view-only access to use it or edit access to improve it, and everyone gets the latest version. Deciding who gets view-only and who gets edit access is a small governance decision, and it belongs in your usage policy.

Video: Claude Skills 2.0: The Advanced Tutorial (With 3 INSANE Fixes) Video · 14 min Claude Skills 2.0: The Advanced Tutorial (With 3 INSANE Fixes) I show how to keep your Claude skills in a shared folder you control, so a team can use and improve the same tools instead of each person keeping a private copy. Watch the video →

Who should own AI governance at a mid-size company?

One senior person should own it, and a small group should review it. The owner is often the head of operations, security or technology, but those people already have full jobs. That is the most common reason AI governance stalls.

When the work matters every week but does not fill a full-time role, some companies hire a fractional Chief AI Officer. For the role itself, see what a Chief AI Officer does. The cost guide lists market rates with dates and sources. If the main gap is a decision-maker and a plan, the owner role is the part to fill first.

What mistakes make AI governance fail?

Four mistakes cause most of the damage:

  • Banning AI outright. Staff keep using it, and now it happens out of sight. A clear approved list works better.
  • Writing the policy and stopping. With no training, no owner and no review date, the document ages quickly.
  • Counting only chatbots. AI is built into email, design, customer support and accounting software. Check those features too.
  • Treating all uses the same. If every request gets the same heavy review, teams route around the process. Tier the risk so low-risk uses move fast.

I think an AI usage policy that only limits people makes them afraid to do anything with AI. I want people to feel free to use it for everything.

What should you do next?

Do three things this week. Name one accountable owner. Ask every team lead to list the AI tools they use. Draft the usage policy from the checklist above and have your lawyer read it. If you want help building the program, or want a senior person to run it for a few months, my fractional Chief AI Officer service owns the plan, the policy, the training and the shared systems.

Sources

FAQs

What is AI governance in simple terms?

The rules, owners and checks that decide which AI tools a company uses, what data may go into them, who approves new uses, how output is checked, and what happens when something goes wrong.

Does a mid-size company need an AI governance framework?

Yes, if staff use AI at work, and most do. Microsoft and LinkedIn found that 78% of AI users bring their own AI tools to work. A short framework with one owner, one tool list and one usage policy is a reasonable first version.

What should an AI usage policy for employees include?

A purpose, who it covers, approved tools, rules for sensitive data, a rule that a person reviews AI output, vendor limits, training, who to ask, and what happens after a breach of the policy. The checklist below lists each part.

Which framework should a company start from?

The NIST AI Risk Management Framework is a common starting point. It is voluntary, works for organizations of any size, and groups the work into four functions: Govern, Map, Measure and Manage.

Is AI governance the same as AI compliance?

No. Compliance means meeting specific laws and contract terms. Governance is the wider system of owners, rules and checks, and compliance is one result of it.

Who should own AI governance?

One named senior person, with a small group from legal, IT, security, HR and the business units. At a company that is still early in its AI program, a fractional Chief AI Officer is one way to fill the seat part time.

Get new resources in your inbox

Consulting

Grow your business with a founder who’s done it

David coaches entrepreneurs and advises companies as a consultant and fractional Chief AI Officer.

Speaking

A speaker your audience will remember

Motivational keynotes & workshops on entrepreneurship, AI, financial literacy, and more.