Emerging AI Risks in the Cyber Landscape for Defense Contractors
Executive Brief
Artificial Intelligence (AI) is reshaping social engineering on two fronts: the voices we trust and the systems we rely on. The voice on the phone and the document on the screen can now be weaponized in ways that didn't exist a few years ago.
For defense contractors handling sensitive program data, that shift raises the stakes on an old problem: verifying that the person, or system, you are trusting is who or what it claims to be.
- AI-generated voice and video can convincingly impersonate an executive, employee, vendor, or family member
- A document, email, or webpage can carry hidden instructions that an AI system follows without anyone typing anything malicious
- Both attacks succeed the same way: by exploiting trust in something that used to be reliable proof of identity or intent
- The answer isn't just another security tool. It starts with independent verification, least privilege, visibility, and human approval for sensitive actions
Dig deeper below.
Phishing Isn't Just an Email Anymore
For years, the advice was simple: check the sender, check the link, check the spelling. That advice still applies, but it no longer covers the whole picture.
AI has extended phishing into new formats:
- A convincing message can arrive as a cloned voice on a phone call
- It can arrive as a realistic video on a Teams message
- It can arrive as an ordinary-looking document that embeds instructions meant for an AI system, not a human reader
The attacker's goal has not changed: get someone, or something, to act on false trust. What has changed is how many forms that false trust can now take.
When a Familiar Voice Isn't Proof of Identity
AI can now generate voice and video convincing enough to impersonate an executive, employee, customer, bank, or vendor. The Federal Bureau of Investigation (FBI) has warned that malicious actors have used AI-generated voice messages to impersonate senior government officials since 2023, often creating urgency around a payment, banking change, login, or confidential request, according to an FBI public service announcement.
The scams themselves are not new. Fake payment requests, banking changes, and urgent “confidential” messages have circulated for years. What is new is how real they sound and look.
In more sophisticated cases, attackers compromise a legitimate mailbox, study real invoices and conversations, and insert themselves into an existing thread. The message may come from a real account and appear in a familiar exchange but still be fraudulent.
This same risk extends beyond the workplace. Voice cloning has also been used in ransom scams targeting families, where a caller's voice sounds exactly like a loved one in distress. Security experts increasingly recommend that families set a private code word, something an AI-generated voice would not know, to verify identity before acting on an urgent request.
A few things worth keeping in front of your mind:
- Independently verify new banking instructions, direct deposit changes, or unusual payment requests
- Use a trusted phone number already on file, not the number or contact included in the message
- If a request arrives by call or voice message, call the person back using their known number
- Don’t let urgency, confidentiality, or an apparent executive request bypass your normal approval process
- Don’t treat a familiar voice, video, or email thread as proof on its own
The Document That Attacks Itself
The second emerging risk does not target a person directly. It targets the AI systems now built into everyday business tools.
AI agents can search through internal knowledge bases, retrieve company information, call Application Programming Interfaces (APIs), and act on a user's behalf. That makes them useful, but it also gives an attacker a new way in through a technique known as prompt injection.
A simple version instructs an AI system directly to ignore its previous instructions and share confidential information. The more concerning version, indirect prompt injection, hides that instruction inside something the AI is simply asked to process: a document, an email, a webpage, or a support ticket.
The employee never types anything malicious. The pattern looks like this:
- An employee asks AI to summarize a document
- The document contains a hidden instruction
- The AI follows it
- The AI accesses data or invokes a tool no one intended
There may be no traditional phishing indicator for the employee to notice. No suspicious link has to be clicked. The AI is processing content it was legitimately asked to read. Our blog on why AI fails without organizational change management covers a related risk: AI tools adopted without governance tend to create blind spots like this one.
Why This Matters for Defense Contractors
Defense contractors working with the Department of Defense (DoD) (also known as the Department of War) and handling Controlled Unclassified Information (CUI), Federal Contract Information (FCI), or other sensitive program data are an attractive target precisely because that information has value. Approval to use an AI tool for general business purposes does not automatically mean it is approved to process CUI or other sensitive government information.
As AI tools become part of normal workflows, the questions worth asking expand beyond “did someone click a bad link” to:
- What data can our AI tools access, and what identity are they using to access it?
- Which actions can an AI system take on its own, and which require human approval?
- Are AI-related activities logged in a way we can review?
- Do we have a way to independently verify high-stakes requests, whether they come from a person or a system?
Not sure if your organization handles either CUI or FCI? Our CUI quiz is a fast way to check and our blog on who is responsible for protecting CUI breaks down the shared ownership question.
WHAT DEFENSE CONTRACTORS SHOULD BE DOING NOW
- Treat AI agents and integrations like privileged identities. Give them only the data, tools, and permissions required for the task, and require human approval for high impact actions
- Build independent verification into financial and access-related requests, regardless of how the request arrives
- Apply the same skepticism to AI-generated voice and video that you would to a suspicious email
- Review what data and actions your AI tools and AI-connected integrations can access
- Treat AI governance as an extension of your existing security program, not a separate initiative
The tools attackers use keep evolving, but the underlying principle does not: verify independently, don’t let urgency override process, and extend that same discipline to the AI systems now operating inside your business.
See Where This Fits into Your Bigger Picture
AI-driven social engineering and AI system risk don’t exist in isolation. They are part of a broader security and compliance posture, one that works best when it’s not pieced together from separate vendors and tools. ISI's Managed IT, Cybersecurity, and Compliance services unify IT, security, and compliance into one integrated program, so your team has a single, accountable partner keeping you secure and audit-ready at every step.
FAQs
Is AI-generated voice cloning something only high-profile targets need to worry about?
No. While government officials and executives are common targets, the same tactics are used against employees, vendors, and families. Anyone who can approve a payment, change a banking detail, or grant access is a potential target.
What is prompt injection, and do we need to worry about it if we don't build our own AI tools?
Prompt injections can affect any AI system that reads outside content, including commercial tools your team already uses. If your AI tools summarize documents, browse the web, or process emails, it’s worth asking your vendor what safeguards exist.
Does this fall under our existing CMMC or cybersecurity program, or is it something new?
It fits within your existing security program. Independent verification, access control, and logging are not new requirements. AI simply adds new channels, voice, video, and automated agents, that those same principles need to cover.
