Passed Review, Then Switched Sides: Be Careful With AI Skills From the Internet

Shared prompts, skills, and MCP servers can change after trust is granted. Treat internet skills like code running inside your own boundary.

A clean install can still become a different thing after trust is granted.

A few years back I used to joke with colleagues about the perfect little heist. You build a clean, harmless app. You submit it to the App Store. It sails through review - nothing to flag, everything by the book. Then, the day after you go live, you quietly change one setting on your server and point all that traffic at something completely different.

We always laughed, because we all knew the ending. Apple pulls the app within a month, bans the account, and that's that. The store has a bouncer at the door and a camera inside. The bait-and-switch is old, obvious, and easy to punish.

Here's the part that stopped being funny: that exact trick is now a working attack against AI skills, prompts, and agents - and unlike the App Store, this ecosystem doesn't have a bouncer yet. It just has you, clicking "allow."

This was sparked by a clip from @antdaviscyber on exactly this risk. The short version of his point, and mine: be careful with AI prompts, skills, and agents you grab from the internet.

"Be careful with skills from the internet" is not paranoia

Quick definition, because the word skill is doing a lot of work here. A skill is a reusable playbook - a bundle of instructions (and sometimes code) that an agent loads to perform a task. Same idea for a shared prompt, a community MCP server, or an agent template you pull off a marketplace. They're fantastic for not reinventing the wheel. They're also, increasingly, downloaded and wired into real systems with about as much scrutiny as a meme.

That's not me being dramatic. Security researchers note that many users - including non-technical ones - install servers and skills straight from public repositories with little to no review, and when those run locally they can reach the whole host machine. Elastic Security Labs

Why "it passed review" means almost nothing

The uncomfortable design detail: a lot of skill systems use progressive disclosure. The agent preloads only the skill's short description to save context, then dynamically loads the full skill later, while it's running. Akamai

Read that again with my old joke in mind. What you skim up front is not what actually runs. The wholesome description is the app that passed review. The body that loads later - in context, with your permissions - is the server-side switch. An attacker doesn't need to sneak anything past your eyes, because the malicious part was never in front of your eyes to begin with.

And it doesn't take exotic code. One documented npm package used a postinstall hook to drop files into the directory a popular coding assistant loads its skills from. Those files told the AI to auto-approve all shell commands and file operations - effectively switching off its own permission system. No data theft, no command-and-control. It just quietly removed the locks, and it persisted even after you uninstalled the package. r/cybersecurity

The river is already full of these

If this sounded hypothetical, the measurements have caught up:

In one supply-chain study of agent skills, prompt injection showed up in 36% of the skills analyzed, alongside 1,467 malicious payloads. Researchers confirmed 76 of them were built for credential theft, backdoor installation, and data exfiltration - and 8 were still publicly downloadable at the time of publishing. Snyk
A large-scale academic study found community registries indexed over 98,000 skills within three months of launch. "Shadow features" - capabilities that appear nowhere in the public docs - showed up in 100% of the advanced attacks. The good news, which is also the bad news: 93.6% were removed within 30 days of disclosure, meaning they were live and installable until someone happened to catch them. arXiv
For MCP servers specifically, Palo Alto's Unit 42 lays out three live attack vectors: resource theft, conversation hijacking, and covert tool invocation - a compromised server can inject persistent instructions, manipulate responses, and exfiltrate data. Unit 42

The platforms aren't asleep, to be fair. Anthropic hard-codes a list of approved network domains so a rogue skill can't silently ship your data to some unknown URL. SecurityWeek But notice where the last line of defense lives: it still comes down to you seeing a suspicious request and clicking "deny." Medium - Idan Habler

Why AI is worse than my App Store joke

The comparison I started with breaks down - in the scary direction.

Reach and speed. Apps trickle through review. Skills get copy-pasted, starred, and installed in seconds, often by people who'd never read a line of what they're running.
No real gate. There's no App Review standing between a public repo and your agent. The "it passed review" reassurance simply doesn't exist.
Blast radius. An approved iOS app runs in a tight sandbox. A skill or MCP server often runs with your shell, your files, your tokens, and your inbox.
Persistence and invisibility. This is the SolarWinds lesson: a backdoor shipped through a trusted update meant that doing everything right - trusting a known vendor, patching promptly - was exactly what got you infected. Guard0 Skills inherit that property, minus the enterprise-grade scrutiny that eventually caught SolarWinds.

So yes - my old App Store joke gets the app banned in a month. The AI version gets shell access on your laptop and a head start, because the thing is so widespread and so easy to adopt that nobody's watching the door.

What I actually do

Deliberately boring, like last time. That's the point.

Read the whole skill, not the README. Especially anything that loads or fetches later. If you can't see what runs, assume the worst version of it.
Treat "auto-approve" and "allow all" as red flags. A skill that wants to disable your confirmations is telling you something.
Pin versions. A skill that can silently update itself can switch sides after you've decided to trust it. That's the whole exploit.
Least privilege, always. Scoped tokens, no standing access to everything. Don't hand an agent the master key because it's convenient.
Prefer readable over remote. A skill you can read end-to-end beats one that phones a server you'll never see.
Keep a human on anything irreversible. Push, deploy, send, delete, pay - those get my eyes, not the model's.

The asymmetry worth sitting with

When you run a skill from the internet, you're executing someone else's instructions inside your own trust boundary. The author got to decide what it does - and, thanks to progressive disclosure and self-updates, they often got to decide that after you approved it. That's not a reason to swear off skills; they're genuinely useful and I use them daily. It's a reason to treat "from the internet" the way you'd treat a USB stick you found in a parking lot.

The joke was always that you could pass review and then switch the traffic. With AI, half the time nobody's even checking the review. So check it yourself.