We mentioned Omer Mayraz from Legit Security in May, when he prompt-injected an AI code bot on GitLab and got it to play a Rick Astley video.

He’s got a new one, this time with Git Hub Copilot Chat. It’s a chatbot that makes helpful suggestions — and has full access to all the user’s private data!

Mayraz’s question was: can we send a pull request — a suggested code fix — that contains a prompt injection? And make the bot spill sensitive user data, like private code or AWS login keys? Yes, we can! [Legit Security]

First, Mayraz just sent instructions in plain text telling the bot to say “HOORAY!” at the end of its response. That worked — so he knew Copilot Chat would act on instructions in pull requests. Just any pull request sent in by a random user on the hostile internet.

Mayraz then put the command inside a comment in the pull request. Then it’s not visible to the user — but Copilot Chat can read it just fine, and act on it.

Next, Mayraz made Copilot Chat suggest the user should install an evil software package. That worked too.

Finally, Mayraz told Copilot Chat to grab the user’s private data, put it in a message to the user with a web address, and tell them to click on the link. You’ve won a gift certificate from GitHub, click here! Just by clicking that, you’ve sent your private data.

But can we do a zero click attack? Can we make Copilot Chat give us the user’s private data if they even look at the pull request page? Yes, we can!

If you could get GitHub to load your chosen image, the image address could encode the user’s data. But GitHub runs an image proxy, Camo. You can’t just put in an image and get data out that way, GitHub will sanitise it.

So first, Mayraz pre-generated a Camo address for every letter and symbol, so he had known addresses for each character.

Second, he got Copilot Chat to render the user secrets as a sequence of 1×1 transparent pixels at the pre-generated Camo addresses — one for each letter of the user’s private data. The user couldn’t see these — but Mayraz’s web server could see them.

So a user would just look at the pull request and Copilot Chat would generate a string of invisible pixels that called out to Mayraz’s web server and sent him the user’s data!

Mayraz used this to extract private code repositories and AWS login secrets. He’s called it CamoLeak.

GitHub says it’s closed the image hole — they disabled all images in Copilot Chat.

They did not fix the bit where you can still prompt-inject Copilot Chat just by sending a pull request. They’re still working on that one.

• Video, podcast

Eine Situation die viele zumindest in Teilen so kennen dürften. Aus den ziemlich fantastischen The MUTE Series, „a collection of one-take microfilms that report on the vagaries of human behaviour.” Für die jeweiligen Filme gelten immer drei Regeln: kein Dialog, keine Kamerabewegungen, nur eine Aufnahme.

(Direktlink, via Kottke)

Microsoft Viva Insights “helps people and businesses thrive with data-driven, privacy-protected insights.” It’s workplace spyware for your boss. [Microsoft; Microsoft]

Viva Insights now has an exciting new data source — “Benchmarks to compare Copilot adoption!” [Microsoft]

You can even get “external benchmarks,” where your boss can compare your company’s Copilot usage to other companies! Can’t fall behind!

The numbers track Copilot in Teams, Outlook, Word, Excel, PowerPoint, and Copilot Chat. Micrsosoft will be adding numbers on total meetings summarised, total hours summarised, and various classes of prompts. [Microsoft]

The Copilot dashboard does not yet separate out individuals. But it’s not going to be hard to filter results down to individuals.

The new Copilot dashboard will be rolled out “to all customers later this month.”

If Copilot was such a productivity enhancer, you wouldn’t need spyware to make your minions use Copilot. They’d just use it.

But on last week’s Copilot video, we got a valuable insight into Microsoft’s thinking, after telling a Microsoft salesperson they weren’t buying any Copilot with their Azure:

whereupon she revealed that the only thing she is being measured on for Q4 is sales of copilot, so if that’s off the table the conversation is not interesting.

They’ve given you a metric. And you know what to do with metrics! Hit that Copilot button! Hit it again! You don’t have to paste the workslop into the final document. But you do have to hit the button! Do it for Microsoft! Do it for productivity!

• Video, podcast

Click here to go see the bonus panel!

Hovertext:
Also you can add irr- to the front to negate the idea.

Click here to go see the bonus panel!

Hovertext:
Slowly the difference between artificial and fake will become more and more blurry.

Click here to go see the bonus panel!

Hovertext:
Is this the one that gets the hatemail? I keep waiting.

(No bonus panel today because of a missing file, it will be posted later.)