Hacker News

The AI aspect of this is a red herring. The real problem is that they're secretly adding in a unique identifier into every image you create. If somebody does not like your meme, they can just send a copyright subpoena to Microsoft to instantly get your full name, address, email, phone number, and any other data associated with your Microsoft account. Just like age verification, this is another weapon in the war against internet anonymity.

I had this trigger the other day incorrectly and went and installed Paint.net. I pasted in a screenshot I took and just wanted to resize it. I got a banner saying it was made with AI and would be updated to reflect that.

AI-generated text warning (I submitted - but did not author - the piece), but it seems MS Paint and MS Photos add both a visible (can be turned off) and invisible (cannot be disabled and happens silently in the background with no user notice) watermarks to photos that have been AI-manipulated, even when using a local model to perform the action. It's not clear if this applies to even things like using AI-enhanced background delete/remove, but the invisible watermark is embedded in both the image pixels and the image metadata, both containing a GUID that can be linked to the exact prompt that was used and the originating device/user (on Microsoft's end).

Obvious next step is to explore if you can replace watermarker.dll with a (signed) no-op shim or MITM the API call to at least use your own (nil?) GUID that isn't linked to your device/account.

In case it's not obvious, my bigger concern isn't "this image can be identified to have been generated with/by AI" so much as it is "digital yellow printer dots have been forced upon us, except they can identify and retrieve the exact user/device/time/place/document/etc", completely destroying any and all illusions of privacy left.

I get the privacy concerns, and we are right to expect Microsoft to say that this is what their tool may be doing. However, I fear that one day we will look back and wonder why we didn't do more to sign and preserve human authenticity. Having a stamp saying "AI manipulated" should be a part of digital lineage tooling.

Keep an eye on this.

A few months back, MS incorrectly tried to stamp a Copilot "watermark" (just an auto-added note) to any and all Azure DevOps commits, regardless of whether an LLM was actually involved. They removed it after a lot of github issues were submitted to the source of the issue which was a VS Code Copilot extension.

MS has been very sloppy in their implementations. I would recommend against using Paint or any other LLM enabled app they use as a result. Things may be getting incorrectly stamped.

Thanks Microsoft, for adding my signature so I won't have to claim authorship when it ends up in a museum in 200 years, and the NSA archives are declassified for art historians filing a FOIA in 2226, who find out, "yep, it was from his PC."

Interesting. I really didn't think watermarks would end up going anywhere, but maybe with enough adoption we can have easy ai generated content flagging after all?

I think it would be nice if all cameras digitally signed pictures. You could prove the photo was real.

[dead]