It turns out that ACS v2 currently supports two “types” of signing certificates for identity providers:

• self signed certificates
• certificates that chain up to a trusted root (from Microsoft’s perspective), e.g. VeriSign

In other words, when you use a signing certificate in ADFS that comes from your internal PKI, it won’t work.

I hope this will get fixed – err – changed soon.