"users will be frustated and leave" exactly the same thing can happen to an instance that adds an instance (or wildcard domain) block. I'd be very surprised if no instance has ever rolled back a block.
Users don't need to worry about instance blocks on ActivityPub, any more than they have to worry about DNS RBLs for email.
infosec.pub is pretty tolerant...