I've been discussing to and fro with Robin Gareus, one of the main developers on Ardour. It seems he can't reproduce the error due to not having the same setup but judging from Ardour's (coded) way of handling plugins it boils down on the same reason the AU does not validate and the VST does not initiate.
Some other test from my side showed that the 'auval' tool validates fine (output: https://pastebin.com/0WF0BMHV), that the AU plugin is capable of running in Audacity and the VST freezes Audacity completely without logs or reasons when invoked, and that the AU plugin will not initiate in Garageband where it says it is not licensed and I need an iLok manager on my computer (which I do have, and the plugin is properly licensed).
Also there is this entry over and over in my Console (/Library/Logs/PACEAntiPiracy/Eden/LicenseServices/LicenseD_000000.log):
[ error] TPaceHttpClient::flush() caught a UtilException with this information:The HTTP server returned an "Bad Gateway" error (status code 502).
Thanks for the additional info. Based on the behavior you describe (especially in GarageBand), I do wonder if the licensing is being finicky. I think we should eliminate that possibility first, since we have occasionally seen similar issues that can be dependent on when/how the DAW loads plugins, and the licensing being tripped up somehow. Can you try these steps, and let me know if it helps the issue?
- Install the latest version of iLok License manager (www.ilok.com)
In iLok License Manager, go to Preferences->Licenses and select "Remove account data saved on computer"
Sign out of your account, and then close iLok License Manager
Relaunch iLok License Manager, and log in
- Right click on the location of the Blackhole license (e.g. iLok or host machine), and click "Synchronize"
Reopen Ardour, or another DAW you've been having probems with, and try to load Blackhole