Credit goes to PJ King. I was trying to add unit tests for this new
behaviour but it turned out there are many hard-coded dependencies and
it's not easy to make the whole machinery support alternative location
of testable (fake) plugins.