There have been a few good suggestions that would improve the flexibility of the configuration files and do a better job of hiding things.
One is to define a reserved classpath location added to web.xml to load Spring beans from a path like classpath:*/META-INF/net.shibboleth.idp/config.xml or something along that line, so plugins can provide system-level configuration without adding files.
Another is to actually use this to put our own system configuration, or at least some of it, inside the jars so it's hidden rather than just marked read only.
Finally, we could leverage additional properties to provide the base location of different sets of configuration files instead of relying on relative paths to locate specific files. That way some files might be left inside WEB-INF by default but overrideable in the filesystem on a piecemeal basis instead of all or nothing like now.