From discussion at the F2F, I think the dynamic plugin may not ever purge metadata that goes unused for long periods.
Environment
None
Activity
Scott Cantor June 1, 2016 at 7:54 PM
6e22d19e77ac07f8060b96787619d8f64c5a8d58
Optimistically speaking I think this should plug all of the race conditions and leaks plus addresses the cleanup of older entries if they go unused.
Entries are freed by calling a new unindex method on the base class, and all insertions are preceded by that call under a write lock.
Also made sure all updates to the cache expiration map are under a write lock.
Scott Cantor June 1, 2016 at 7:45 PM
And the real punchline is that it must be leaking memory, I don't see any way for it to be cleaning up old metadata entries when it replaces them with new copies, which should be happening every time it queries after the cache window.
Scott Cantor May 31, 2016 at 9:27 PM
I think there's also a race condition maintaining a data structure under a read lock, can't say I'm surprised given the lack of testing.
Will address both issues by adding an iterating background thread. Won't be terribly efficient, but I think we can assume a manageable number of IdPs being used at any given time to keep the size reasonable. O(1000) or under will be negligible, should only matter if we hit millions.
From discussion at the F2F, I think the dynamic plugin may not ever purge metadata that goes unused for long periods.