Uploaded image for project: 'Shibboleth SP - C++'
  1. Shibboleth SP - C++
  2. SSPCPP-757

Race condition in size-triggered log rotation in Native logger

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 2.6.0
    • Fix Version/s: None
    • Component/s: Logging
    • Labels:
      None
    • Operating System:
      Linux
    • CPU Type:
      x86_64
    • Web Server:
      Apache 2.4

      Description

      Hi,

      There is a race condition in the log rotation in the NativeSP module linked into Apache.

      When Apache is reloading the configuration (which is also triggered as post-rotate action from /etc/logrotate.d/httpd on CentOS 6 and 7), all httpd forked children are active at the same time.

      And all of them do some logging from mod_shib.

      And if the native.log is just below the threshold for sized-based log rotation, the children try to rotate it each independently, stepping on each other's toes.

      I've even seen:

      ls -lA /var/log/shibboleth-www/
      total 2240
      -rw-r--r--. 1 root root 990963 Oct 26 15:31 native.log
      -rw-r--r--. 1 root root 135521 Oct 26 15:31 native_warn.log
      

      service httpd reload

      ls -lA /var/log/shibboleth-www/
      total 2268
      -rw-r--r--. 1 apache apache    4411 Oct 26 15:32 native.log
      -rw-r--r--. 1 apache apache       0 Oct 26 15:32 native.log.1
      -rw-r--r--. 1 apache apache    4411 Oct 26 15:32 native.log.2
      -rw-r--r--. 1 apache apache    4510 Oct 26 15:32 native.log.3
      -rw-r--r--. 1 apache apache    2632 Oct 26 15:32 native.log.4
      -rw-r--r--. 1 root   root   1000462 Oct 26 15:32 native.log.5
      -rw-r--r--. 1 root   root    136656 Oct 26 15:32 native_warn.log
      

      (the host runs in NZ timezone; hence the future timestamps).

      IMHO, the only way to get around this is to move the rotation out to external tools - eg, logrotate, as documented eg at https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPLogRotation

      PS: and the log rotation also doesn't go well with SELinux: Apache is only allowed to append to logs, not move/rename/delete them. Another reason for not doing it in Apache.

      PS2: this is different from SSPCPP-646

        Attachments

          Activity

            People

            Assignee:
            cantor.2@osu.edu Scott Cantor
            Reporter:
            tuakiriadmin-vmencl@virtualhome.tuakiri.ac.nz Vladimir Mencl
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: