From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-4.0 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 2E67B1F4BD for ; Fri, 4 Oct 2019 15:02:41 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id CBE16120ACE; Sat, 5 Oct 2019 00:02:28 +0900 (JST) Received: from o1678948x4.outbound-mail.sendgrid.net (o1678948x4.outbound-mail.sendgrid.net [167.89.48.4]) by neon.ruby-lang.org (Postfix) with ESMTPS id EFC4B120AC4 for ; Sat, 5 Oct 2019 00:02:25 +0900 (JST) Received: by filter0144p3mdw1.sendgrid.net with SMTP id filter0144p3mdw1-29673-5D975EFC-A0 2019-10-04 15:02:20.667348459 +0000 UTC m=+74890.110638263 Received: from herokuapp.com (unknown [18.212.244.163]) by ismtpd0026p1mdw1.sendgrid.net (SG) with ESMTP id _x9P2BiPRMGeRLvD2Jr31A for ; Fri, 04 Oct 2019 15:02:20.641 +0000 (UTC) Date: Fri, 04 Oct 2019 15:02:20 +0000 (UTC) From: merch-redmine@jeremyevans.net Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 70813 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 16236 X-Redmine-Issue-Author: cosmo0920 X-Redmine-Sender: jeremyevans0 X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: =?us-ascii?Q?RVE3t853K5scBhbmJHUzZTFFeVC=2FZSUmHZ0Dc+26wcEi2CTgsF1oz0wTSSxGGN?= =?us-ascii?Q?BIA0qNpNFd6g8Z2EGQoQsZPkheZ=2FNvEKQkOlxh3?= =?us-ascii?Q?T4wuhiKHg38RgWAar3=2FkMCGB9C1kHRq=2FMDri8HK?= =?us-ascii?Q?IxQOk0ehhUxQ6ODYujlCGkEVzjDymIZ5nsgapVL?= =?us-ascii?Q?1yAEg6O6vZEauhEik=2FM74eYNDFHTmo3Z=2FGQ=3D=3D?= To: ruby-core@ruby-lang.org X-ML-Name: ruby-core X-Mail-Count: 95226 Subject: [ruby-core:95226] [Ruby master Bug#16236] Cannot handle logrotate on Windows when one file is written from multi process/thread X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #16236 has been updated by jeremyevans0 (Jeremy Evans). Log rotation relies on renaming the current log file (`app.log` -> `app.log.0`) and the reopening the log file (`app.log`). This does not work on Windows in a multi-process environment because you cannot rename a file open by another process on Windows. I think it should work in a multi-thread single-process environment on Windows. I don't think this problem is fixable in `logger`. You should probably use an alternative logger implementation, maybe one that logs to another single process that can handle log rotation. Alternatively, you can have each worker process use its own log file, and then combine the log files later. ---------------------------------------- Bug #16236: Cannot handle logrotate on Windows when one file is written from multi process/thread https://bugs.ruby-lang.org/issues/16236#change-81900 * Author: cosmo0920 (Hiroshi Hatake) * Status: Open * Priority: Normal * Assignee: * Target version: * ruby -v: ruby 2.4.6p354 (2019-04-01 revision 67394) [x64-mingw32] * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN ---------------------------------------- ### TL;DR This issue occurred in Windows version of Fluentd. It uses ServerEngine and its logger functionality delegates into Ruby logger library. Fluentd's logrotate functionality is originated from Ruby logger library, Then, I found that this library does not seems to support logrotate under multi process/thread condition on Windows. ### Issue Detail When a log file which is written from the one process or thread, it works correctly: ``` require "logger" logger = Logger.new('test.log', 5, 30*1024) loop do logger.error "lost connection" logger.debug "got new connection" end ``` But logger writing file collision circumstances, it cannot handle logrotate correctly: ``` require "logger" logger = Logger.new('test.log', 5, 30*1024) myOtherLogger = Logger.new('test.log', 5, 30*1024) loop do logger.error "lost connection" logger.debug "got new connection" end ``` ``` ... log shifting failed. closed stream log writing failed. closed stream log shifting failed. closed stream log writing failed. closed stream log shifting failed. closed stream log writing failed. closed stream log shifting failed. closed stream log writing failed. closed stream log shifting failed. closed stream log writing failed. closed stream .... (after that many of these lines) ``` Fluentd launches spawn model (which has supervisor and several workers) on Windows. Currently, Fluentd implementation uses one logfile from supervisor and worker(s). So, it causes log shifting operation collision. ### Expected behavior Logrorating operation works correctly under multi process/threading circumstances on Windows like as Linux environment. -- https://bugs.ruby-lang.org/