[ruby-core:95216] [Ruby master Bug#16236] Cannot handle logrotate on Windows when one file is written from multi process/thread

[ruby-core:95216] [Ruby master Bug#16236] Cannot handle logrotate on Windows when one file is written from multi process/thread

[cosmo0920.oucc]

Issue #16236 has been reported by cosmo0920 (Hiroshi Hatake).

----------------------------------------
Bug #16236: Cannot handle logrotate on Windows when one file is written from multi process/thread
https://bugs.ruby-lang.org/issues/16236

* Author: cosmo0920 (Hiroshi Hatake)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: ruby 2.4.6p354 (2019-04-01 revision 67394) [x64-mingw32]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
### TL;DR

This issue occurred in Windows version of Fluentd.
It uses ServerEngine and its logger functionality delegates into Ruby logger library.
Fluentd's logrotate functionality is originated from Ruby logger library,
Then, I found that this library does not seems to support logrotate under multi process/thread condition on Windows.

### Issue Detail

When a log file which is written from the one process or thread, it works correctly:

```
require "logger"

logger = Logger.new('test.log', 5, 30*1024)

loop do
  logger.error "lost connection"
  logger.debug "got new connection"
end
```

But logger writing file collision circumstances, it cannot handle logrotate correctly:

```
require "logger"

logger = Logger.new('test.log', 5, 30*1024)
myOtherLogger = Logger.new('test.log', 5, 30*1024)

loop do
  logger.error "lost connection"
  logger.debug "got new connection"
end
```

```
...
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
.... (after that many of these lines)
```

Fluentd launches spawn model (which has supervisor and several workers) on Windows.
Currently, Fluentd implementation uses one logfile from supervisor and worker(s).
So, it causes log shifting operation collision.

### Expected behavior

Logrorating operation works correctly under multi process/threading circumstances on Windows like as Linux environment.



-- 
https://bugs.ruby-lang.org/

[ruby-core:95226] [Ruby master Bug#16236] Cannot handle logrotate on Windows when one file is written from multi process/thread

[merch-redmine]

Issue #16236 has been updated by jeremyevans0 (Jeremy Evans).


Log rotation relies on renaming the current log file (`app.log` -> `app.log.0`) and the reopening the log file (`app.log`).  This does not work on Windows in a multi-process environment because you cannot rename a file open by another process on Windows.  I think it should work in a multi-thread single-process environment on Windows.

I don't think this problem is fixable in `logger`.  You should probably use an alternative logger implementation, maybe one that logs to another single process that can handle log rotation.  Alternatively, you can have each worker process use its own log file, and then combine the log files later.

----------------------------------------
Bug #16236: Cannot handle logrotate on Windows when one file is written from multi process/thread
https://bugs.ruby-lang.org/issues/16236#change-81900

* Author: cosmo0920 (Hiroshi Hatake)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: ruby 2.4.6p354 (2019-04-01 revision 67394) [x64-mingw32]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
### TL;DR

This issue occurred in Windows version of Fluentd.
It uses ServerEngine and its logger functionality delegates into Ruby logger library.
Fluentd's logrotate functionality is originated from Ruby logger library,
Then, I found that this library does not seems to support logrotate under multi process/thread condition on Windows.

### Issue Detail

When a log file which is written from the one process or thread, it works correctly:

```
require "logger"

logger = Logger.new('test.log', 5, 30*1024)

loop do
  logger.error "lost connection"
  logger.debug "got new connection"
end
```

But logger writing file collision circumstances, it cannot handle logrotate correctly:

```
require "logger"

logger = Logger.new('test.log', 5, 30*1024)
myOtherLogger = Logger.new('test.log', 5, 30*1024)

loop do
  logger.error "lost connection"
  logger.debug "got new connection"
end
```

```
...
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
.... (after that many of these lines)
```

Fluentd launches spawn model (which has supervisor and several workers) on Windows.
Currently, Fluentd implementation uses one logfile from supervisor and worker(s).
So, it causes log shifting operation collision.

### Expected behavior

Logrorating operation works correctly under multi process/threading circumstances on Windows like as Linux environment.



-- 
https://bugs.ruby-lang.org/

[ruby-core:95541] [Ruby master Bug#16236] Cannot handle logrotate on Windows when one file is written from multi process/thread

[cosmo0920.oucc]

Issue #16236 has been updated by cosmo0920 (Hiroshi Hatake).


Thanks for the information.
This issue should be Fluentd logging mechanism issue not Ruby core logger library.
Fluentd side logrotate issue is fixed.
So, it can be closed.

----------------------------------------
Bug #16236: Cannot handle logrotate on Windows when one file is written from multi process/thread
https://bugs.ruby-lang.org/issues/16236#change-82317

* Author: cosmo0920 (Hiroshi Hatake)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: ruby 2.4.6p354 (2019-04-01 revision 67394) [x64-mingw32]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
### TL;DR

This issue occurred in Windows version of Fluentd.
It uses ServerEngine and its logger functionality delegates into Ruby logger library.
Fluentd's logrotate functionality is originated from Ruby logger library,
Then, I found that this library does not seems to support logrotate under multi process/thread condition on Windows.

### Issue Detail

When a log file which is written from the one process or thread, it works correctly:

```
require "logger"

logger = Logger.new('test.log', 5, 30*1024)

loop do
  logger.error "lost connection"
  logger.debug "got new connection"
end
```

But logger writing file collision circumstances, it cannot handle logrotate correctly:

```
require "logger"

logger = Logger.new('test.log', 5, 30*1024)
myOtherLogger = Logger.new('test.log', 5, 30*1024)

loop do
  logger.error "lost connection"
  logger.debug "got new connection"
end
```

```
...
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
.... (after that many of these lines)
```

Fluentd launches spawn model (which has supervisor and several workers) on Windows.
Currently, Fluentd implementation uses one logfile from supervisor and worker(s).
So, it causes log shifting operation collision.

### Expected behavior

Logrorating operation works correctly under multi process/threading circumstances on Windows like as Linux environment.



-- 
https://bugs.ruby-lang.org/

[ruby-core:95542] [Ruby master Bug#16236] Cannot handle logrotate on Windows when one file is written from multi process/thread

[cosmo0920.oucc]

Issue #16236 has been updated by cosmo0920 (Hiroshi Hatake).


Fluentd side patch: https://github.com/fluent/fluentd/pull/2663
Fluentd side issue: https://github.com/fluent/fluentd/issues/2446

----------------------------------------
Bug #16236: Cannot handle logrotate on Windows when one file is written from multi process/thread
https://bugs.ruby-lang.org/issues/16236#change-82318

* Author: cosmo0920 (Hiroshi Hatake)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: ruby 2.4.6p354 (2019-04-01 revision 67394) [x64-mingw32]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
### TL;DR

This issue occurred in Windows version of Fluentd.
It uses ServerEngine and its logger functionality delegates into Ruby logger library.
Fluentd's logrotate functionality is originated from Ruby logger library,
Then, I found that this library does not seems to support logrotate under multi process/thread condition on Windows.

### Issue Detail

When a log file which is written from the one process or thread, it works correctly:

```
require "logger"

logger = Logger.new('test.log', 5, 30*1024)

loop do
  logger.error "lost connection"
  logger.debug "got new connection"
end
```

But logger writing file collision circumstances, it cannot handle logrotate correctly:

```
require "logger"

logger = Logger.new('test.log', 5, 30*1024)
myOtherLogger = Logger.new('test.log', 5, 30*1024)

loop do
  logger.error "lost connection"
  logger.debug "got new connection"
end
```

```
...
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
log shifting failed. closed stream
log writing failed. closed stream
.... (after that many of these lines)
```

Fluentd launches spawn model (which has supervisor and several workers) on Windows.
Currently, Fluentd implementation uses one logfile from supervisor and worker(s).
So, it causes log shifting operation collision.

### Expected behavior

Logrorating operation works correctly under multi process/threading circumstances on Windows like as Linux environment.



-- 
https://bugs.ruby-lang.org/