Add file-based logging for worker, scheduler, and monitor services

[yarikoptic-gitmate]

Summary

This PR adds file-based logging for the worker, scheduler, and monitor (Flower) services in Datalad-Registry. Instead of relying on the container runtime to capture logs (e.g., to the systemd journal), these services now write their logs to files on the host, reducing noise in the container runtime's logging system.

Key Changes

• docker-compose.yml: Updated worker, scheduler, and monitor service commands to write logs to files under /var/log/datalad-registry inside containers

  • Worker service: Added --logfile /var/log/datalad-registry/worker.log
  • Scheduler service: Added --logfile /var/log/datalad-registry/scheduler.log
  • Monitor service: Added --log_file_prefix=/var/log/datalad-registry/monitor.log
  • Added volume mounts for LOG_PATH_AT_HOST to all three services

• template.env: Added new LOG_PATH_AT_HOST environment variable (defaults to ./services/log) to specify the host directory where log files are written

• README.md: Added comprehensive "Logs" section documenting:

  • Why file-based logging is used for these services
  • The need for host-level log rotation since Datalad-Registry doesn't rotate logs itself
  • Reference to the new logrotate configuration example

• tools/logrotate-datalad-registry: Added example logrotate configuration file with installation instructions, using copytruncate to handle long-running container processes

Implementation Details

The logging setup uses copytruncate in the logrotate configuration rather than the default rotate-then-signal approach, because the log files are held open by long-running processes inside containers that logrotate on the host cannot directly signal. This is a common pattern for containerized applications.

The changes are backward compatible—the LOG_PATH_AT_HOST environment variable can be configured to any desired location, and the volume mounts ensure logs are accessible on the host for monitoring and archival purposes.

https://claude.ai/code/session_0127zpCF6N24jrkd9JoSBUDe

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.82%. Comparing base (820c689) to head (e8ab360).

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #418   +/-   ##
=======================================
  Coverage   98.82%   98.82%           
=======================================
  Files          55       55           
  Lines        2633     2633           
=======================================
  Hits         2602     2602           
  Misses         31       31           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.