During or after deployment creation, errors are logged in rabbit regarding connections being dropped without being closed:
This suggests that one of our AMQP clients is not correctly closing connections.
Currently this does not have a major impact, but it does cause log noise and it is unclear without investigation whether this would begin to cause issues at higher manager load (e.g. due to ports which are no longer in use waiting to be closed).
This happens because amqp-influx is running before influxdb. It's possible to suppress these errors by creating the right rules in systemd's unit file to make sure amqp-influx is up and running only after influxdb and rabbitmq are running. However it still takes few moments for the actual port to be open so I'm not sure we will entirely remove these errors from logs.
Uncertain on that- I gave it a quick test wrapping that code with try/except and restarting amqpinflux on a test manager yesterday and it still showed the same errors coming up in rabbit's logs, but nothing on my hacky debug output. Might've been messed up debug output, of course, but could also be somewhere else causing this.
This may have something to do that the AMQPClient instance used on a per-thread basis never actually closes its connection:
The close() function is never called. We found this in 3.2.1, on a client installation. This bug also exists in 3.3 and is currently being investigated.