Log with logging instead of stdout/stderr

[josephernest]

Is there an option to make bottle use the Python built-in logging package instead of stderr/stdout?

Here is a basic example, would you see how to redirect the "Bottle v0.12.23 server starting up (using WSGIRefServer())... Listening on http://localhost:8080/ Hit Ctrl-C to quit." information to logging, as well as the different requests logs?

import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("bottle_server")
logger.info("hello world!")

from bottle import route, run, template

@route('/')
def index():
    return "hhello"

run(host='localhost', port=8080)

It would also have the benefit to avoid the issue of not being able to use bottle with pythonw (which has no stderr/stdout), as seen in #1104 (comment).

[defnull]

You can tell bottle to be quiet=True. It does not log anything meaningful, so disabling output completely is always an option.
There is no strong reason against switching to logging, but it seems excessive for two lines of info during startup, which is skipped anyway if you are not using the bottle CLI or run().

[josephernest]

Thanks for your answer.

You can tell bottle to be quiet=True

Even with quiet=True, then the script pythonw test.py still does not work:

import bottle
app = bottle.Bottle()
@bottle.route('/')
def index():
    return 'hello'
bottle.run(quiet=True, port=80)

because it probably still looks for stderr/stdout, even with quiet=True.

Do you think we could solve #1104 (comment) by setting

# Workaround for the "print is a keyword/function" Python 2/3 dilemma
# and a fallback for mod_wsgi (resticts stdout/err attribute access)
try:
    _stdout, _stderr = sys.stdout.write, sys.stderr.write
except IOError:
    _stdout = lambda x: sys.stdout.write(x)
    _stderr = lambda x: sys.stderr.write(x)
except:                   # <--- this 
    _stdout, _stderr = lambda *args: None, lambda *args: None

if quiet=True in next release? It would allow the use of pythonw (useful in some configuration where Bottle is a background server for a GUI interface).

it seems excessive for two lines of info during startup

We also have the logging of each request (this is maybe present in debug mode only?) which could be useful in logging.

which is skipped anyway if you are not using the bottle CLI or run()

How can it be started except run() ? Do you mean WSGI?

BTW: what is the bottle CLI? Is there a CLI tool available in bash: $ bottlepy --do-something?

[josephernest]

PS: the really interesting application would be to log in a file the uncaught exceptions in routes:

import logging, sys, traceback
logging.basicConfig(filename='test.log', filemode='a', format='%(asctime)s %(levelname)s %(message)s', datefmt='%Y-%m-%d %H:%M:%S', level=logging.DEBUG)
sys.excepthook = lambda exctype, value, tb: logging.error("", exc_info=(exctype, value, tb))
logging.info("hello")

from bottle import route, run, template
@route('/')
def index():
    sqdf         # uncaught exception !!! not logged in the test.log file. How to enable this?
    return "hhello"
run(quiet=True, port=80)

Here the exception in index() is still printed in the console (even if quiet=True), and not caught by sys.excepthook, would you know how to enable this @defnull ?

[defnull]

Request logging is the job of the HTTP server, not bottle. Exceptions are logged to environ['wsgi.error'], which should be provided by the WSGI server implementation. Both is out of scope for bottle.

How can it be started except run() ? Do you mean WSGI?

Yes. Bottle is a WSGI framework, you do not need to use run(), you can mount a bottle application with any WSGI server you want (e.g. gunicorn).

BTW: what is the bottle CLI? Is there a CLI tool available in bash: $ bottlepy --do-something?

python3 -m bottle --help

because it probably still looks for stderr/stdout, even with quiet=True.

This was all discussed in the other issue already.

Here the exception in index() is still printed in the console (even if quiet=True), and not caught by sys.excepthook, would you know how to enable this @defnull ?

Calling run() without a server parameter uses wsgiref.simple_server and that server prints to stdout/err I guess. Switch to a proper WSGI server that supports logging?

[josephernest]

Thanks @defnull, I read the code and I now understand that "request logging" is out of scope of bottle. Here is a nice way to do it: python bottle always logs to console, no logging to file.

About uncaught exceptions:

Exceptions are logged to environ['wsgi.error']

Here https://github.com/bottlepy/bottle/blob/master/bottle.py#L1007 we see that the traceback is written to environ['wsgi.errors'] using a file IO API .write(...).

Even if I use another server, or if I subclass class WSGIRefServer(ServerAdapter), then it seems that it won't be possible to use logging.error(error_str) because bottle expects a file IO object (or StringIO).

I don't see how another server than the default WSGIRefServer could expose environ['wsgi.errors'] as something that can both be written with .write(...) and use logging. Any idea?

[defnull]

environ['wsgi.errors'] is a file-like object, as defined by the WSGI spec. You are not supposed to provide that as an app-developer, that's the job of your WSGI server implementation. wsgiref.simple_server uses sys.stderr (see https://github.com/python/cpython/blob/main/Lib/wsgiref/handlers.py#L159) and does not provide an easy way to change that. wsgiref.simple_server is a bad choice anyway for anything but testing. Switch to a different server implementation that supports proper logging.

If you feel adventurous, you can also replace sys.stdout and sys.stderr with wrappers that implement the file-like API but redirect writes to logging. Do that before importing bottle, and bottle (or wsgiref if you insist in using that) will simply write to those instead. Python allows for this kind of monkey-patching. It's dirty, but it may work.

[defnull]

Do not bother with ServerAdapter or its subclasses. Those are just simple helpers to setup various existing WSGI server implementations with sane default configurations, that's all. If those defaults do not suit your needs, follow the docs of the WSGI server of your choice instead. Bottle apps are WSGI apps.

[josephernest]

Thanks @defnull.

The following code seems to work, is it what you were thinking about?

PS: I posted a question about this here: https://stackoverflow.com/questions/74130588/redirect-python-bottle-stderr-stdout-to-logging-logger-or-more-generally-usin

import logging, sys

logging.basicConfig(filename='test4.log', filemode='a', format='%(asctime)s %(levelname)s %(message)s', datefmt='%Y-%m-%d %H:%M:%S', level=logging.DEBUG)
log = logging.getLogger('foobar')
log.info("hello")

class LoggerWriter:
    def __init__(self, level):
        self.level = level
    def write(self, message):
        self.level(message.strip())
    def flush(self):
        pass

sys.stdout = LoggerWriter(log.info)
sys.stderr = LoggerWriter(log.error)

from bottle import route, run, template, request
@route('/')
def index():
    sqdf         # uncaught exception !!! 
    return "hello"

run()

[agalera]

@josephernest as defnull comments, you can use different application servers, like uwsgi, gunicorn... where all this is already covered.

https://uwsgi-docs.readthedocs.io/en/latest/Logging.html
https://docs.gunicorn.org/en/stable/settings.html#logging

just change your run() to:
run(server='gunicorn', workers=4, quiet=True)

[josephernest]

@agalera I am in a context where I need to keep the lowest number of dependencies.
The server is done with Bottle, the client is in the browser on the same (Windows) computer. This allows to make a web GUI for a software in Python. For it to be as simple as possible, I want to keep the default server, instead of adding gunicorn, uwsgi, etc. BTW gunicorn wouldn't work on Windows. Again client and server are on the same computer, so there is no real networking issue.

@agalera what do you think about this approach?