Python Logging to File: A Beginners Guide

What is Logging?

Logging is a python module in the standard library that offers the ability to use the framework for releasing python logging to file messages from Python programs. The software uses python logging to track events that occur while running.

It is a crucial tool in software creation, operation, and debugging.

Python Logging to file helps you understand what happens in your application, why, and when.   

Python Logging makes it easier to identify problems and accelerates the debugging process because it leaves a cookie-crumb trail that leads directly to the problem.

You can troubleshoot faults by python logging relevant data from the appropriate locations. You can also utilize the data to examine the application’s performance to plan for scaling or look at usage trends to prepare for marketing.

It is used instead of the print() statement.

We can add python logging to our program by adding the following:

            import logging

Example:

import logging

logging. debug('This is a debug message')
logging. info('This is an info message')
logging. warning('This is a warning message')
logging. error('This is an error message)
logging. critical('This is a critical message')

The output displays the root, the python logging module’s name to its default logger, and the severity level before each message. The default output format separates the level, name, and message with a colon (:) and can customize to include other information like the timestamp and line number.

Why is Logging used?

When you run a python script, you can know what part of the script is getting executed and check which variables are on hold.

 While performing small programs, you can use the “print()” statement and get the required messages. But while working on bigger problems, you must use the python logging method.

Thus, python logging to file helps:

1. Control message level to only log necessary messages
2. Control the shown or saved logs.
3. Utilize built-in message templates to regulate how the logs are formatted.
4. Identify the module from which the messages are coming.

How does Python Logging work?

The Python logging module is a robust module that both enterprises and beginners use. This python logging module allows users to arrange various control handlers and send log messages to these handlers.

The python logging module has five levels that define the severity of the occurrences available in the logging module.

1. DEBUG- It gives detailed information and diagnoses problems.
2. INFO- It confirms if things are working as expected.
3. WARNING- Serves as a warning that something unexpected occurred or that you may encounter difficulty shortly.
4. ERROR- It is a warning when the software hasn’t completed some tasks, and we are in serious trouble.
5. CRITICAL- It indicates a severe error, which may prevent the program from continuing to run.

The levels mentioned above are sufficient to tackle any problems. The levels’ associated numerical values are listed below.

The python logging module has a wide range of functionality. There are numerous constants, classes, and methods in it.

Here are a few methods

Methods:

• Logger.info(msg)
• Logger. warning(msg)
• Logger. error(msg)
• Logger. critical(msg)
• Logger.log(lvl, msg)
• Logger. exception(msg)
• Logger.setLevel(lvl)
• Logger.addFilter(filt)
• Logger.removeFilter(filt)
• Logger.filter(record)
• Logger.addHandler(hdlr)
• Logger.removeHandler(hdlr)
• Logger.hasHandlers().

Example:

import logging   

logging.debug('The debug message is displaying') 
logging.info('The info message is displaying') 
logging.warning('The warning message is displaying') 
logging.error('The error message is displaying') 
logging.critical('The critical message is displaying')

Output:

WARNING:root: The warning message is displaying

ERROR:root: The error message is displaying

CRITICAL:root: The critical message is displaying

As we can see from the output above, each message is along with the root, which is the name given by the python logging module’s default logger. In addition, the messages print in default output format, with a colon (:) separating the message from the level name. Because the log module, by default, logs messages with a severity level of WARNING, ERROR, and CRITICAL, we can see that the debug() and info() messages didn’t display any messages.

Basic Configurations

The primary function of python logging is to record events and store them in a file. The basicConfig(**kwarg), used to configure the logging, is provided by the logging module.

Some commonly used arguments in python logging are

level – The root level determines the chosen severity level.

filename –specifies the file.

filemode-This feature opens a file in a particular mode. We can append the content because the opening file’s default mode is a.

format – The format specifies how a log message should be formatted.

The basicConfig() function allows us to use filename and filemode as well as format characteristics to determine the message’s format rather than displaying it on the console. Let’s analyze the following example.

Example:

import logging 

logging.basicConfig(filename='msg.log', filemode='w', format='%(name)s - %(levelname)s - %(message)s') 
logging.warning('This will log to a file ')

Output: root -WARNING – This will log to a file. 

The above output is in the msg.log file instead of the console. We opened the file with the letter “w,” which denotes that it opens in “write mode. The log file updates each time the program executes if basicConfig() calls. The extra arguments can be passed to the basicConfig() function to change it.

Formatting the Output

A string supplied to the computer as a message to log can be changed to meet our needs. The given string has a few fundamental components that are also a part of the Logrecord.

Example:

import logging 

logging.basicConfig(format='%(process)d-%(levelname)s-%(message)s') 
logging.warning('This is a Warning Message')

Output:

18472-WARNING-This is a Warning Message

Explanation-A string with Logrecord characteristics in whatever form we desire to pass as the format argument in python logging.

Logging Variable Data

We occasionally add dynamic data from the application to the log. The python logging methods require a string as an argument; thus, it recommends formatting a string with variable data before passing it to a log function.

However, we can also use a format string for the message and add the variable data as an argument in python logging to the file.

Example:

import logging 

name = 'Harry Potter '
logging.error('%s raised an error', name)

Output:

root: Harry Potter raised an error

 Explanation: The message would contain ambiguous variable data representing the arguments supplied to the method.

Capturing Stack Traces

We may record an application’s complete stacks of traces using the Python logging module. The Python logging function has an argument called exc_info parameter that, if set to True, can record information about exceptions.

Example: 

import logging   

a = 10 
b = 0  

try:
 c = a / b 
except exception as e:
 logging.error("Exception occurred", exc_info=True)

Output:     File “main.py,” line 5, in <module>

       c= a / b

                   ZeroDivisionError: division by zero

Explanation: The output will not alert us to the exception if we do not set exc info to true. If an error produces the following output, tracking down thousands of lines of code would be challenging.

 ERROR:root: Exception has occurred.

The Python logging module offers the exception() method, records a message with the text ERROR, and includes the relevant details. Call the logging.exception() function, in the same way, you would call python logging to use it. (exc info = True) error

Classes and Functions

The default logger is known as the root. When logging.debug(), logging.error(), etc., are called, and the python logging module invokes. By establishing a Logger class object, we can also define our logger.

The classes and functions listed below are those defined in the python logging module.

Logger- Calls the functions directly using the logger object.

LogRecord – It automatically creates a log record file containing information about every event logged, including the logger’s name, the function, the line number, the message, and more.

Handler– The LogRecord is sent to the output endpoint using the handlers. The subclasses of a Handler are FileHandler, StreamHandler, HTTPHandler, and SMTTPHandler.

Formatters- Formatters are used to specify the output’s structure.

The default uses the raw message if we do not have a message to format. The default format of the date format is

%Y-%m-%d %H:%M:%S 

Usually, we work with objects of the Logger class produced by a python logging method called getLogger(name). Therefore, the same logger object’s reference returns if the getLogger() method is used more than once with the same name.

Working with Handlers

Handlers are typically used to set up loggers and simultaneously send logs to numerous locations. The log messages are sent by email, an HTTP file, or the regular output stream.

Example:

import logging   

# Create a custom logger_obj 
logger_obj = logging.getLogger(__name__)   

# Create handlers 
w_handler = logging.StreamHandler() 
e_handler = logging.FileHandler('file.log')
w_handler.setLevel(logging.WARNING) 
e_handler.setLevel(logging.ERROR) 

# Create formatters and add them to handlers 
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s') 
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') 
w_handler.setFormatter(c_format) 
e_handler.setFormatter(f_format) 

# Add handlers to the logger_obj 
logger_obj.addHandler(w_handler)
logger_obj.addHandler(e_handler) 
logger_obj.warning('This is a warning message')
logger_obj.error('This is an error message ')

Output

                        __main__ – WARNING – This is a warning

__main__ – ERROR – This is an error

Explanation: In the program, a particular logger called logger obj generates, as well as a LogRecord that saves all records of logging events and passes them to all of the w handlers and e handlers the custom logger has.

A stream handler with the level WARNING is the w handler. To create the output in the format string and print it to the screen, it accepts the log from the LogRecord.

A file handler with the level ERROR is the e handler. We ignore the LogRecord because it is at the WARNING level.

Other Configuration Methods

We can set up Python Logging as previously demonstrated using the module and class functions, as well as by generating a configuration file or dictionary and loading it with the appropriate fileConfig() or dictConfig() method. These are helpful if you need to modify the logging settings for an application that is already running.

Conclusion

The logging module is known for its extreme flexibility. It helps track the logging records and shows the user the appropriate message. In addition, it offers the freedom to design unique handler classes, log levels, and other valuable techniques. Hope this article was helpful for you.