andy-landy / traceback_with_variables Public

Notifications You must be signed in to change notification settings
Fork 32
Star 708

Adds variables to python traceback. Simple, lightweight, controllable. Customize formats and colors. Debug reasons of exceptions by logging or pretty printing colorful variable contexts for each frame in a stacktrace, showing every value. Dump locals environments after errors to console, files, and loggers. Works in Jupyter too.

License

MIT license

708 stars 32 forks Branches Tags Activity

Star

Notifications

andy-landy/traceback_with_variables

Branches Tags

Folders and files

Name		Name	Last commit message	Last commit date
Latest commit History 79 Commits
.github/workflows		.github/workflows
examples		examples
tests		tests
traceback_with_variables		traceback_with_variables
CHANGELOG.md		CHANGELOG.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
MANIFEST.in		MANIFEST.in
README.md		README.md
README.tmpl		README.tmpl
check_coverage_reports.py		check_coverage_reports.py
color_schemes.png		color_schemes.png
gen_readme.py		gen_readme.py
header.png		header.png
setup.py		setup.py

Repository files navigation

Python Traceback (Error Message) Printing Variables

Very simple to use and fast, but versatile when needed. Try for debug and keep for small-scale production.

"It is useless work that darkens the heart." - Ursula K. Le Guin

Tired of useless job of putting all your variables into debug exception messages? Just stop it. Automate it and clean your code. Once and for all.

I'm open to update this module to meet new use cases and to make using it easier and fun: so any proposal or advice or warning is very welcome and will be taken into account of course. When I started it I wanted to make a tool meeting all standard use cases. I think in this particular domain this is rather achievable, so I'll try. Note next_version branch also. Have fun!

Installation

pip install traceback-with-variables==2.2.1

conda install -c conda-forge traceback-with-variables=2.2.1

to use shorter tb alias in interactive mode call this once:

python3 -c 'from traceback_with_variables.tb_alias import create_tb_alias as c; c()'

Quick Start

Using without code editing, running your script/command/module:

traceback-with-variables tested_script.py ...srcipt's args...

Simplest usage, for the whole program:

from traceback_with_variables import activate_by_import

or just (if you added an alias by the above command)

import tb.a

Decorator, for a single function:

@prints_exc # def main(): or def some_func(...):

Context, for a single code block:

with printing_exc():

Work with traceback lines in a custom manner:

lines = list(iter_exc_lines(e))

No exception but you want to print the stack anyway?:

print_cur_tb()

Using a logger [with a decorator, with a context]:

with printing_exc(file_=LoggerAsFile(logger)): # or @prints_exc(file_=LoggerAsFile(logger)):

Print traceback in interactive mode after an exception:

>>> print_exc()

Customize any of the previous examples:

from traceback_with_variables import fmt fmt.max_value_str_len = 10000 fmt.skip_files_except = 'my_project' fmt.custom_var_printers = [] # show all variables, no skips, no hides

Colors

How does it save my time?

Turn a code totally covered by debug formatting from this:

def main(): sizes_str = sys.argv[1] h1, w1, h2, w2 = map(int, sizes_str.split()) - try: return get_avg_ratio([h1, w1], [h2, w2]) - except: - logger.error(f'something happened :(, variables = {locals()[:1000]}') - raise - # or - raise MyToolException(f'something happened :(, variables = {locals()[:1000]}') def get_avg_ratio(size1, size2): - try: return mean(get_ratio(h, w) for h, w in [size1, size2]) - except: - logger.error(f'something happened :(, size1 = {size1}, size2 = {size2}') - raise - # or - raise MyToolException(f'something happened :(, size1 = {size1}, size2 = {size2}') def get_ratio(height, width): - try: return height / width - except: - logger.error(f'something happened :(, width = {width}, height = {height}') - raise - # or - raise MyToolException(f'something happened :(, width = {width}, height = {height}')

into this (zero debug code):

+ from traceback_with_variables import activate_by_import def main(): sizes_str = sys.argv[1] h1, w1, h2, w2 = map(int, sizes_str.split()) return get_avg_ratio([h1, w1], [h2, w2]) def get_avg_ratio(size1, size2): return mean(get_ratio(h, w) for h, w in [size1, size2]) def get_ratio(height, width): return height / width

And obtain total debug info spending 0 lines of code:
return mean([get_ratio(h, w) for h, w in [size1, size2]]) .0 = h = 300 w = 0 File "./temp.py", line 13, in get_ratio return height / width height = 300 width = 0 builtins.ZeroDivisionError: division by zero">Traceback with variables (most recent call last): File "./temp.py", line 7, in main return get_avg_ratio([h1, w1], [h2, w2]) sizes_str = '300 200 300 0' h1 = 300 w1 = 200 h2 = 300 w2 = 0 File "./temp.py", line 10, in get_avg_ratio return mean([get_ratio(h, w) for h, w in [size1, size2]]) size1 = [300, 200] size2 = [300, 0] File "./temp.py", line 10, in return mean([get_ratio(h, w) for h, w in [size1, size2]]) .0 = h = 300 w = 0 File "./temp.py", line 13, in get_ratio return height / width height = 300 width = 0 builtins.ZeroDivisionError: division by zero

Automate your logging too:

logger = logging.getLogger('main') def main(): ... with printing_exc(file_=LoggerAsFile(logger)) ...

2020-03-30 18:24:31 main ERROR return mean([get_ratio(h, w) for h, w in [size1, size2]]) 2020-03-30 18:24:31 main ERROR .0 = 2020-03-30 18:24:31 main ERROR h = 300 2020-03-30 18:24:31 main ERROR w = 0 2020-03-30 18:24:31 main ERROR File "./temp.py", line 13, in get_ratio 2020-03-30 18:24:31 main ERROR return height / width 2020-03-30 18:24:31 main ERROR height = 300 2020-03-30 18:24:31 main ERROR width = 0 2020-03-30 18:24:31 main ERROR builtins.ZeroDivisionError: division by zero">2020-03-30 18:24:31 main ERROR Traceback with variables (most recent call last): 2020-03-30 18:24:31 main ERROR File "./temp.py", line 7, in main 2020-03-30 18:24:31 main ERROR return get_avg_ratio([h1, w1], [h2, w2]) 2020-03-30 18:24:31 main ERROR sizes_str = '300 200 300 0' 2020-03-30 18:24:31 main ERROR h1 = 300 2020-03-30 18:24:31 main ERROR w1 = 200 2020-03-30 18:24:31 main ERROR h2 = 300 2020-03-30 18:24:31 main ERROR w2 = 0 2020-03-30 18:24:31 main ERROR File "./temp.py", line 10, in get_avg_ratio 2020-03-30 18:24:31 main ERROR return mean([get_ratio(h, w) for h, w in [size1, size2]]) 2020-03-30 18:24:31 main ERROR size1 = [300, 200] 2020-03-30 18:24:31 main ERROR size2 = [300, 0] 2020-03-30 18:24:31 main ERROR File "./temp.py", line 10, in 2020-03-30 18:24:31 main ERROR return mean([get_ratio(h, w) for h, w in [size1, size2]]) 2020-03-30 18:24:31 main ERROR .0 = 2020-03-30 18:24:31 main ERROR h = 300 2020-03-30 18:24:31 main ERROR w = 0 2020-03-30 18:24:31 main ERROR File "./temp.py", line 13, in get_ratio 2020-03-30 18:24:31 main ERROR return height / width 2020-03-30 18:24:31 main ERROR height = 300 2020-03-30 18:24:31 main ERROR width = 0 2020-03-30 18:24:31 main ERROR builtins.ZeroDivisionError: division by zero

Free your exceptions of unnecessary information load:

def make_a_cake(sugar, eggs, milk, flour, salt, water): is_sweet = sugar > salt is_vegan = not (eggs or milk) is_huge = (sugar + eggs + milk + flour + salt + water > 10000) if not (is_sweet or is_vegan or is_huge): raise ValueError('This is unacceptable, look why!') ...

-- Should I use it after debugging is over, i.e. in production?

Yes, of course! That way it might save you even more time (watch out for sensitive data like passwords and tokens in you logs). Note: you can deploy more serious monitoring frameworks, e.g. Sentry

Stop this tedious practice in production:

step 1: Notice some exception in a production service.
step 2: Add more printouts, logging, and exception messages.
step 3: Rerun the service.
step 4: Wait till (hopefully) the bug repeats.
step 5: Examine the printouts and possibly add some more info (then go back to step 2).
step 6: Erase all recently added printouts, logging and exception messages.
step 7: Go back to step 1 once bugs appear.
-- Is it fast? I have huge lists and dicts and data objects!

Data size doesn't matter! Printing is smart, so only visible data portions are inpsected, e.g.

list(range(100000000)) becomes just "[0, 1, 2, ...9998, 99999999]" in 0.00012s

Use fmt's .max_value_str_len, and .ellipsis_rel_pos (0.0 to 1.0) to tweak the output.

Examples and recipes

Reference

All functions have `fmt=` argument, a `Format` object with fields:

max_value_str_len max length of each variable string, -1 to disable, default=1000
objects_details depth of details of objects inspection
ellipsis_rel_pos when truncating long strings where to put the "...", from 0.0 to 1.0, default=0.7
max_exc_str_len max length of exception, should variable print fail, -1 to disable, default=10000
before number of code lines before the raising line, default=0
after number of code lines after the raising line, default=0
ellipsis_ string to denote long strings truncation, default=...
skip_files_except use to print only certain files; list of regexes, ignored if empty, default=None
brief_files_except use to print variables only in certain files; list of regexes, ignored if empty, default=None
custom_var_printers list of pairs of (filter, printer); filter is a name fragment, a type or a function or a list thereof; printer returns None to skip a var, standard ones are hide, skip, show
color_scheme is None or one of ColorSchemes: .none , .common, .nice, .synthwave. None is for auto-detect

`activate_by_import`

Just import it. No arguments, for real quick use in regular Python.

from traceback_with_variables import activate_by_import

`activate_in_ipython_by_import`

Just import it. No arguments, for real quick use in Jupyter or IPython.

from traceback_with_variables import activate_in_ipython_by_import

`global_print_exc`

Call once in the beginning of your program, to change how traceback after an uncaught exception looks.

def main(): override_print_exc(...)

`global_print_exc_in_ipython`

Call once in the beginning of your program, to change how traceback after an uncaught exception looks.

def main(): override_print_exc(...)

`print_exc`

Prints traceback for a given/current/last (first being not None in the priority list) exception to a file, default=sys.stderr. Convenient for manual console or Jupyter sessions or custom try/except blocks. Note that it can be called with a given exception value or it can auto discover current exception in an except: block or it can auto descover last exception value (long) after try/catch block.

print_exc()

`print_cur_tb`

Prints current traceback when no exception is raised.

print_cur_tb()

`prints_exc`

Function decorator, used for logging or simple printing of scoped tracebacks with variables. I.e. traceback is shorter as it includes only frames inside the function call. Program exiting due to unhandled exception still prints a usual traceback.

@prints_exc def f(...): @prints_exc(...) def f(...):

`printing_exc`

Context manager (i.e. with ...), used for logging or simple printing of scoped tracebacks with variables. I.e. traceback is shorter as it includes only frames inside the with scope. Program exiting due to unhandled exception still prints a usual traceback.

with printing_exc(...):

FAQ

In Windows console crash messages have no colors.

The default Windows console/terminal cannot print [so called ansi] colors, but this is fixable , especially with modern Windows versions. Therefore colors are disabled by default, but you can enable them and check if it works in your case. You can force enable colors by passing --color-scheme common (for complete list of colors pass --help) console argument.
Windows console prints junk symbols when colors are enabled.

The default Windows console/terminal cannot print [so called ansi] colors, but this is fixable , especially with modern Windows versions. If for some reason the colors are wrongly enabled by default, you can force disable colors by passing --color-scheme none console argument.
Bash tools like grep sometimes fail to digest the output when used with pipes (|) because of colors.

Please disable colors by passing --color-scheme none console argument. The choice for keeping colors in piped output was made to allow convenient usage of head, tail, file redirection etc. In cases like | grep it might have issues, in which case you can disable colors.
Output redirected to a file in > output.txt manner has no colors when I cat it.

This is considered a rare use case, so colors are disabled by default when outputting to a file. But you can force enable colors by passing --color-scheme common (for complete list of colors pass --help) console argument.
activate_by_import or global_print_exc don't work in Jupyter or IPython as if not called at all.

In Jupyter or IPython you should use activate_in_ipython_by_import or global_print_exc_in_ipython. IPython handles exceptions differently than regular Python.
The server framework (flask, streamlit etc.) still shows usual tracebacks.

In such frameworks tracebacks are printed not while exiting the program (the program continues running), hence you should override exception handling in a manner proper for the given framework. Please address the flask example.
How do I reduce output? I don't need all files or all variables.

Use skip_files_except, brief_files_except, custom_var_printers to cut excess output.
I have ideas about good colors.

Please fork, add a new ColorScheme to ColorSchemes and create a Pull Request to next_version branch. Choose the color codes and visually test it like python3 -m traceback_with_variables.main --color-scheme {its name} examples/for_readme_image.py.
My code doesn't work.

Please post your case. You are very welcome!
Other questions or requests to elaborate answers.

Please post your question or request. You are very welcome!