panicparse
Parses panic stack traces, densifies and deduplicates goroutines with similar
stack traces. Helps debugging crashes and deadlocks in heavily parallelized
process.
panicparse helps make sense of Go crash dumps:
Features
- >50% more compact output than original stack dump yet more readable.
- Exported symbols are bold, private symbols are darker.
- Stdlib is green, main is yellow, rest is red.
- Deduplicates redundant goroutine stacks. Useful for large server crashes.
- Arguments as pointer IDs instead of raw pointer values.
- Pushes stdlib-only stacks at the bottom to help focus on important code.
- Usable as a library!
- Warning: please pin the major version (i.e. vendor it via
dep) as breaking changes happen on
major version update.
- Warning: please pin the major version (i.e. vendor it via
- Parses the source files if available to augment the output.
- Works on Windows.
Authors
panicparse was created with ❤️️ and passion by Marc-Antoine
Ruel and
friends.
Installation
go get github.com/maruel/panicparse/cmd/pp
Usage
Piping a stack trace from another process
TL;DR
- Ubuntu (bash v4 or zsh):
, & - OSX, install bash 4+, then:
, & - Windows or OSX with stock bash v3:
2>&1, - Fish shell:
^,
Longer version
pp streams its stdin to stdout as long as it doesn't detect any panic.
panic() and Go's native deadlock detector print to
stderr via the native print()
function.
Bash v4 or zsh: , & tells the shell to redirect stderr to stdout,
it's an alias for 2>&1, (bash
v4,
zsh):
go test -v, &pp
Windows or OSX native bash (which is
3.2.57): They don't
have this shortcut, so use the long form:
go test -v 2>&1, pp
Fish: It uses ^ for stderr
redirection
so the shortcut is ^, :
go test -v ^, pp
PowerShell: It has broken 2>&1 redirection. The workaround is to shell out to cmd.exe. :(
Investigate deadlock
On POSIX, use Ctrl-\ to send SIGQUIT to your process, pp will ignore
the signal and will parse the stack trace.
Parsing from a file
To dump to a file then parse, pass the file path of a stack trace
go test 2> stack.txt
pp stack.txt
Tips
Disable inlining
Starting with go1.11, the toolchain starts to inline more often. This causes
traces to be less informative. You can use the following to help diagnosing
issues:
go install -gcflags '-N -l' path/to/foo
foo, & pp
or
go test -gcflags '-N -l' ./..., & pp
GOTRACEBACK
Starting with Go 1.6, GOTRACEBACK defaults
to single instead of all / 1 that was used in 1.5 and before. To get all
goroutines trace and not just the crashing one, set the environment variable:
export GOTRACEBACK=all
or set GOTRACEBACK=all on Windows. Probably worth to put it in your .bashrc.
Updating bash on OSX
Install bash v4+ on OSX via homebrew or
macports. Your future self will appreciate having
done that.
If you have /usr/bin/pp installed
If you try pp for the first time and you get:
Creating tables and indexes...
Done.
and/or
/usr/bin/pp5.18: No input files specified
you may be running the Perl PAR Packager instead of panicparse.
You have two choices, either you put $GOPATH/bin at the begining of $PATH or
use long name panicparse with:
go get github.com/maruel/panicparse
then using panicparse instead of pp:
go test 2> panicparse