graftcp: inspect any program’s HTTPS traffic through a proxy!
April 30, 2022
April 30, 2022
Recently, I needed to sniff an app’s HTTPS traffic.
While sniffing plaintext HTTP traffic is easy, by targeting the
transport layer with tools like tcpdump(8)
or Wireshark, HTTPS is another beast.
Because of the TLS encryption, all we see at the transport layer is a bunch of unusable encrypted data (and that’s the whole point of HTTPS). So we need to resort to solutions at a higher layer in the stack.
My go-to for this kind of task is mitmproxy, an interactive HTTPS proxy, as well as its headless counterpart mitmdump.
But those are only half of the solution. A proxy server is useless until we route HTTPS traffic through it. And depending on the context, this task can go from pretty trivial to quite tricky.
There’s 3 main ways that you can use to configure a proxy:
I already gave an introduction to mitmproxy last year on the blog, in which I explored the OS level and the app level, but I’ll give a quick refresher here.
Your OS usually lets you configure a proxy in the networking settings. For example on macOS it’s in the advanced network preferences, and on Android it’s in the advanced Wi-Fi settings.
It’s a good way to globally configure a proxy, but there’s no guarantee that apps are going to respect it. Typically, the default browser that ships with the OS (e.g. Safari) will use it, and some third-party browsers might too, but in general most other apps just ignore it. Not so good.
While most apps don’t respect OS-level proxy configuration, some can provide you with a way to configure a proxy at their own level.
Typically this will be Firefox’s connection settings, with its very own
proxy configuration, Spotify’s advanced settings that let you configure a proxy,
or more recently I’ve explored the --proxy_hostname argument
to the osqueryd program.
But it’s up to the app’s developers to decide if they want or not to let you configure a proxy, and how rigorously they use it… (they might use the proxy for some requests but not all of them).
Finally, there are two pretty commonly used environment variables
(although I believe not standard per se) to configure a proxy:
http_proxy and https_proxy. They respectively configure a proxy to
route HTTP and HTTPS traffic through.
For example the Python language supports those
in its native urllib package, and while Node.js doesn’t,
the popular axios library
also supports them
out of the box.
So basically, it might or might not work depending on the implementation of the software that you’re using, but it’s definitely worth trying!
So far all we’ve done is configuring a proxy in interfaces that explicitly allow us to set a proxy. But sometimes this is just not enough. That’s when we resort to ways to configure a proxy in places that don’t explicitly let us do so. 😏
There’s two ways to do this. The most common one is to leverage
LD_PRELOAD with dynamically linked binaries to override symbols in a
library. This is the approach that tsocks(1),
ProxyChains and ProxyChains-NG
use, hijacking the connect(2)
libc function to
route requests through the proxy of your choice.
This is a great method when using programs that are dynamically linked against libc, but it will fail for statically liked programs, as well as programs that don’t use libc (Go programs for example).
This is where graftcp shines.
Instead of hooking at the libc level, it leverages
ptrace(2) to modify the
connect(3) syscall arguments!
Essentially, it’s acting at a lower level and that’s how it’s able to
proxy against statically liked programs that don’t use libc. Their
detailed how does it work
explanation is really worth a read.
git clone https://github.com/hmgle/graftcp.git
cd graftcp
make
From there, you can use local/graftcp-local to start the graftcp
server, and configure it to use your proxy (for example mitmproxy
starts a HTTP proxy on port 8080 by default). Because graftcp also
defaults to a SOCKS5 proxy on localhost:1080, we need to force it to
use the HTTP proxy we configured by using --select_proxy_mode only_http_proxy:
local/graftcp-local --http_proxy localhost:8080 --select_proxy_mode only_http_proxy
We can do the same thing by “emptying” the preconfigured SOCKS5 proxy:
local/graftcp-local --http_proxy localhost:8080 --socks5 ''
Or we can instead run mitmproxy as a SOCKS5 proxy, here on port 1080 (the default for graftcp):
mitmproxy --mode socks5 -p 1080
Then we can run local/graftcp-local without arguments and it’ll just
work.
Either way, once the proxy and local/graftcp-local program is started,
we can prefix any command with ./graftcp to force it to run its
network calls through the proxy!
./graftcp curl https://www.codejam.info/
However, this should complain that the SSL certificate from mitmproxy is
untrusted. We can make it go through by appending -k (--insecure) to
the curl command:
./graftcp curl https://www.codejam.info/ --insecure
A better solution though would be to add the mitmproxy CA certificate
found in ~/.mitmproxy/mitmproxy-ca-cert.pem to the system trusted
certificates. This will vary depending on your OS and distribution, but
in my case that would be done with:
sudo trust anchor ~/.mitmproxy/mitmproxy-ca-cert.pem
Then the curl command should work without --insecure!
Finally, if you don’t want to bother running local/graftcp-local and
./graftcp separately, you can instead use local/mgraftcp. If you
still use the SOCKS5 proxy on port 1080:
local/mgraftcp curl https://www.codejam.info/
Or with a HTTP proxy on port 8080:
local/mgraftcp --http_proxy localhost:8080 --select_proxy_mode only_http_proxy curl https://www.codejam.info/
This is useful if you only want to use graftcp for a single command, or
don’t mind configure the proxy settings every single time. Otherwise the
graftcp server method with local/graftcp-local works better as you
only have to configure your proxy once and any call to ./graftcp will
use it!
graftcp is a really powerful tool that allows you to redirect HTTPS traffic through a proxy of your choice, even in situations where this wouldn’t be allowed or planned for.
Because I love inspecting programs’ network traffic to know how they work, and it’s not always easy to get access to their requests logs, graftcp is now a go-to of mine for this kind of task, as it’s proven to work flawlessly and very reliably, even with statically linked binaries and programs that don’t link against libc like it’s the case with Go!
I hope you learnt something with this post, and I wish you a happy network sniffing. 🤘