Trouble getting current mitmproxy working on 2.2.2
-
I'm trying to get mitmproxy-0.12.1 from http://mitmproxy.org working on pfSense 2.2.2. FreeBSD ports still includes mitmproxy-0.10.1, where Transparent mode isn't supported. I understand this was fixed in mitmproxy-0.11.
I'm a noob at massaging build dependencies manually so I may well be missing some obvious -devel library.
All attempts to either 'pip install mitmproxy' or to build from source fail at the same point.
Here's netlib failing to build.
running build_ext building '_cffi__x88a8fd90x94e38fe2' extension creating /tmp/pip-build-SrTHd4/netlib/netlib/__pycache__/netlib creating /tmp/pip-build-SrTHd4/netlib/netlib/__pycache__/netlib/__pycache__ cc -fno-strict-aliasing -O2 -pipe -fstack-protector -fno-strict-aliasing -DNDEBUG -fPIC -I/usr/local/include/python2.7 -c netlib/__pycache__/_cffi__x88a8fd90x94e38fe2.c -o /tmp/pip-build-SrTHd4/netlib/netlib/__pycache__/netlib/__pycache__/_cffi__x88a8fd90x94e38fe2.o -w In file included from /usr/local/lib/gcc5/gcc/x86_64-portbld-freebsd10.1/5.1.0/include-fixed/syslimits.h:7:0, from /usr/local/lib/gcc5/gcc/x86_64-portbld-freebsd10.1/5.1.0/include-fixed/limits.h:34, from /usr/local/include/python2.7/Python.h:19, from netlib/__pycache__/_cffi__x88a8fd90x94e38fe2.c:2: /usr/local/lib/gcc5/gcc/x86_64-portbld-freebsd10.1/5.1.0/include-fixed/limits.h:168:61: fatal error: limits.h: No such file or directory compilation terminated.
Full log: http://pastebin.com/YYzz9qCX
(I've also tried with gcc-4.8.4 and gcc49-4.9.3, it didn't make a difference.)
Here's cffi failing to build:
http://pastebin.com/PRaDaFM1
Here's the dump of all the packages I currently have installed.
http://pastebin.com/Czt0F8cHedit: here's what I have installed via pip http://pastebin.com/yiZ9jWVx
Although I have py27-cffi-0.8.6_3 and libffi-3.2.1 installed, but neither seems to satisfy the dependency.
I also filed this with the mitmproxy folks at
https://github.com/mitmproxy/mitmproxy/issues/609
Any pointers on getting this sorted would be greatly appreciated. -
Have you looked for the limits.h file on your system in the folder/directory location its looking in?
The GCC library is fairly common imo, dont know about freebsd but in Linux its common, so it could be missing from the library package and or the mitm script might need tweaking. Have you looked to see if limits.h is in a different location on your machine?
I havent looked at your links as this machine I'm on only allows access to pfsense forum webpages out of the whole web.
-
Have you looked for the limits.h file on your system in the folder/directory location its looking in?
The GCC library is fairly common imo, dont know about freebsd but in Linux its common, so it could be missing from the library package and or the mitm script might need tweaking. Have you looked to see if limits.h is in a different location on your machine?
The only limits.h I can find is /usr/local/lib/gcc5/gcc/x86_64-portbld-freebsd10.1/5.1.0/include-fixed/limits.h itself. The line referenced in the error is just the include:
#ifdef _GCC_NEXT_LIMITS_H #include_next <limits.h> /* recurse down to the real one */ #endif</limits.h>
I havent looked at your links as this machine I'm on only allows access to pfsense forum webpages out of the whole web.
Heh, very impressive. Given how much of the net I'm blocking now, I'm not far from that. Thanks for the response!
-
The only limits.h I can find is /usr/local/lib/gcc5/gcc/x86_64-portbld-freebsd10.1/5.1.0/include-fixed/limits.h itself. The line referenced in the error is just the include:
#ifdef _GCC_NEXT_LIMITS_H #include_next <limits.h> /* recurse down to the real one */ #endif</limits.h>
Ok so the file exists, but in the code its just got #include_next which would suggest a path to the folder/directory might not exist. Might be worth checking the paths, alternatively what would happen if you put the full path into
#include_next
making it
#include_nextThe error message No such file or directory is the clue at this stage.
Heh, very impressive. Given how much of the net I'm blocking now, I'm not far from that. Thanks for the response!
PC's can still communicate through DNS in ways and the same with other network protocols as you might find/work out in time. ;)
Google and others dont provide a DNS services just for everyone's pleasure.
Edit. Phishing is also quite effective with multiple profiles as well.
-
Gotcha.
Fwiw, I was able to uninstall nearly all of the extraneous cruft and get mitmproxy-0.12 working using python virtualenv.
On both my pfsense box and on a virgin VMware copy of FreeBSD 10.1 from ftp://ftp.freebsd.org/pub/FreeBSD/releases/VM-IMAGES/10.1-RELEASE/amd64/Latest/
I installed the following (pkg first, then python, then I believe py27-lxml, which pulls in most of the rest.)
gettext-runtime-0.19.4 indexinfo-0.2.3 libffi-3.2.1 libgcrypt-1.6.3 libgpg-error-1.19_1 libxml2-2.9.2_2 libxslt-1.1.28_6 pkg-1.5.3 py27-lxml-3.4.1_1 py27-setuptools27-5.5.1_1 python-2.7_2,2 python2-2_3 python27-2.7.9_1
Also on both pfSense and VMware I grabbed https://bootstrap.pypa.io/get-pip.py and ran 'python get-pip.py' to get that set up.
I ran 'pip install virtualenv lxml' giving me the following with pip:
lxml (3.4.4) pip (7.0.3) setuptools (17.0) virtualenv (13.0.3) wheel (0.24.0)
Then in VMware:
# mkdir mitmproxy_env # virtualenv --no-site-packages mitmproxy_env # source mitmproxy_env/bin/activate.csh
At this point, we're inside the mitmproxy_env virtualenv (read more at https://virtualenv.pypa.io/en/latest/userguide.html.)
pip install cryptography pyOpenSSL mitmproxy
This should result in the following python modules installed:
# pip list backports.ssl-match-hostname (3.4.0.2) blinker (1.3) certifi (2015.4.28) cffi (1.1.0) ConfigArgParse (0.9.3) cryptography (0.9) enum34 (1.0.4) hpack (1.0.1) idna (2.0) ipaddress (1.0.7) lxml (3.4.4) mitmproxy (0.12.1) netlib (0.12.1) passlib (1.6.2) Pillow (2.8.1) pip (7.0.3) pyasn1 (0.1.7) pycparser (2.13) pyOpenSSL (0.15.1) pyperclip (1.5.11) setuptools (17.0) six (1.9.0) tornado (4.2) urwid (1.3.0) wheel (0.24.0)
It should now be configured and operational inside VMware:
# which mitmproxy /root/mitm_env/bin/mitmproxy # mitmproxy --version mitmproxy 0.12.1 # mitmproxy -T --host
After confirming that it does in fact run, type 'deactivate' to exit the virtualenv. At this point you can 'tar zcvf mitmproxy_env.tar.gz ./mitmproxy_env' and move the tarball over to the pfSense machine.
On the pfSense box:
# tar zxvf mitmproxy_env.tar.gz # source mitmproxy_env/bin/activate.csh # setenv LANG en_US.UTF-8 ; mitmproxy -T --host
And it should be up and running. This lets you avoid setting up full dev toolchain and messing with build dependencies like I started out. Hopefully this will be of some use to others.
-
Thanks, this is an approach which will probably work for getting Dtrace working on pfsense. :D
-
Fookme…
This is over my head.
#sigh
-
Its not hard, just ask the right questions, give it a go. ;)
What dont you know?
Edit:
Worth starting a getting Dtrace to work on pfsense thread at this point?