Mac OS X, Python, ports and dlib compilation

This topic is not very well covered by documentation and assumes that everything is as easy as possible…

OK, lets build…

cd python-examples
./compile_dlib_python_module.bat
-- The C compiler identification is AppleClang 6.1.0.6020053
-- The CXX compiler identification is AppleClang 6.1.0.6020053
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Boost version: 1.58.0
-- Found the following Boost libraries:
-- python
-- Found PythonLibs: /usr/lib/libpython2.7.dylib (found suitable version "2.7.6", minimum required is "2.6")
-- Looking for png_create_read_struct
-- Looking for png_create_read_struct - found
-- Looking for jpeg_read_header
-- Looking for jpeg_read_header - found
-- Searching for BLAS and LAPACK
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
-- Found LAPACK library
-- Found CBLAS library
-- Looking for cblas_ddot
-- Looking for cblas_ddot - found
-- Check for STD namespace
-- Check for STD namespace - found
-- Looking for C++ include iostream
-- Looking for C++ include iostream - found
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/suff/Downloads/dlib-18.16/python_examples/build
Scanning dependencies of target dlib
[ 0%] Building CXX object dlib_build/CMakeFiles/dlib.dir/base64/base64_kernel_1.o
[....]
[ 84%] Built target dlib
Scanning dependencies of target dlib_
[ 85%] Building CXX object CMakeFiles/dlib_.dir/src/dlib.cpp.o
[....]
[100%] Building CXX object CMakeFiles/dlib_.dir/src/gui.cpp.o
Linking CXX shared library dlib.so
[100%] Built target dlib_
Install the project...
-- Install configuration: "Release"
-- Installing: /Users/suff/Downloads/dlib-18.16/tools/python/../../python_examples/dlib.so

Perfect… So we have a SO file and we can copy it to site-packages of our virtualenv…

$ cp dlib.so ~/projects/dlib/venv/lib/python2.7/site-packages/
$ source ~/projects/dlib/venv/bin/activate
(venv) $ python 
Python 2.7.10 (default, May 23 2015, 22:32:15)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dlib
Segmentation fault: 11

Well… that was unexpected…

I’ve started to poke around the output of cmake and found something that caught my eye.

-- Found PythonLibs: /usr/lib/libpython2.7.dylib (found suitable version "2.7.6", minimum required is "2.6")

In my virtualenv I’m using Macports provided version of python2.7 and boost but dlib compiles against python provided by the system. While googling I found a stackoverflow post. It’s similar issue. The difference is that the brew version of Python2.7 is used. Adrian Rosebrock had fixed it by manually editing CMakeFiles/dlib_.dir/build.make and /CMakeFiles/dlib_.dir/link.txt.
After some digging on how cmake works and what’s going on in the compile_dlib_python_module.bat I found what needs to be done to compile dlib module.

$ mkdir build
$ cd build
$ cmake ../../tools/python -DUSE_SSE2_INSTRUCTIONS=ON -DUSE_SSE4_INSTRUCTIONS=ON -DUSE_AVX_INSTRUCTIONS=ON -DPYTHON_LIBRARY:FILEPATH=/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib
$ grep -Ri libpython2.7 *

CMakeCache.txt:PYTHONLIBS:PATH=/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib
CMakeCache.txt:PYTHON_LIBRARY:FILEPATH=/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib
CMakeCache.txt:dlib__LIB_DEPENDS:STATIC=general;/opt/local/lib/libboost_python-mt.dylib;general;/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib;general;dlib;
CMakeCache.txt:FIND_PACKAGE_MESSAGE_DETAILS_PythonLibs:INTERNAL=[/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib][/System/Library/Frameworks/Python.framework/Headers][v2.7.6(2.6)]
CMakeFiles/dlib_.dir/build.make:dlib.so: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib
CMakeFiles/dlib_.dir/link.txt:/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++   -DDLIB_PNG_SUPPORT -DDLIB_JPEG_SUPPORT -DDLIB_USE_BLAS -DDLIB_USE_LAPACK -O3 -DNDEBUG -dynamiclib -Wl,-headerpad_max_install_names  -o dlib.so -install_name @rpath/dlib.so CMakeFiles/dlib_.dir/src/dlib.cpp.o CMakeFiles/dlib_.dir/src/matrix.cpp.o CMakeFiles/dlib_.dir/src/vector.cpp.o CMakeFiles/dlib_.dir/src/svm_c_trainer.cpp.o CMakeFiles/dlib_.dir/src/svm_rank_trainer.cpp.o CMakeFiles/dlib_.dir/src/decision_functions.cpp.o CMakeFiles/dlib_.dir/src/other.cpp.o CMakeFiles/dlib_.dir/src/basic.cpp.o CMakeFiles/dlib_.dir/src/cca.cpp.o CMakeFiles/dlib_.dir/src/sequence_segmenter.cpp.o CMakeFiles/dlib_.dir/src/svm_struct.cpp.o CMakeFiles/dlib_.dir/src/image.cpp.o CMakeFiles/dlib_.dir/src/rectangles.cpp.o CMakeFiles/dlib_.dir/src/object_detection.cpp.o CMakeFiles/dlib_.dir/src/shape_predictor.cpp.o CMakeFiles/dlib_.dir/src/correlation_tracker.cpp.o CMakeFiles/dlib_.dir/src/gui.cpp.o /opt/local/lib/libboost_python-mt.dylib /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib dlib_build/libdlib.a /usr/lib/libpthread.dylib /opt/local/lib/libX11.dylib /opt/local/lib/libpng.dylib /usr/lib/libcblas.dylib /usr/lib/liblapack.dylib /usr/lib/libsqlite3.dylib

Now it looks OK. Let’s compile…

$ cmake --build . --config Release --target install
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   python
-- Found PythonLibs: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib (found suitable version "2.7.6", minimum required is "2.6")
-- Searching for BLAS and LAPACK
-- Found OpenBLAS library
-- Looking for sgetrf_single
-- Looking for sgetrf_single - not found
-- Found LAPACK library
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/suff/Downloads/dlib-18.16/python_examples/build
Scanning dependencies of target dlib
[  0%] Building CXX object dlib_build/CMakeFiles/dlib.dir/base64/base64_kernel_1.o
[  1%] Building CXX object dlib_build/CMakeFiles/dlib.dir/bigint/bigint_kernel_1.o
[....]
[ 84%] Built target dlib
Scanning dependencies of target dlib_
[ 85%] Building CXX object CMakeFiles/dlib_.dir/src/dlib.cpp.o
[....]
[100%] Building CXX object CMakeFiles/dlib_.dir/src/gui.cpp.o
Linking CXX shared library dlib.so
[100%] Built target dlib_
Install the project...
-- Install configuration: "Release"
-- Installing: ../python_examples/dlib.so
cp dlib.so ~/projects/dlib/venv/lib/python2.7/site-packages/
$ source ~/projects/dlib/venv/bin/activate
(venv) $ python 
Python 2.7.10 (default, May 23 2015, 22:32:15)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dlib
>>>

And we have a working dlib python module.

Mac OS X, Python, ports and dlib compilation

One thought on “Mac OS X, Python, ports and dlib compilation

  1. I added following line below log.info in setup.py file in dlib directory which resolved the Seg Fault issue.

    log.info(‘Configuring cmake …’)
    cmake_extra_arch += [‘-DPYTHON_LIBRARY={lib}’.format(lib=’/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib’)]

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.