Mac OS X, Python, ports and dlib compilation

This topic is not very well covered by documentation and assumes that everything is as easy as possible…

OK, lets build…

cd python-examples
./compile_dlib_python_module.bat
-- The C compiler identification is AppleClang 6.1.0.6020053
-- The CXX compiler identification is AppleClang 6.1.0.6020053
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Boost version: 1.58.0
-- Found the following Boost libraries:
-- python
-- Found PythonLibs: /usr/lib/libpython2.7.dylib (found suitable version "2.7.6", minimum required is "2.6")
-- Looking for png_create_read_struct
-- Looking for png_create_read_struct - found
-- Looking for jpeg_read_header
-- Looking for jpeg_read_header - found
-- Searching for BLAS and LAPACK
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
-- Found LAPACK library
-- Found CBLAS library
-- Looking for cblas_ddot
-- Looking for cblas_ddot - found
-- Check for STD namespace
-- Check for STD namespace - found
-- Looking for C++ include iostream
-- Looking for C++ include iostream - found
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/suff/Downloads/dlib-18.16/python_examples/build
Scanning dependencies of target dlib
[ 0%] Building CXX object dlib_build/CMakeFiles/dlib.dir/base64/base64_kernel_1.o
[....]
[ 84%] Built target dlib
Scanning dependencies of target dlib_
[ 85%] Building CXX object CMakeFiles/dlib_.dir/src/dlib.cpp.o
[....]
[100%] Building CXX object CMakeFiles/dlib_.dir/src/gui.cpp.o
Linking CXX shared library dlib.so
[100%] Built target dlib_
Install the project...
-- Install configuration: "Release"
-- Installing: /Users/suff/Downloads/dlib-18.16/tools/python/../../python_examples/dlib.so

Perfect… So we have a SO file and we can copy it to site-packages of our virtualenv…

$ cp dlib.so ~/projects/dlib/venv/lib/python2.7/site-packages/
$ source ~/projects/dlib/venv/bin/activate
(venv) $ python 
Python 2.7.10 (default, May 23 2015, 22:32:15)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dlib
Segmentation fault: 11

Well… that was unexpected…

I’ve started to poke around the output of cmake and found something that caught my eye.

-- Found PythonLibs: /usr/lib/libpython2.7.dylib (found suitable version "2.7.6", minimum required is "2.6")

In my virtualenv I’m using Macports provided version of python2.7 and boost but dlib compiles against python provided by the system. While googling I found a stackoverflow post. It’s similar issue. The difference is that the brew version of Python2.7 is used. Adrian Rosebrock had fixed it by manually editing CMakeFiles/dlib_.dir/build.make and /CMakeFiles/dlib_.dir/link.txt.
After some digging on how cmake works and what’s going on in the compile_dlib_python_module.bat I found what needs to be done to compile dlib module.

$ mkdir build
$ cd build
$ cmake ../../tools/python -DUSE_SSE2_INSTRUCTIONS=ON -DUSE_SSE4_INSTRUCTIONS=ON -DUSE_AVX_INSTRUCTIONS=ON -DPYTHON_LIBRARY:FILEPATH=/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib
$ grep -Ri libpython2.7 *

CMakeCache.txt:PYTHONLIBS:PATH=/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib
CMakeCache.txt:PYTHON_LIBRARY:FILEPATH=/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib
CMakeCache.txt:dlib__LIB_DEPENDS:STATIC=general;/opt/local/lib/libboost_python-mt.dylib;general;/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib;general;dlib;
CMakeCache.txt:FIND_PACKAGE_MESSAGE_DETAILS_PythonLibs:INTERNAL=[/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib][/System/Library/Frameworks/Python.framework/Headers][v2.7.6(2.6)]
CMakeFiles/dlib_.dir/build.make:dlib.so: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib
CMakeFiles/dlib_.dir/link.txt:/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++   -DDLIB_PNG_SUPPORT -DDLIB_JPEG_SUPPORT -DDLIB_USE_BLAS -DDLIB_USE_LAPACK -O3 -DNDEBUG -dynamiclib -Wl,-headerpad_max_install_names  -o dlib.so -install_name @rpath/dlib.so CMakeFiles/dlib_.dir/src/dlib.cpp.o CMakeFiles/dlib_.dir/src/matrix.cpp.o CMakeFiles/dlib_.dir/src/vector.cpp.o CMakeFiles/dlib_.dir/src/svm_c_trainer.cpp.o CMakeFiles/dlib_.dir/src/svm_rank_trainer.cpp.o CMakeFiles/dlib_.dir/src/decision_functions.cpp.o CMakeFiles/dlib_.dir/src/other.cpp.o CMakeFiles/dlib_.dir/src/basic.cpp.o CMakeFiles/dlib_.dir/src/cca.cpp.o CMakeFiles/dlib_.dir/src/sequence_segmenter.cpp.o CMakeFiles/dlib_.dir/src/svm_struct.cpp.o CMakeFiles/dlib_.dir/src/image.cpp.o CMakeFiles/dlib_.dir/src/rectangles.cpp.o CMakeFiles/dlib_.dir/src/object_detection.cpp.o CMakeFiles/dlib_.dir/src/shape_predictor.cpp.o CMakeFiles/dlib_.dir/src/correlation_tracker.cpp.o CMakeFiles/dlib_.dir/src/gui.cpp.o /opt/local/lib/libboost_python-mt.dylib /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib dlib_build/libdlib.a /usr/lib/libpthread.dylib /opt/local/lib/libX11.dylib /opt/local/lib/libpng.dylib /usr/lib/libcblas.dylib /usr/lib/liblapack.dylib /usr/lib/libsqlite3.dylib

Now it looks OK. Let’s compile…

$ cmake --build . --config Release --target install
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   python
-- Found PythonLibs: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib (found suitable version "2.7.6", minimum required is "2.6")
-- Searching for BLAS and LAPACK
-- Found OpenBLAS library
-- Looking for sgetrf_single
-- Looking for sgetrf_single - not found
-- Found LAPACK library
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/suff/Downloads/dlib-18.16/python_examples/build
Scanning dependencies of target dlib
[  0%] Building CXX object dlib_build/CMakeFiles/dlib.dir/base64/base64_kernel_1.o
[  1%] Building CXX object dlib_build/CMakeFiles/dlib.dir/bigint/bigint_kernel_1.o
[....]
[ 84%] Built target dlib
Scanning dependencies of target dlib_
[ 85%] Building CXX object CMakeFiles/dlib_.dir/src/dlib.cpp.o
[....]
[100%] Building CXX object CMakeFiles/dlib_.dir/src/gui.cpp.o
Linking CXX shared library dlib.so
[100%] Built target dlib_
Install the project...
-- Install configuration: "Release"
-- Installing: ../python_examples/dlib.so
cp dlib.so ~/projects/dlib/venv/lib/python2.7/site-packages/
$ source ~/projects/dlib/venv/bin/activate
(venv) $ python 
Python 2.7.10 (default, May 23 2015, 22:32:15)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dlib
>>>

And we have a working dlib python module.

Mac OS X, Python, ports and dlib compilation

Mac OS X Lion vs JavaWS

After upgrading Java with Java for OS X 2013-0002 suddenly I lost ability to manage our Cisco ASA Firewall via ASDM.

Mrman told me, that I need to download and install two packages

JavaDeveloper for 10.7 [dropbox.com]

Apple Java Revert [dropbox.com]

After installation in fact I had working Java environment 1.6.0_35 but JavaWS still was a no go.

After googling a bit for the next error:

Can not find message file: No such file or directory

I’ve found #1 and #2.

the ultimate solution to the JavaWS canundrum is to remove  JavaWS minimum version string from the XProtect.meta.plist with

sudo /usr/libexec/PlistBuddy -c "Delete :JavaWebComponentVersionMinimum" /System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/XProtect.meta.plist
Mac OS X Lion vs JavaWS

OSX Updates vs Unsupported SSD vs Trim

About a year ago I’ve upgraded harddrive in my MacBook to SSD. This is one hell of an upgrade and you can belive it or not it’s one of the most important ones you should perform.

This upgrade is not so pain less as it looks like and after every system upgrade (Security Upgrades, Version updates, etc) you should check if Trim is enabled and active.

I use Trim Enabler. It has a easy to use GUI interface which tells you if it’s activeImage

But when you can’t or don’t want to install any software you can perform 4 commands that do the magic.

Before proceeding further down the spiral (;-)) we need to do backup of the file. It can be helpfull if something will go wrong:

sudo cp /System/Library/Extensions/IOAHCIFamily.kext/Contents/PlugIns/IOAHCIBlockStorage.kext/Contents/MacOS/IOAHCIBlockStorage /IOAHCIBlockStorage.original

Patch the file to enable TRIM support:

sudo perl -pi -e 's|(\x52\x6F\x74\x61\x74\x69\x6F\x6E\x61\x6C\x00).{9}(\x00\x51)|$1\x00\x00\x00\x00\x00\x00\x00\x00\x00$2|sg' /System/Library/Extensions/IOAHCIFamily.kext/Contents/PlugIns/IOAHCIBlockStorage.kext/Contents/MacOS/IOAHCIBlockStorage

Clear the kext caches:

sudo kextcache -system-prelinked-kernel

sudo kextcache -system-caches

And finally reboot your system
OSX Updates vs Unsupported SSD vs Trim