bii internals: Compiling your python application with Cython

We are starting a series of blog posts about how we package biicode. We are starting from grounds up, we will discuss compilation, packaging and last but not least installers creation.

Cython

Distributing your python app to clients it’s a common task that can become hard when “obfuscated code” come as requirement. Common answers in forums are on the lines of “Python is not the language you’re looking for” or “What are you trying to hide?” which are a way of trying to avoid the problem, not to solve it. This task can be easily achieved with Cython.

Cython can be used to compile your python app with biicode

Cython-logo” by The logo may be obtained from Cython.. Licensed under Fair use via Wikipedia.

Cython is a static compiler for both Python and their own extended Cython programming language that allows us creating C extensions for Python. I generates C code taking your source python code as input and then compiles it into python extensions with all major C/C++ compilers. We are using clang, gcc and MS Visual 9.0 for our different clients.

Compiling your app with cython is an easy task, you just need to know what do you want to include and what do you want to exclude from your build.

In biicode, for example, we just compile code under biicode.client and biicode.common modules, we want to exclude tests and also we want to keep __init__.py files as python files because we later package the app with PyInstaller. We need to define following variables:

Then you can traverse your source tree, for every file cython will analyze it’s dependencies tree and recompile only if needed.

After this function executes you will have a source tree equivalent to your python one but in C inside your build_dir/src. Then c files that do not longer match to an existing py file need to be deleted (that’s an easy one to automate as you have the list of c_files). You can avoid that last step if you don’t need incremental builds and you always clean your build dir before recompiling.

Now that we have the list of c files we want to compile them into python extensions but before we may want to ignore python asserts and remove debug flags:

Now you are ready to compile your native extensions!

Now you have your python extensions in build_dir/lib.ARCHITECTURE If you wish to later package them with PyInstaller you now need to copy all your __init__.py to their correspondant folders in build_dir/src but we will cover it in a future post.
If you liked this post please comment below. If you want to try biicode just click on the sidebar button and if you have any doubts check our docs, forum, Stackoverflow tag and Github repos.

Stay tuned


Related Posts