Some time ago I wrote about GitLab Continuous Integration and explained how code changes can be catched up to create or update software packages on remote systems. In order to communicate with the development systems, GitLab utilizes agents that are availble for plenty of platforms – including Microsoft Windows, we will focus on in this article.
Java, PHP, Go, Python or even LaTeX, no limit here! In this blog post we review a few examples for the Python programming language. In GitLab, the different tests are called jobs. These jobs are executed by a gitlab-runner, that can be installed on the same server as you main GitLab. Runners are external processes used to run CI jobs. They are deployed by the administrator and registered to the GitLab instance. Shared runners are available for all projects. Specific runners are enabled for a list of projects.
In the following example, Python applications (*.py) are converted automatically into executables (*.exe) after code changes. But – why? I’m using this concept for generic Icinga monitoring plugins. Of course, Python scripts can also be executed on Windows after installing the runtime environment. Think about deploying this on a big amount of systems – it’s a huge administration effort. A single executable can be distributed easier and also offers less security risks than a complete runtime environment that consists of multiple sub-programs.
There are multiple applications for converting Python scripts into executables – I’m using pyInstaller. This tool combines all required Python modules and also the Microsoft Visual C++ runtime environment necessary for Windows into a single file. pyInstaller is also availble for other platforms and currently supports Python 2.7 and 3.
My goal was to achieve that all code changes in the master branch forces the re-creation of the appropriate executables.
First of all, it is necessary to enable CI for a GitLab project. Proceed with the following steps:
- Select the project, click Settings and Project Settings.
- Click Builds and Save Changes.
- Move to the CI Settings pane to set advanced parameters such as timeouts and automatic builds.
- Click Save Changes.
- Select Runners, note/copy the CI-URL and the CI-Token – we will need this information later for registrating the runners.
To implement agent communication and automatic builds, additional preparations need to be fulfilled. The most important is to create a dedicated user account or service user. Depending on your infrastructure this step might vary (e.g. local account, Active Directory). It is important that this user has the permission to run services. Unfortunately, client systems like Windows 7 don’t offer a setting like this – in this case the user needs to have local administrative permissions. 🙁
Beyond that, the following software packages are required:
- Python: [click me!]
- Git for Windows: [click me!]
- PyInstaller: [click me!]
- GitLab Runner: [32bit] or [64bit]
The particular install process are self-explanatory. When installing Python, it is necessary to check “Add python.exe to path” – otherwise Python programs are not executable from the command line or PowerShell. For the Git installation a similar option “Use Git from the Windows Command Prompt” needs to be set, too.
PyInstaller is installed using a Python script – make sure to start it inside a command line with administrator privileges:
Python and PyInstaller
Let’s check whether Python and PyInstaller work as expected. The following source code should be catched up by PyInstaller without any issues:
The script prints a sentence and waits for an input before it closes itself. The following command creates an executable including the script, runtime environment and required Python modules:
The switch -F integrates the Python and Microsoft Visual C++ runtime environments, -y overwrites pre-existing files. If the utility finishes without any errors, new folders build and dist are created in the current directory. There is a folder with the same name like the script including the binary inside the dist folder. You can start it like this:
By the way – for Python scripts that need to be executed under Windows, it is necessary to load the Python module os. Otherwise, the following error message is created when the executable is started:
Powershell and CI-Runner
For automating the packaging process, I’m using a PowerShell script. To ensure that custom and thus unsigned PowerShell scripts can be executed, I had to alter the execution policy:
The Microsoft Knowledge base offers good explanations on the particular settings.
The GitLab CI runner is moved into a directory on the hard drive, e.g. c:gitlab-ci-runner. To make manual debugging easier, it is also advisable to shorten the long file name. Afterwards the runner is registered to the GitLab system – for this task we need to enter the project token. After this, the runner is installed and started as Windows service:
To make GitLab push the code changes to the runners, we need to create the .gitlab-ci.yml file. As I already mentioned in the last post, this file controls the behavior of runners. The following script automatically executes a PowerShell script after every commit to the master branch.
Gitlab Runner Python Download
The PowerShell script looks like this:
The script is kinda simple, it moves into every sub-folder and converts a Python script with the same name.
Gitlab Runner Python Tutorial
Conclusion
There are still a couple of things in the scenario that could be extended – for example:
- Compiling only on particular nodes (e.g. depending on additional dependencies or hardware ressources)
- Functional tests after compiling – e.g. using unittest
- Automatic upload of converted files using artefacts
Gitlab Runner Python Free
I’m really interested in deep-diving into artefacts, so stay tuned. 🙂