Creating a Nix Package from a Python Project

Packing a Python project into a NixOS package is pretty tricky. There are a few things that you need to know to get it to work. I’ll walk through how I packaged a Python project of mine.

My project is a collection of scripts that I needed to run on a recurring basis. This post does not go into scheduling the jobs on NixOS it only goes over packaging the Python project as a Nix Package. I wrote about scheduling jobs on NixOS using systemd in another post.

Python Project Setup

One thing I didn’t know when setting up this project was how difficult it was to be able to package a Python project. There are a ton of ways to do it.

I found a way to do it and just stuck with it. Once you get it packaged with the packaging of your choosing, the NixOS packaging is the same.

Python Project Basic Structure

First we’re going to start with our Python project structure. My project has the directory structure below. The scripts directory contains the files that we acually call and execute, while py-library contains all of our tools and code that the scripts use.

├── scripts/                      # Contains the scripts that we'll execute
├── py-library/                   # Library of common functions
├── steup.py                      # This declares how our libary is packaged
└── requirements.txt              # Helpful for pinning versions.

Defining our Python Project

Since there are so many ways to package a python script, I just went ahead and picked one. I just settled for the setup.py and setuptools method. Below my setup file definition.

setup.py

from setuptools import setup, find_packages
setup(
    name='py-library',
    version='0.0.1',
    install_requires=[
        'jsonpath-ng',
        'requests',        
        'gspread',
        'duckdb; python_version == "3.11"',
    ],
    packages = find_packages(),
    scripts=[
      'scripts/script.py',
    ],
)

Note that there is an explicit mention of the scripts that are callable. Do not skip this step. I spent a bunch of time debugging this issue becuase my script was not defined here.

Complete Nix Configuration

This is the conplete nix configuration for the python project. You can go ahead and replace the variable names for your project.

# Below, we can supply defaults for the function arguments to make the script
# runnable with `nix-build` without having to supply arguments manually.
# Also, this lets me build with Python 3.11 by default, but makes it easy
# to change the python version for customised builds (e.g. testing).
{ nixpkgs ? import <nixpkgs> {}, pythonPkgs ? nixpkgs.pkgs.python311Packages }:

let
  # This takes all Nix packages into this scope
  inherit (nixpkgs) pkgs;
  # This takes all Python packages from the selected version into this scope.
  inherit pythonPkgs;

  # Inject dependencies into the build function
  f = { buildPythonPackage, requests, gspread, duckdb, jsonpath-ng }:
    buildPythonPackage rec {
      pname = "py-library";
      version = "0.0.1";
      
      # Pull source from a Git server. Optionally select a specific `ref` (e.g. branch),
      # or `rev` revision hash.
      src = builtins.fetchGit {
        url = "https://{github_pat}@github.com/{user}/{py-library}.git";
        ref = "master";
        rev = "{your-commit_string}";
      };

      # Specify runtime dependencies for the package
      propagatedBuildInputs = [ 
        requests
        gspread
        duckdb
        jsonpath-ng
      ];


      # If no `checkPhase` is specified, `python setup.py test` is executed
      # by default as long as `doCheck` is true (the default).
      # I want to run my tests in a different way:
      #  checkPhase = ''
      #    python -m unittest tests/*.py
      #  '';
        doCheck = false;


      # Meta information for the package
      meta = {
        description = ''
          My Python project
        '';
      };
    };

  drv = pythonPkgs.callPackage f {};
in
  if pkgs.lib.inNixShell then drv.env else drv

Deploying and Starting

Once, you get a working Python configuration working for Nix, it’s very easy to use. One thing to note is that the Nix config file has to be updated when your code does, especially with something like dependencies.

Overall I have mixed feelings about it. Now that I have it running I’m going to contiue using it. However, I think my time could have been spent better had I just thrown everything into a container and deployed it like that. Hopefully this helps you if you need to write your own Nix package so you spend less time fumbling around than I did.