This post continues on from part 2. If you’ve gone through the prior posts in this series, you have a fully functioning package. Now we’ll distribute that package.

Package It

To create a release, your source code needs to be packaged into a single archive file. There are two concepts in the Python ecosystem to be aware of here: source distributions and wheel distributions.

Source Distribution

Making a source distribution will create a dist subdirectory in your project and will wrap up all of your project’s source code files into a distribution file. That distribution file is a compressed archive file, which defaults to .tar.gz files on POSIX systems and a standard .zip file on Windows systems. The term “source distribution” refers to an unbuilt distribution. In the Python ecosystem, this is distinct from a “build distribution.” A source distribution requires a separate build step when installed by a tool like pip. You create the the distribution file by running sdist as part of the setup.py file. You can also create a bdist_wininst distribution file, which creates a Windows installable file. There are a few different file formats that Python distributions can be created for and, at some point, you might want to investigate those although most times you’ll find that what I describe here is more than enough. Here’s an example of how to generate these files:

python setup.py sdist bdist_wininst

Wheel Distribution

You should also create a “wheel” for your project. A wheel is a built package that can be installed without needing to go through any separate build process. Installing wheels is substantially faster for the end user than installing from a source distribution. But there are some complicating elements. Let’s break this down a bit.

• If your project is what is called “pure python” (meaning it contains no compiled extensions) and your project natively supports both Python 2 and 3, then you can create what’s called a Universal Wheel.

• If your project is pure python but does not natively support both Python 2 and 3, then you’ll be creating a non-universal wheel, which is called a Pure Python Wheel.

• And, finally, if your project contains any compiled extensions, then you’ll have to create what’s called a Platform Wheel. Platform Wheels are wheels that are specific to a certain platform like Linux, MacOS, or Windows. The reason, of course, is because the compiled extensions are generally compiled for a specific platform.

Let’s just keep this simple for ourselves. For pure python, you do this:

python setup.py bdist_wheel

But if you did want to go universal, you just run the same command with a qualifier:

python setup.py bdist_wheel --universal

This will create a file in your dist directory with an extension of “.whl”.

Wheel? Source? What?!?

All of this can be a bit complicated. Just know that pip can install from either source distributions (sdist) or build distributions (wheels). If both are present on PyPI, pip will put precedence on a compatible wheel. This is because wheels are a pre-built distribution format that provides faster installation compared to source distributions (sdist). If pip doesn’t find a wheel to install, then it will grab the source distribution, locally build a wheel, and then cache that built wheel for future installs. So you can see that even with a source distribution, Python will not rely on rebuilding the source distribution in the future if that’s at all possible. Incidentally, the concept of the wheel came about as a result of the desire for a binary package format. Wheel is intended to replace the Egg format. You can get a comparison of some aspects of wheel vs egg if you’re really curious. Just know that Egg is going away.

Register It

There is a central index of all publically available Python projects. This is the the Python Package Index (PyPI), which I mentioned in the first post. This is where you will want to release your package distribution. Projects are published on PyPI in the format of:

http://pypi.python.org/pypi/<projectname>

As you would imagine, your project does have to have a name that’s not already taken on PyPI. Assuming you have a unique name, you can register that name which in effect means registering your package. But here’s where things get interesting. A lot of the tutorials out there will show you something like this:

python setup.py register

Then they will have you do something like this:

python setup.py upload

These methods of registering and uploading a package are now very strongly discouraged by the Python team, largely for security reasons. What you want to do is use Twine, which I mentioned installing in the first post. If you didn’t do that, you can just do this:

pip install twine

There’s a lot of information out there about this, a lot of it confusingly stated. So let’s just take this step-by-step. I will say at this point that you’re going to want to create the accounts on both TestPyPI and PyPI (as I mentioned in the first post) and, optionally, create the .pypirc file.

Deploy to Test Server

Generally what people will do is first release their distribution to PyPI Test. The process is that releasing to PyPI Test allows you to check your packaging, distribution, and release process before you upload a specific version to PyPI and make that version official. So here’s what you can do with proverb:

twine register -r test dist/*.whl
twine upload -r test dist/*

Notice that “-r test” in there. This is is what is telling Twine which server to deploy to. Also note that on Windows you might be better off typing in the file name. Final note is that you’ll only run the register once. When your package name has been registered, you certainly don’t need to do it again. Success looks something like this:

Registering package to https://testpypi.python.org/pypi
Registering proverb-0.1-py3-none-any.whl

Uploading distributions to https://testpypi.python.org/pypi
Uploading proverb-0.1-py3-none-any.whl
Uploading proverb-0.1.tar.gz
Uploading proverb-0.1.win-amd64.exe

Note that if you did the Windows distribution like I did, you’ll likely see this error:

invalid distribution file for url: https://testpypi.python.org/pypi

That’s okay. I put this in place to show that this can happen but it does not impact your deployment. To see if this all worked, check if the package you wrote was deployed by going to the URL: https://testpypi.python.org/pypi/proverb. As I mentioned in the first post, you’ll clearly have to have thought up your own project name in order to be able to deploy. Now let’s try to install proverb (or your own package) from the test server. Try this:

pip install -i https://testpypi.python.org/pypi proverb

Make sure it works as you did before from a Python prompt:

>>> import proverb
>>> print(proverb.saying())

Had any part of this process not worked, you would want to correct that before deploying to PyPI.

Deploy to Production Server

Now finally we can upload the package to PyPI and test its installation one last time.

twine register -r pypi dist/*.whl
twine upload -r pypi dist/*

Notice the “-r pypi” here to specify the non-test server. Also note that as with the test server, you only have to run the register command once. Check if the package you wrote was deployed by going to the URL: https://pypi.python.org/pypi/proverb. Now let’s install it:

pip install -U proverb

Once again, try out the logic at the Python prompt. You should see that it works just fine.

Build / Deploy Steps

Just to make sure the basic steps are clear, given that I covered a lot, what you’ll do most often for the package and deploy bits are these:

• python setup.py sdist bdist_wheel
• twine upload -r test dist/*
• twine upload -r pypi dist/*

And That’s It!

So if you made it through these posts, you’ve seen how to create, package, and deploy your own Python solutions. Along the way, I tried to clear up some possible confusions or ambiguities around the Python ecosystem, which is continuing to evolve even as I write this. Hopefully these posts helped you a bit on your way to becoming a Python solution developer.