Tao
Posts Tags Categories Series About Contact
Tao

Step-by-Step Guide to Packaging Python Projects for PyPI

Mttao avatarMttao  included in  category Python
     1282 words  7 minutes 

This blog post will guide you step-by-step through the entire process from project preparation to successful publication on PyPI, including modern packaging tools and best practices such as using pyproject.toml, the build tool, and testing safely with TestPyPI.

Core tools: pyproject.toml, setuptools, build, twine, uv (or pip)

A clear project structure is the foundation for successful packaging. The recommended src layout effectively isolates your actual package code from other project files (such as tests, documentation, configuration files).

Assuming your package is called mytoolpack:

text

mytoolpack_project/
├── src/
│   └── mytoolpack/          <-- This is your actual package code
│       ├── __init__.py      # Package initialization file, makes the directory a package
│       └── module.py        # Your core code, e.g., containing a cool_function
├── tests/                   <-- Test code
│   └── test_module.py
├── .gitignore
├── LICENSE                  <-- Choose an open source license!
├── README.md                <-- Detailed project description
└── pyproject.toml           <-- Core file for packaging and project configuration

The pyproject.toml file is the cornerstone of modern Python packaging (PEP 517/518/621). It defines the project’s build system, metadata (such as name, version, author), and dependencies.

Here’s an example of a pyproject.toml:

toml

# mytoolpack_project/pyproject.toml

[build-system]
requires = ["setuptools>=61.0", "wheel"] # Declare build dependencies
build-backend = "setuptools.build_meta"
backend-path = ["."] # Optional, ensures setuptools can find everything

[project]
name = "mytoolpack" # Name on PyPI, also the name users import
version = "0.1.0"   # Follow semantic versioning (semver.org)
authors = [
    { name="Your Name", email="[email protected]" },
]
description = "An example toolkit that does cool things."
readme = "README.md" # Points to your README file
requires-python = ">=3.8" # Python versions supported by your project
license = { file = "LICENSE" } # Points to your license file, or use {text = "MIT"}, etc.
classifiers = [ # PyPI classifiers to help users find your package
    "Development Status :: 3 - Alpha",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License", # Corresponds to your chosen license
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.8",
    "Programming Language :: Python :: 3.9",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Topic :: Software Development :: Libraries :: Python Modules",
    "Topic :: Utilities",
]
keywords = ["tool", "utility", "example", "packaging"] # Keywords for your package

# Runtime dependencies for your project
dependencies = [
    # "requests>=2.20.0", # For example: if your package needs requests
    # "numpy",
]

[project.urls] # Optional links
"Homepage" = "https://github.com/yourusername/mytoolpack_project"
"Bug Tracker" = "https://github.com/yourusername/mytoolpack_project/issues"
"Documentation" = "https://readthedocs.org/projects/mytoolpack/" # If you have documentation

# --- Setuptools specific configuration (for src-layout) ---
[tool.setuptools.packages.find]
where = ["src"]  # Tells setuptools to look for packages in the src directory
# include = ["mytoolpack*"] # More precise specification

Key points:

  • [build-system]: Specifies the build tools required to build your project.
  • [project]: Contains all important metadata about your project.
    • name: Unique name on PyPI.
    • version: Package version number, needs to be updated with each new release.
    • dependencies: Other packages your package depends on at runtime.
  • [tool.setuptools.packages.find]: Helps setuptools find your package code, especially when using the src layout.

In the src/mytoolpack/__init__.py file, you can control which features are available when users import your package:

python

# src/mytoolpack/__init__.py

from .module import cool_function  # Import cool_function from module.py in the same directory

__version__ = "0.1.0"  # Keep consistent with the version in pyproject.toml
__all__ = ['cool_function']  # Define what is imported with `from mytoolpack import *`

Assuming src/mytoolpack/module.py contains:

python

# src/mytoolpack/module.py

def cool_function():
    print("This is a cool feature!")
    return True

Now that your project structure and configuration files are ready, it’s time to build your distribution package. These are the files that will ultimately be uploaded to PyPI.

  1. Install the build tool build: If you haven’t installed the build package yet, you can use uv (or pip) to install it:

    bash

    uv python install build
# or pip install build

  2. Run the build command: In your project root directory (mytoolpack_project/), run:

    bash

    python -m build

    This command will call the build backend specified in your pyproject.toml (here, setuptools) to create distribution files. After success, you’ll see a new dist/ directory in your project root containing two files:

    • A .tar.gz file (source distribution, sdist)
    • A .whl file (wheel package, pre-compiled binary distribution)

Before directly uploading your package to the real PyPI, it’s strongly recommended to test it first on TestPyPI (test.pypi.org). TestPyPI is a separate test instance of PyPI where you can freely upload and delete packages without cluttering the official repository.

  1. Register a TestPyPI account: Go to test.pypi.org and register an account.

  2. Install twine: twine is the tool for securely uploading packages to PyPI (and TestPyPI).

    bash

    uv python install twine
# or pip install twine

  3. Upload to TestPyPI: Use twine to upload the files in your dist/ directory to TestPyPI. You’ll need to use your previously registered TestPyPI username and password (or create an API Token).

    bash

    twine upload --repository testpypi dist/*

    Follow the prompts to enter your TestPyPI API token

  4. Install and test from TestPyPI: After a successful upload, try installing your package from TestPyPI in a new virtual environment to verify that everything works as expected:

    bash

    # Create and activate a new virtual environment (using uv or venv)
# uv venv .test_env
# source .test_env/bin/activate

uv python install -i https://test.pypi.org/simple/ mytoolpack
# or pip install -i https://test.pypi.org/simple/ mytoolpack

# Now test your package
# python -c "import mytoolpack; mytoolpack.cool_function()"

    If the installation and basic functionality work correctly, your package is likely ready for the real PyPI!

Once you’ve confirmed everything is working smoothly on TestPyPI, you can publish your package to the official PyPI (pypi.org).

  1. Register a PyPI account: Go to pypi.org and register an account (if you don’t have one yet). This is separate from your TestPyPI account.

  2. Create an API Token (recommended): For security reasons, don’t use your PyPI username and password directly for uploading. Create an API Token in your PyPI account settings. When creating it, you can limit its permission scope (such as only allowing uploads for specific projects). Copy and save this Token immediately after creation, as it’s only shown once.

  3. Upload to PyPI:

    bash

    twine upload dist/*

    When twine prompts for a username, enter __token__. When it prompts for a password, paste the PyPI API Token you just created.

    If your ~/.pypirc file is correctly configured with tokens for PyPI and TestPyPI, twine might select automatically. But typically, manually specifying or using the __token__ method is clearer.

Congratulations! Your Python package is now published on PyPI, and Python users around the world can install and use it with pip install mytoolpack!

  • Choose a unique package name: Before selecting a name in pyproject.toml, search on PyPI.org to ensure your chosen name isn’t already taken.
  • Excellent README.md: This is the first window for users to understand your project, make sure it’s clear and comprehensive.
  • Include a LICENSE file: Choose an appropriate open source license, and declare it in your pyproject.toml.
  • Version control: Follow semantic versioning (e.g., 1.0.1, 1.1.0, 2.0.0), and update both pyproject.toml (and __version__ in __init__.py) with each new release.
  • Continuous Integration/Continuous Deployment (CI/CD): For more mature projects, consider setting up CI/CD pipelines (like GitHub Actions, GitLab CI) to automate testing, building, and release processes.

Packaging and publishing a Python project to PyPI might seem complex, but once you master the modern tools and processes, it becomes very straightforward. By using pyproject.toml for clear configuration, leveraging the build tool, testing safely with TestPyPI, and uploading via twine, your code can be used by others through PyPI.

Updated on 2025-05-15
 PyPI, Packaging, Pyproject.toml
 | Home