In our last article on my journey into Python, we talked at depth about the various operators that Python can use. If you managed to plough your way through that post, my hat off to you.
To refresh your memory on the first three articles, you can read them at the links shown below:
- Moving up the stack – time to learn Python – part 1
- Moving up the stack – time to learn Python – part 2
- Moving up the stack – time to learn Python – part 3
- Moving up the stack – time to learn Python – part 4
- Moving up the stack – time to learn Python – part 5
- Moving up the stack – time to learn Python – part 6
Today we will finalise our little meander around the building site looking at the brick work and foundations. This the final article on the basics of Python will discuss, Python Packages. Packages are the final building block, the roof of your new application house, of readable and functional code in python at scale.
Python Packages: The Building Blocks of Applications
Python is an incredibly versatile and powerful programming language, known for its simplicity, readability, and extensive standard library. One of the key factors contributing to Python’s widespread adoption and popularity is its rich ecosystem of packages. In our last article we discussed Modules, we described them as building blocks to applications, modules and packages are both used to organize and structure code, but they serve different purposes and have different characteristics.
As we already know a Python module is a single file containing Python code, typically with a .py extension. It can include functions, classes, variables, and other code elements. Modules help in organizing related code into separate files for better readability and maintainability. This modules are imported using the “import” keyword and can import the whole module or by using “from module import function” just import a specific function or object. Logically it will look similar to this image:
A Python package, on the other hand, is a directory or folder containing multiple Python modules. To be considered a package, the directory must contain an “__init__.py” file, which can be empty or contain initialization code for the package. Packages allow you to group related modules together, creating a hierarchical structure for your code. You can import modules or specific objects from a package using the import statement with the package name followed by the module name, separated by a dot.
As we have just seen Python packages are modular, reusable components that extend the language’s functionality, packages make it easier for developers to tackle complex problems and streamline their development process.
Like modules, packages, are often developed and maintained by the open-source community, cater to a wide range of applications, from web development and data analysis to artificial intelligence and machine learning. By utilizing Python packages, developers can save time and effort, as they don’t need to reinvent the wheel for every new project. Instead, they can focus on building upon existing solutions and refining their code.
Python’s official package repository, the Python Package Index (PyPI), hosts a vast collection of packages that can be easily installed using package managers like pip or conda. With thousands of packages available, developers can find solutions for nearly any task or domain-specific challenge.
In this article, we will explore the world of Python packages, discussing their importance, how to use them effectively, and showcasing some of the most popular and widely-used packages in various fields. By the end of this piece, you will have a better understanding of Python packages and their role in accelerating development and fostering innovation in the Python ecosystem.
Writing Packages
We have already mentioned that Packages are namespaces containing multiple modules, that said they can also contain multiple packages. Effectively they are just directories, however they must meet certain requirements.
Package Structure
A Python package is essentially a directory containing multiple related Python modules. For a Python package to be valid the package structure must consists of the following components:
__init__.py file (Required): This file is required for a directory to be considered a Python package. It can be empty or contain initialization code for the package. The __init__.py file is executed when the package is imported, allowing you to define package-level variables, functions, or classes.
Python modules (Required): Modules are single files containing Python code, such as functions, classes, variables, and executable statements. They can be written in Python or C (loaded dynamically at runtime). A package can contain multiple modules, each serving a specific purpose within the package.
Sub-packages (Optional): A package can also contain sub-packages, which are directories with their own __init__.py files and modules. This hierarchical structure allows for better organization of related code.
Documentation (Optional but recommended): Including a README file in your package is essential for providing an overview of the package’s purpose, usage instructions, and any other relevant information.
License (Optional but recommended): A license file (typically LICENSE) should be included in your package to specify the terms under which the package can be used, modified, and distributed.
Setup files (Optional): These files are necessary for building and distributing your package. They include setup.py, which contains configuration information for building and installing the package, and requirements.txt, which lists the dependencies required for your package to function properly.
Our first Python package
Each package in Python is a directory which MUST contain a special file called “__init__.py”. This file, which can be empty, indicates that the directory it’s in is a Python package. That way it can be imported the same way as a module.
Creating a simple Python package as we have mentioned involves organizing related Python modules into a directory structure. So let’s start the process and create our first package we will call it “greetings” and this package will contain two modules: “hello” and “goodbye”. Each module will have a function to print a greeting message. So without further ado, lets get started.
To start create a directory named greetings:
greetings/
Inside the greetings directory, create an empty __init__.py file. This file is required for Python to treat the directory as a package:
greetings/ __init__.py
Next, we create our two Python modules, create two files called “hello.py” and “goodbye.py” inside the greetings directory. To our “hello.py” file add a function called say_hello()
def say_hello():
print("Hello!")
Next in the “goodbye.py file create a function called say_goodbye.py
def say_goodbye():
print("Goodbye!")
The final package structure should look like this:
greetings/ __init__.py hello.py goodbye.py
Now that we have created our greetings package, we can import it and use its functions in another Python script. Let’s see this work, firstly we need to create a script named main.py outside the greetings package add the following code to it:
from greetings.hello import say_hello from greetings.goodbye import say_goodbye say_hello() say_goodbye()
When you save and run main.py, it will output:
This basic example demonstrates how to create a simple Python package by organizing related modules into a directory structure and using an __init__.py file to indicate that it’s a package.
Creating a more complex package
It is safe to say that the package we created above is next to useless in the real world, so let’s have a look at creating a more complex package. this time we will create a package that preforms basic arithmetic operations like addition, subtraction, multiplication, and division. The package will contain four modules, each responsible for a specific operation, for simplicity we will call it “math_operations”, just because I like my names to be descriptive no opaque.
As before lets start with our directory called math_operations:
mkdir math_operations/
and inside the “math_operations” directory, create our empty “__init__.py” file.
math_operations/ __init__.py
Now its time to get funcky, create four Python modules named addition.py, subtraction.py, multiplication.py, and division.py inside the math_operations directory. Add a function in each module to perform the corresponding arithmetic operation:
addition.py:
def add(a, b): return a + b
subtraction.py:
def subtract(a, b): return a - b
multiplication.py:
def multiply(a, b): return a * b
division.py:
def divide(a, b):
if b == 0:
raise ValueError("Division by zero is not allowed.")
return a / b
Our final package structure should look like this:
math_operations/ __init__.py addition.py subtraction.py multiplication.py division.py
Now that we have created our “math_operations” package, let’s import our 
Time for something potentially useful
So far, the packages we have written have no real-world use. So let’s write a potentially useful package. this package will create a random password of between 8 and 16 characters, which must contain, contain uppercase and lowercase letters, numbers, and special characters:
As normal, we must create a directory, so let’s create ours let’s call it something understandable so “password” should suffice:
mkdir password/
Once inside the “password” directory, create an empty “__init__.py” file. This file is required for Python to treat the directory as a package:
password/ __init__.py
Next create a file called “generate.py” and add the following, we have called it generate to prevent a potential conflict with our function “generate_password”, contained in the file:
import random
import string
def generate_password(length):
if length < 8 or length > 16:
raise ValueError("Password length must be between 8 and 16 characters.")
characters = string.ascii_letters + string.digits + string.punctuation
password = ''.join(random.choice(characters) for _ in range(length))
# Ensure the password contains at least one uppercase letter, lowercase letter, digit, and special character.
if (not any(c.isupper() for c in password) or
not any(c.islower() for c in password) or
not any(c.isdigit() for c in password) or
not any(c in string.punctuation for c in password)):
return generate_password(length)
return password
The final package structure should look like this:
password/ __init__.py generate.py
OK now that is created let’s see it in action, Add the following to our main.py file in the folder level above:
import random
import password
def main():
password_length = random.randint(8, 16)
password =password.generate_password(password_length)
print(f"Generated password: {password}")
if __name__ == "__main__":
main()
We can now run the main.py script, which will now generate a random password using the “password package”.This script defines the “generate_password” function that takes a length parameter. It checks if the length is between 8 and 16 characters and raises a “ValueError” if it’s not. Then, it generates a random password using the string.ascii_letters, string.digits, and string.punctuation character sets.
Our function also checks if the generated password contains at least one uppercase letter, lowercase letter, digit, and special character. If it doesn’t meet these criteria, it recursively calls itself to generate a new password.
The main function generates a random password length between 8 and 16 characters and calls the “generate_password” function to create the password. Finally, it prints the generated password. But you know what they say the proof of the pudding is in the eating so here is the proof it works:
Diving deeper
So far you will have noticed that we have only created two artefacts in our package, the init file__init__.py and some simple level Python modules. So far we have only created empty “__init__.py” files; so what are some of the common things you would place in the initialization file in a Python package:
- Package-level variables: You can define variables that are accessible across all modules within the package. For example, constants or configuration settings that are used in multiple modules.
- Package initialization code: This code is executed when the package is imported for the first time. It can be used to set up resources, configure logging, or perform other tasks that are needed to be done once at the beginning of the package’s lifetime.
- Importing submodules or objects: You can import submodules or specific objects from them in the __init__.py file to make them directly accessible when importing the package.
Here’s an example of how you could write an “__init__.py” file to include these elements in a package:
# Package-level variables
CONSTANT_1 = "value1"
CONSTANT_2 = "value2"
# Package initialization code
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
logger.info(f"Initializing package {__name__}")
# Importing submodules or objects
from .submodule1 import function1, function2
from .submodule2 import Class1, Class2
In this example file, we defined two package-level variables CONSTANT_1 and CONSTANT_2, set up a logger and informational log message when the package is initialized. Finally, we have imported functions and classes from two submodules, making them directly accessible when the package is imported. As we continue down our journey, we will actually see this concept in action, but at the moment it is beyond the scope of this particular article.
Let’s have a look see in to some of the other files that a package can potentially include starting with the setup file.
Python Setup file
In a Python package the setup file is used to define the package’s metadata, dependencies, and installation instructions. The most common setup file used in Python packages is “setup.py”. The reason that the setup file is optional is that it contains the information required to build, and more importantly distribute, and install the package using the Python Package Index (PyPI) or other package management systems. The main purposes of a setup file are:
- Package metadata: Define package name, version, author, description, and other relevant information.
- Dependencies: List the package’s dependencies, so they are automatically installed when the package is installed.
- Installation instructions: Provide instructions on how to build and install the package.
So what would a sample setup.py file for a Python package named mypackage look like:
from setuptools import setup, find_packages
setup(
name="mypackage",
version="0.1.0",
author="Your Name",
author_email="your.email@example.com",
description="A brief description of your package",
long_description=open("README.md").read(),
long_description_content_type="text/markdown",
url="https://github.com/yourusername/mypackage",
packages=find_packages(),
classifiers=[
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
],
install_requires=[
"requests", # Example dependency
],
python_requires=">=3.6",
)
In our example script, we use the “setuptools” package to define the package metadata, dependencies, and other information; “setuptool” is not a built-in package, but it is commonly installed alongside many Python distributions and package management systems; if it is not installed use pip to install it.
pip install setuptools
The “find_packages()” function automatically discovers and includes all the packages in the package directory. This file would be placed in the root of your package directory.
Now when we need to install this package into a different project, we would simply install the package by navigating to the package directory in our terminal or command prompt and running the following command:
python setup.py install
Another benefit is that when we want to show our work to the world and allow everyone and their got to use it you can distribute our package on PyPI, we would need to create a source distribution and a wheel distribution using the following commands:
python setup.py sdist python setup.py bdist_wheel
and finally, we would use “twine” to upload the distributions to PyPI:
twine upload dist/*
Python package documentation
I am a firm believer in documentation, that said we have not actually created any in our sample packages, this is because they are very simple. However for completens we have provided an example of a Python package documentation README file for a package called mypackage:
## MyPackage
# MyPackage is a simple yet powerful Python library for performing various operations on strings. It provides functions to manipulate, analyze, and transform text data efficiently.
## Features
# - Reverse a string
# - Count occurrences of a substring
# - Replace all occurrences of a substring
# - Convert a string to uppercase or lowercase
# - Remove leading and trailing whitespace
## Installation
# MyPackage is installed using pip:
# pip install mypackage
## Usage
# Import the desired functions from the mypackage module and use them in your Python script:
# from mypackage import reverse_string, count_substring, replace_substring, to_upper, to_lower, strip_whitespace
# Reverse a string
# reversed_string = reverse_string("Hello, World!")
# Count occurrences of a substring
# count = count_substring("abracadabra", "abra")
# Replace all occurrences of a substring
# replaced_string = replace_substring("Hello, World!", "World", "Universe")
# Convert a string to uppercase or lowercase
# upper_string = to_upper("hello")
# lower_string = to_lower("HELLO")
# Remove leading and trailing whitespace
# stripped_string = strip_whitespace(" Hello, World! ")
## Documentation
# For more detailed documentation and examples, please visit our official documentation.
## Contributing
# We welcome contributions to MyPackage! Please follow these steps to contribute:
# - Fork the repository on GitHub.
# - Create a new branch for your feature or bugfix.
# - Implement your changes and write tests if applicable.
# - Submit a pull request with a clear description of your changes.
# - Please make sure to follow our code of conduct when contributing.
## License
# MyPackage is released under the MIT License.
This example README file can provide an excellent template for your documentation, it provides all the key ingredients needed to provide good documentation, starting with an overview of what the package is supposed to do, its features, installation instructions, basic usage examples, links to detailed documentation usually on your GitHub repository, and guidelines on how to contribute to the project. You can customize this template based on the specific requirements and features of any package you later write.
Summary
This article has proven to be a marathon, but at a sprint pace, we have cover a vast amount of information, but effectually only just skimmed the surface, as we move on in this series, the knowledge we have learnt in this and the previous six articles will stand us in good stead.
