Moving up the stack – time to learn Python – part 6

In our last article on my journey into Python, we talked at depth about the various operators that Python can use.  If you managed to plough your way through that post, my hat off to you.

To refresh your memory on the first three articles, you can read them at the links shown below:

• Moving up the stack – time to learn Python – part 1
• Moving up the stack – time to learn Python – part 2
• Moving up the stack – time to learn Python – part 3
• Moving up the stack – time to learn Python – part 4
• Moving up the stack – time to learn Python – part 5

Today we will continue meandering around the building site looking at the brick work and foundations. This article will discuss, Python Modules. These are a major building block of readable and functional code in python.

Python Modules: The Building Blocks of Applications

In our last article we discussed functions, and was introduced to the “def” keyword. This keyword defines something and is commonality between a function and a module.  However, whereas functions are used to break down complex tasks in to smaller and more manageable pieces of code and are designed to be reusable across different parts of a script or across different scripts. A module is used to organise code into logical units and provide reusable functionality across different scripts or across different projects. Say What now! That doesn’t appear to say anything different, but it does, a function is a lower-level construct.  The diagram below highlights where a module will fit in to a python project, over where functions do. As you can see a module is a fully features mini-program, whereas the functionally similar function is just a code snippet.

At the base level a module is a file containing Python definitions and statements. These can be used to perform specific tasks or provide certain functionality within your code.

One of the key reasons to use a module in Python is that simplify your code, by logically breaking up the program into functional segments; this is especially useful when you are creating larger and more complex applications. Your Modules are then easily imported and called from other modules or scripts. Importation of a module is undertaken using the “import” statement, followed by the name of the module you want to use.

For simplicity we will only cover the most common format of a module which is the interpreted form and denoted with the “*.py” suffix. However, another popular option is the “*.pyc” file, which is a compiled version of the “*.py” file; these can be loaded more quickly by the interpreter. Other file formats include “*.pyd” files (used for Windows DLLs), “*.so” files (used for Unix shared libraries), and “*.egg” files (used for Python packages).

What are some example of modules

There are three main type of module:

• Builtin:- those modules that are installed as part of the core python package and can be utilised directly without having to install any extra code. Random, os, math, and sys are some common examples.
• Public, Common or Third-party: These are pseudo-official modules that freely available and are commonly used, however as they are not a part of the core installation package they need to be installed into the development environment before use. Numpy, Requests, and Pandas are examples of modules that are commonly utilised in python projects but are not part of the core package.

And finally

• Private or Self-created:- these are modules written by yourself for a particular purpose.

Overall, understanding how to use and call Python modules is an essential skill for any Python developer. By breaking your code up into smaller, more manageable pieces, you can make your code more modular, easier to maintain, and more efficient overall. So whether you’re building a small script or a large-scale application, taking advantage of Python modules can help you get the job done more effectively and efficiently.

Your First Module

Now that we have got the how’s and the why’s of module utilisation lets start to look at how we will create them. As we have already alluded to a module is a file consisting of Python code. It can define functions, classes, and variables, and can also include runnable code. Any Python file can be referenced as a module. A file containing Python code, for example: test.py, is called a module, and its name would be “test”, calling this code is a simple as issuing an “import test” call in your script, one thing that must be remembered is that the path to the module must be in the path for it to be found 😉.

We have already used a module during this series, remember this code? Notice the first line.  That right there is your module importation (import turtle).

import turtle

t = turtle.Turtle()
t.speed(0)

for i in range(100):
    t.forward(i *10)
    t.right(144)
turtle.done()

Usually we would introduce the concept of project folder structure here, but currently that would over complicate this article. So until we understand the process of module control we will follow KISS (keep things stupidly simple). Therefore our two files are in the same root folder.

Our first module

The first is our module, it is rather pithily called “mymodule.py”.  our first line defines a function “def hello(name)”, this function simply prints to screen the entered name.  secondly we create a class, this class defines “self, name and age”, this class is used to populate the second function “greet()” which takes the input “self”. And then prints to screen a string of text and the class values, finally we create two variables, “greeting” which has a string value and “PI” which has an integer value; in our case the value of pi is to 5 decimal places

def hello(name):
    print(f"Hello, {name}!")

class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

    def greet(self):
        print(f"My name is {self.name} and I am {self.age} years old.")

greeting = "Welcome to the Amazic modules article!"

PI = 3.14159

To use your module in another script or program, we simply import it using the “import” statement. For example, you can create a program called “main.py” that imports and uses your module:

import mymodule

mymodule.hello("Catherine")

print(mymodule.greeting)

tom = mymodule.Person("Tom", 58)
tom.greet()

print(f"Guess What, Pi to 5 places is ",mymodule.PI)

looking at the above script you can see that we have used the “import” keyword.  However to use each individual part of the module you will notice that we call the name of the desired module, followed by the action. For example you can see “mymodule.hello (“Catherine”)”, here as with a normal function, we are passing the input of “Catherine” to the modules function “hello” rather than passing the input “Catherine” directly to a function in that “main.py” script.

The output of running main.py will be:

The process regarding modules is very powerful, it is similar to a level of abstraction, we prefix, module entities with the module name. and import the module into the main script.  It is the same process when we import built-in modules.  There one more thing before we move on to using public modules that we need to talk about.  Importing individual sections of a module into a program.  There are few reasons you would want to do this:

• To avoid name conflicts. If you import the whole module, you may have to use the module name as a prefix to access its attributes, such as “module.function”. This can cause problems if you have another variable or function with the same name as the module in your namespace. For example, if you have a variable called math and you import the math module, you will not be able to access the variable anymore. To avoid this, you can import only the function you need from the module, such as“from math import sqrt”, and use it directly without the prefix, such as “sqrt”.
• To improve readability. If you import only the function you need from the module, your code may be more clear and concise. You will not have to write long names like “module.function” every time you use the function, but only the function name itself. This may make your code easier to read and understand.
• To optimize performance. If you import only the function you need from the module, you may save some memory and execution time. When you import a module, Python loads and executes the whole module file, which may contain many definitions and statements that you do not need. This can take some space in memory and some time to run. If you import only the function you need, Python will still load and execute the whole module file, but it will also delete the reference to the module object after assigning the function to a local variable. This can free some memory and reduce the number of lookups when you call the function.

As with everything in life and programming where there are benefits there are also disadvantages, not surprisingly the importation of a function from a module also has some downsides, such as:

• It may take longer to import. As mentioned above, Python still loads and executes the whole module file when you import only a function from it. This can take longer than importing the whole module once and using its attributes later.
• It may cause confusion. If you import only a function from a module, you may not know which module it comes from or what other functions are available in that module. This can make your code less modular and reusable. You may also have to check the documentation or the source code of the module to find out more information about the function.

It is important to note that whether you should import a function from a module or the whole module depends on the specific needs and preferences of the developer and the application being built. When making such a decision you should consider the trade-offs between clarity, performance, and convenience when choosing how to import modules in Python.

Using public modules

The process to use a public module, ie. one that is not in the built-in modules and not written by you (private) is slightly different.  Functionally, usage is same as previously mentioned, however before the module can be utilised it needs to be installed into the environment.  This can be achieved by using a package manager such as pip. A package is a collection of modules (but we are getting ahead of ourselves here).

However, before you install any packages, it is recommended, but not necessary to use a virtual environment to manage your Python projects. A virtual environment is an isolated Python installation that allows you to install packages without affecting the system-wide Python or other projects. This way, you can avoid conflicts, errors, and security issues caused by different versions or dependencies of packages.  That said, we want to do things correctly don’t we.

There are different ways to create and use virtual environments in Python, depending on the version of Python you are using and the tools you prefer. One of the most common tools is for Python 3 “venv”. This tool allow the creation of a virtual environment in a folder which is activated as and when you need it.

Creating your virtual environment

To create a virtual environment that uses the folder “venv”, follow these steps:

• Open a terminal and navigate to the folder where you want to create the virtual environment. You can name the folder anything you want, but usually you will call it env or venv.
• Run the command python3 -m venv venv. This will create a virtual environment named env in the current folder.

There will be no output to stdout when running this command, but it is creating many things in the background. Once the command prompt has returned verify that the folder “venv” has been created and is accessible.

• Activate the virtual environment by running the command source venv/bin/activateon Unix/macOS or venv\Scripts\activate on Windows. You should see (env) at the beginning of your prompt, indicating that you are in the virtual environment.

Creating a virtual environment and installing a third-party module

We are now going to install a common module package called “NumPy” in to your virtual environment. To do this we will first need to install pip (if you have been following the series, you will already have carried out this task)

Install NumPy in the virtual environment

pip install numpy

if everything is correct you will received an output similar to the following:

On thing to note here is that my environment is complaining that I am using an older version of “pip”. If you receive this notice, you can just follow the mentioned command one you have exited the virtual environment.

# Import numpy
import numpy as np

Wait what, that import command is different! Yes, you are correct. What we have done is import “NumPy” into a script using an alias. This will allow us to save time when programming, because when we use  the function as we just write “np.Function” rather than “NumPy.Function” 😊. Because programmers are lazy too LOL.

Finally remember to deactivate the virtual environment once your modules have been installed.

Finally let’s have a look in the “venv” directory.

Here we can see three folders and a “pyvenv.cfg” file.  Let’s first discuss the folders, the “Scripts” folder contains the files needed for the virtual environment including the active and deactivate scripts. Looking in the Include directory it is currently empty; this is expected as the purpose of the Include directory in a virtual python environment is to store the header files that are needed to compile Python extensions or embed Python in other applications. Header files are files that contain declarations of functions, variables, constants, and data structures that are defined in other source files. They usually have the extension .h or .hpp and are used by compilers to check the syntax and types of the code.  The final folder “Lib” contains a second folder “Site-packages”, which as you most likely will have realised contains the files needed for the ”NumPy” package.  Finally, there is the “pyenv.cfg” file which is the virtual environments configuration file.

Summary

As normal we have covered a lot of ground in this article, but hopefully it has been worth it. You should now have a good understanding of the three types of general python modules, Built-in, Public or Third-party, and Private or self-written. We are moving ever so closer to being able to join the dots and start a proper application. In our next article we will finally close out this introduction by talking about packages.