Welcome to the eleventh article in our series on learning GO; All the previous articles in this series can be found on the links below:
- It is time to “Go” and learn how to code with Go
- It is Time to Go: our journey into grown-up code Part 2
- It is Time to Go: our journey into grown-up code Part 3
- It is Time to Go: our journey into grown-up code Part 4
- It is Time to Go: our journey into grown-up code Part 5
- It is Time to Go: our journey into grown-up code Part 6
- It is Time to Go: our journey into grown-up code Part 7
- It is Time to Go: our journey into grown-up code Part 8
- It is Time to Go: our journey into grown-up code Part 9
- It is Time to Go: our journey into grown-up code Part 10
Our journey to date has covered a lot of ground, and we have slowly built up an understanding of several core features and concepts. As a result, we are genuinely building firm foundations.
- The core Variables types
- Variadic variables
- Redeclaration
- Constants
- inputs
- Structs
- Pointers
- Interfaces
- Function basics
- Conditional statements, if statements and switch statements.
- Loops, breaks, deferment, continue, goto, return and recover
- Range
- Importation of built-in functions (“fmt”, “sync”, “reflect”, “strings”, “strconv”, “math”, “net/html”, “html/template”)
- Arrays and Slices
- Importing third-party packages from Github.com.
By the end of our last article, we had successfully created our first API, which we can use to create, update, delete and list the items we had stored in a slice of multiple slices.
What are we looking at today?
Today we will continue our journey into CRUD; if you did not install your MySQL database as a part of the previous article, we must do it now; it is one of the prerequisites for this article. Installing MySQL is a simple enough task; point your favourite browser at the following website https://dev.mysql.com/downloads/installer/ and download the version for your particular operating system.
Time to improve on our API Fu with more Go
As per our standard modus operandi, we will look at an architectural diagram of our application to gain a visual representation of what we are building. The first change we will notice is that we have a folder structure for our application; we have a “CMD” Folder and a “PKG” folder, under which there are five more folders. Creating folders is a common way of building out complex applications. If you have been coding in Terraform, it may remind you of the differentiation between our modules and our configuration or” tfvars” folders, and that is a valid analogy. With this structure, we can start commercialising our code by beginning the process of making it DRY.
As already alluded, we will introduce a project structure to this application; our folder structure will look similar to that shown below. As we have said, splitting our code into several distinct folders will allow the separation of roles and functions, and ease the ability to understand what our code is doing; yes, we could have all the functions and procedures in the “main.go” file, but as a project‘s complexity rises, the ability to manage to work on your code will decrease. This issue will only compound when your project grows in size beyond the ability of a single person to handle it.
We also mentioned this time that we would be backing our application with a database, which means we can have persistent storage for our data. So this also means that we will be importing some more third-party packages, GORM; we will also retain the services of our trusty Gorilla MUX. We will also dive deeper into JSON Marshalling and Unmarshalling the acts of encoding and decoding JSON in Go.
Let’s get started with Bookstore V2.0 in Go.
I am assuming you have already created your file structure. So we get started with our “main.go” file, we need to prepare our environment; this means creating the “go.mod” file and installing our third-party packages.
To prepare your environment, the first command to issue is “go mod”, which creates the ”go.mod” file in your working directory.
Next, issue “go mod tidy” command to confirm everything is as it should be. Next, we install the third-party packages. There are three packages to install; these are “GORM”, which is an “Object Relation Mapper”, this is a piece of software that aids in mapping objects in your code to the relevant objects in a database and the “GORM dialect for MySQL” and finally our trusty “Gorilla/MUX”
Now that we have initialised our environment, we can get down to our coding. Typically, we would start with writing the “main.go” file. However, starting with “main.go” would be confusing, as there is little code in there. So we need a strategy for writing out our code. So we will start with writing out the “routes”, now historically these were in our “main.go” file; however, today, we will be placing them in a separate file in the folder “routes”, here we create the “bookstore-routes.go” file. First, you will notice that we have written “package routes” rather than “package main”. This is because there can only be a single package main file in any project. So just like with Terraform modules, you should give your packages meaningful names to identify their use.
package routes
Next, as expected, we import our packages; here, we import the “gorilla/mux” package again, and also we place a pointer to our controllers’ folder so that “routes.go” can import the “controller.go” file. You may be asking why we use an “absolute path” rather than a “relative path”. The simple answer is because that is just the way it is. However, if you remember the “init” command we issued at the start of this post “github.com/tomhowarth/bookstore-v2”, all we are adding to this is the “package/controllers” to the path.
import ( "github.com/gorilla/mux" "github.com/tomhowarth/bookstore-v2/package/controllers" )
Finally, we arrive at our first package’s meat and two veg, our function “RegisterBookStoreRoutes”. There is nothing new here other than adding a folder pointer to the relevant module; we are effectively repeating what we wrote in the original bookstore post in the “main.go” file.
var RegisiterBookStoreRoutes = func(router *mux.Router){
router.HandleFunc(“/book/”, controllers.CreateBook).Method("POST")
router.HandleFunc(“/book/”, controllers.GetBook).Method("GET")
router.HandleFunc(“/book/{bookId}”, controllers.GetBookByID).Method("GET")
router.HandleFunc(“/book/{bookId}”, controllers.UpdateBook).Method("PUT")
router.HandleFunc(“/book/{bookId}”, controllers.DeleteBook).Method("DELETE")
}
Next, we will write the “app.go” and the “utils.go” files. The “app.go” file is to aid us in connecting to our new MySQL database, and the “util.go” file aids in parsing “JSON” into a format that the database understands. Both these are small files and should not cause you too much trouble.
Firstly the “app.go” file, this file is what controls the connection to the backend database. We have called this package “config”, and we have imported to third-party packages “github.com/jinzhu/gorm” and “github.com/jinzhu/gorm/dialects/mysql”, both these relate to connectivity. The first package, “gorm”, directly addresses the “ORM” (“Object Relational Mapping”) of objects between programs and the backend database; the second import provides the nuances for the MySQL database.
Finally, we have two functions; the first, “GetDB()” obtains the database connection object. And the “func connect()” handles the physical connection.
package config import ( "github.com/jinzhu/gorm" _ "github.com/jinzhu/gorm/dialects/mysql" )
Next, we create a variable “db” from the “GORM.DB” object.
var ( db * gorm.DB )
The “func Connect()” function handles the physical connection to the database. This simple function utilises “gorm.Open” to assign the variable “db” to the connection object. Next, we will briefly explain the connection string in the “gorm.Open” line. The first option is the Database type, in our case “MySQL”; the second section is the “DSN” (“Data source name”); this is broken down into two distinct sections, the first, the location, together with the credentials needed for access “username:password@IP address or FDQN / Database instance”. The second half relates to connection string attributes, “?charset=utf8&parseTime=True&Loc=Local”. So we see we are setting the charset to “utf8”, as shown with “charset=utf8”. Next, we tell the DB to convert TIMESTAMP, DATETIME, TIME, and DATE values into time with the setting ”parseTime=True”, and finally, ”Loc=Local” is used to set the local timezone, to the timezone set on the client, not the server.
func Connect(){
d, err := gorm.Open("mysql","root:password@localhost@/simplerest?charset=utf8&parseTime=True&Loc=Local")
if err != nil{
panic(err)
}
db = d
}
With the “GetDB()” functions, we connect to a MySQL database and set the global variable “db” to the connection object.
func GetDB() *gorm.DB{
return db
}
Next, we will write the “utils.go” file in the utils folder. This code reads the body of an HTTP request and unmarshals (marshaling and unmarshalling refer to transforming an object into a specific data format suitable for transmission) and passing it onto an “interface{}” object. The function takes two arguments: an “*http.Request” and an “interface{}” object. The HTTP request is passed as a pointer to the function. The “interface{}” object is passed as a pointer to the function. The function reads the body of the HTTP request using “ioutil.ReadAll()” and then unmarshals it into the interface{} object using “json.Unmarshal()”. If there is an error while reading the body or unmarshaling it, the function returns without doing anything.
package utils
import(
“encoding/json”
“io/ioutil”
“net/http”
)
func ParseBody(r *http.Request, x interface{}) {
if body, err := ioutil.ReadAll(r.Body); err == nil {
if err := json.Unmarshal([]byte(body), x); err != nil {
return
}
}
}
Now we can move on to our “main.go” file; first, you will notice that it is significantly shorter than our last couple of packages. Next, you will see the “_” in front of the importation of the “gorm/dialects/mysql” package. The underscore is used as a blank identifier in Golang; in this case, we are importing the package without assigning it to a variable. The “func main()” function is easily recognisable. Per the original bookstore package, we are creating a new “MUX”, invoking an HTTP service listening on port 9010. This function’s only line of interest is “routes.RegisterBookStoreRoutes(r)”. This line informs that function that the MUX’s routes are held in the folder “routes” and the function “RegisterBookStoreRoutes”.
package main
import (
“log”
“net/http”
"github.com/gorilla/mux"
_ "github.com/jinzhu/gorm/dialects/mysql"
"github.com/tomhowarth/bookstore-v2/package/routes"
)
func main() {
r := mux.NewRouter()
routes.RegisterBookStoreRoutes(r)
http.Handle("/", r)
log.Fatal(http.ListenAndServe("localhost:9010", r))
}
Then we will create our “book.go” file in the models’ folder, which is slightly larger and more complicated. Most of the code in this file should be understandable by now. We have our importation block, and we also create a struct. Which uses “gorm.Model” and contains three entries, which are the basis of our database table.
package models
import (
"github.com/tomhowarth/bookstore-v2/package/config"
"github.com/jinzhu/gorm"
)
var db *gorm.DB
type Book struct {
gorm.Model
Name string `gorm: ““json: “name”`
Author string `json: “author”`
Publication string `json: “publication”`
}
We next introduce another “Special” function, “ func init()”. This function runs automatically at the beginning of the program’s execution. The function is used to perform initialisation tasks; ours establishes a connection to our database and the instance defined in our “db” variable, finally the line “db.AutoMigrate(&Book{})” creates a table for the Book struct is it does not already exist.
func init() {
config.Connect()
db = config.GetDB()
db.AutoMigrate(&Book{})
}
Our next four functions relate to “creating”, “reading”, and “deleting” records from our backend database. Our first function, “CreateBook()”, creates a new record, writes the data in the struct Book into the DB and returns the same.
func (b *Book) CreateBook() *Book {
db.NewRecord(b)
db.Create(&b)
return b
}
“GetAllBooks()” will return all book records held in the database.
func GetAllBooks() []Book {
var Books []Book
db.Find(&Books)
return Books
}
“GetBooksById()” will return an individual book as defined by its record ID number.
func GetBookById(Id int64) (*Book, *gorm.DB) {
var getBook Book
db := db.Where(“ID=?”, Id).Find(&getBook)
return &getBook, db
}
Our final function, “DeleteBook()”, will delete an individual book as defined by the given ID
func DeleteBook(Id int64) Book {
var book Book
db.Where(“ID=?”, Id).Delete(book)
return book
}
Our final file is the “controller.go” file; this is the most complex of the files we have been working through today.
package controllers import( “encoding/json” “fmt” "github.com/gorilla/mux" “net/http” “strconv” "github.com/tomhowarth/bookstore-v2/package/utils" "github.com/tomhowarth/bookstore-v2/package/models" )
This is an interesting variable; we have created “NewBook” from the contents of our “book.go” package, which is stored in our models’ folder. This is a nifty little trick to help reduce your code base. By importing the “book.go” package as a variable, we can directly use any variables defined in that package rather than redeclaring them here.
var NewBook models.Book
There is a lot of repetition in the following four functions; this is to be expected. There should also be a lot that should look familiar from our previous articles. Our first function, “GetBook()”, should look vaguely familiar; the function takes two arguments, “http.ResponseWriter” and “http.Request”. Now we have discussed these previously. Our following line creates and sets a new variable which takes the contents of the return of “model.GetAllBooks()”. Our subsequent line is interesting as it sets a variable “res”, which is made up of the output of the “json.Marshal()” function, which encoded the “newBooks” variable to return a JSON string. You will also notice the “_”; this is used to ignore any errors returned by the function. The final three lines take the response and write it out to the client, a Postman instance.
func GetBook(w http.ResponseWriter, r *http.Request){
newBooks := models.GetAllBooks()
res, _ :=json.Marshal(newBooks)
w.Header().Set("Content-Type", "pkglication/json")
w.WriteHeader(http.StatusOK)
w.Write(res)
}
Our second function is functionally similar to our first, except it will return a single Book rather than the whole catalogue. I will only explain the differences as we have previously discussed most lines. We have five variables, “vars”, “bookID”, “ID”, “bookDetails” and “res”. These variables build up a path to a single book entry in the database. Our function extracts the book “ID” from the given URL path using the mux.Vars(r) and then converts it to an integer using the strconv.ParseInt(). We then call the models.GetBookById() to retrieve the book details from the database. Finally, we marshal the book details into JSON using the json.Marshal() and write our respounce out to HTTP using w.Write().
func GetBookById(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
bookId := vars["bookId"]
ID, err := strconv.ParseInt(bookId, 0, 0)
if err != nil {
fmt.Println("error while parsing")
}
bookDetails, _ := models.GetBookById(ID)
res, _ := json.Marshal(bookDetails)
w.Header().Set("Content-Type", "pkglication/json")
w.WriteHeader(http.StatusOK)
w.Write(res)
}
Our third function, “CreateBook,” adds new books to the database. First, we add a new instance to the “book” struct, which is contained in the modules package and store it in the variable “CreateBook” we then use “utils.ParseBody” to take our request body from our Postman request and keep the response in our “CreateBook” variable. The function then calls the “CreateBook” method on the “CreateBook” variable to create a new book and stores the result in the “b” variable. The function then marshals the result into JSON format using the” json.Marshal” function and holds the response in the “res” variable, and we write an HTTP status code of 200 (OK) to the response writer and write the marshaled JSON data to it.
func CreateBook(w http.ResponseWriter, r *http.Request) {
CreateBook := &models.Book{}
utils.ParseBody(r, CreateBook)
b := CreateBook.CreateBook()
res, _ := json.Marshal(b)
w.WriteHeader(http.StatusOK)
w.Write(res)
}
Our final functions are our most complicated, but once again, there is a large amount of repetition in the code; we should now be starting to recognise patterns. We will first look at the “DeleteBook()” function; again, a lot should be well-known by now. This function is an almost perfect copy of the “GetBookByID()” Function; however, instead of calling “Models.GetBookByID()” we call the “models.DeleteBook(“)” function.
func DeleteBook(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
bookId := vars["bookId"]
ID, err := strconv.ParseInt(bookId, 0, 0)
if err != nil {
fmt.Println("error while parsing")
}
book := models.DeleteBook(ID)
res, _ := json.Marshal(book)
w.Header().Set("Content-Type", "pkglication/json")
w.WriteHeader(http.StatusOK)
w.Write(res)
}
Our final function is “UpdateBook()”; this function will change data associated with the named record as defined by the “bookID”, again we have a lot of repetition; you will recognise a significant amount of code from the “GetBookById()” function. We then enter into a set of three “if” statements, where each will validate if either the “Name”, “Author”, or “Publisher” string is empty; if it is not, then the value of the Struct entity is updated to the value contained in the relevant object. We then use the Gorm package to save the updated details to the database. Finally, the updated book details are marshaled into JSON format using the “json.Marshal()” function and written to the response with a content type of “pkglication/json” and a status code of 200 (OK).
func UpdateBook(w http.ResponseWriter, r *http.Request) {
var updateBook = &models.Book{}
utils.ParseBody(r, updateBook)
vars := mux.Vars(r)
bookId := vars["bookId"]
ID, err := strconv.ParseInt(bookId, 0, 0)
if err != nil {
fmt.Println("error while parsing")
}
bookDetails, db := models.GetBookById(ID)
if updateBook.Name != "" {
bookDetails.Name= updateBook.Name
}
if updateBook.Author != "" {
bookDetails.Author = updateBook.Author
}
if updateBook.Publication != "" {
bookDetails.Publication = updateBook.Publication
}
db.Save(&bookDetails)
res, _ := json.Marshal(bookDetails)
w.Header().Set("Content-Type", "pkglication/json")
w.WriteHeader(http.StatusOK)
w.Write(res)
}
I made a big mistake in my go code.
So now that all our code has been written, it is time to test it. So navigate to the cmd/main folder run the command “go build”, yes with the build command, you do not have to give it the “main.go” file. One thing to note is that using “go run” will require you to provide it with the pointer to your necessary go file. For example, when running the command “go build”, I received the following error.
I thought, “Well that’s not meant to happen; I should be deep in Postman by now testing my API’s”. At this time, I realised that I had fallen into a schoolboy error; I had recently updated my Go version from 1.16 to the latest and greatest 1.20 and had not bothered to check my version dependencies. I quickly checked with our best friend Mr G. Oogle, who glibly informed me that the latest supported version of GoLang for the “Gorilla/Mux” is “1.17”, more alarmingly it also told me that the repro is in archive read-only mode and has been since February 2022, so there have been no fixes for potential security vulnerabilities in that repo’s code base. The end position is that it means we need a new mux. At first, I thought of scrapping this article and rewriting it with the new mux, but this is a learning lesson, and as such, it is valuable, also, there is a lot of good information on how to organise a project. So the focus of our next article we be refactoring code to take account of legacy repos no longer being supported.
