How To Create Golang REST API: Project Layout Configuration [Part 2]

In a previous post I was explaining the basics of setting up GO application for REST API. Now I’ll go into details by first creating configurable server, adding http router (mux) and some DB interaction. Let’s get (the indoors party) started!

The application is now running in docker, can respond to code changes and reload for instant feedback. For handling http requests I will add another dependency, http router (mux). You can read more about it here.

This is a lightweight, high performance HTTP request router that is easy to use and has everything most api’s will need.

$ go get -u github.com/julienschmidt/httprouter

Time to create the server. I’ll place it in pkg/ directory as it could potentially be reused:

As usual, Get function returns pointer to our server instance that exposes some public methods that are pretty self explanatory. It will become more obvious when I put this server to work in the main program.

Server will need routes and handlers to communicate with outside world. I’ll add that next:

I define all my routes in router.go and call handlers by explicitly passing application configuration so that each handler has access to things like database, configuration with env vars and more.

I keep my handlers separate from router and group them under sub folderscmd/api/handlers/{handlerName}. One reason is that handler will have corresponding test file. It will also have multiple middleware files that will also have tests and there can be a lot of handlers. If not grouped correctly, it can get out of hand very fast.

Now there are few more building blocks: server, router, logger. Let’s assemble it all in the main program:

New things to mention is that I assemble the server by chaining method calls and setting properties on the instance. One interesting thing isWithErrLogger(logger.Error), this is just to instruct the server to use my custom logger for consistency.

I start the server in separate go routine so thatexithandlercan still run and gracefully handle program shutdowns. pkg/logger contains 2 instances of standard library Logger. Info is printing out messages to os.Stdout and Error to os.Stderr. I could have used some fancy logger like logrus or any other but I was planning to keep it simple.

Next let’s take care of the databases. I use migration tool written in GO that can be used as CLI or as a library. You can read more about it and find installation instructions here. After installing it, let’s create few migration files. As seen from above, I’ll be operating on /users resource so it’s natural I’ll have users table:

$ migrate create -ext sql -dir ./db/migrations create_user

This will generate 2 migration files indb/migrations, up and down for user table. All files are empty so let’s add some sql

Up:

And down:

Pretty simple, but that’s how it should be, right? Before running migrations, let’s usegolang-migrate library and create a program to simplify this process. This will also work nicely in CI/CD pipelines as it will let us skip the installation ofgolang-migrate cli as a separate step of the pipeline build. For that to happen I’ll add yet another dependency:

go get -u github.com/golang-migrate/migrate/v4

I’ll name my program dbmigrate:

Quick overview of what’s happening here. First of all I load env vars. I then get pointer to instance of config that will give me easy access to all vars I need with some helper methods. You might have noticed that there’s a newGetMigration method. It will simply return up or down string to instruct my program if it should migrate database up or down. You can see latest changes here.

Now since I have this tool in place I can put it to work. Best place I found for it isscripts/entripoint.dev.sh. By running it there I’m avoiding common “oh, I forgot to run migrations” issue. Updated version of entrypoint.dev.sh:

What’s happening here? First run of dbmigrate will use all default values from .env file, so it will run the migrations up againstboilerplate db. In second run I pass-dbname=boilerplatetest so that it does the same but againstboilerplatetest db. Next I’ll start my app with clean state:

# remove all containers docker container
rm -f $(docker container ps -a -q)
# clear volumes
docker volume prune -f
# start app
docker-compose up --build

If all the above has worked, we should see users table in bothboilerplate andboilerplatetest databases. Let’s check that:

# connect to pg
docker container docker exec -it $(docker ps --filter name=pg --format "{{.Names}}") /bin/bash
# launch psql cli
psql -U postgres -W
# ensure both DBs still present
\l
# connect to boilerplate database and list tables
\c boilerplate
\dt
# do same for boilerplatetest
\c boilerplatetest
\dt
# in both databases you should see users table

This is what I see when the above commands are run:

And sure thing it’s all as expected. Now what if we add new migrations while application is running in docker. I’m pretty sure it’s not very convenient to stop docker-compose and rerun the command again for changes to take place. Well, dbmigrate program is capable of handling this scenario. In new terminal tab:

# migrate boilerplatetest db down
go run cmd/dbmigrate/main.go \
  -migrate=down \
  -dbname=boilerplatetest \
  -dbhost=localhost

# you can now repeat steps from above to connect to pg container
# and ensure that users table is missing from boilerplatetest DB.

# now bring it back up
go run cmd/dbmigrate/main.go \
  -migrate=up \
  -dbname=boilerplatetest \
  -dbhost=localhost

One thing to mention here is-dbhost=localhost. This is because we connect to pg container from our host machine. Within docker-compose we can refer to container by service name, which is pg, but we can’t do same from host.

I hope you have learned something useful. In part 3 I’ll go through simple CRUD operations for our users resource. It will include middleware usage, validations and more. You can also see the whole project and follow the progress here. Stay safe!