Connecting Dots: Go, Docker and k8s [Part 1]
Nowadays successful applications often consist of containers and container management system to: 1) ease scaling 2) reduce down time 3) add self healing capabilities. Let’s take a high level overview of most commonly used tools and build something cool too.
My plan is to create open tcp port scanning tool. Use GO and worker pool to make it very fast. Expose it via REST resource, containerise, deploy and scale using kubernetes.
For this example I’ll use my favourite project structure. You can find more about it here. Let’s review the REST server:
I created an instance of cfg package that will be used to hold program wide configuration and will ease the DI (dependency injection). This usually is a good place to parse any command line arguments, assemble and hold DB connection details and more.
I then create and configure reusable server instance by passing port, router and logger to it. Server is started in separate go routine so I could initiate a graceful shutdown service to handle program exits and perform cleanup if needed.
I will only have one GET open-ports end-point. Let’s review the handler
I expect to receive few query params in this handler: domain and toPort. Both params are mandatory and toPort must be valid decimal string representation. I have validator in place to check exactly that. The toPort number will represent the port limit that my scanner will check and also define the size of worker pool. This will allow the scanner to perform all calls at once.
Before going any further, let’s review the validator:
Required and ValidDecimalString are my basic validations. Given the field name and value, I keep track of error messages if validation fails. Call to Valid checks to see if there're any errors and if errors are present I can retrieve them with GetErrResp and send that information to consumer.
Now let’s review the scanner:
The ScanTo function does the heavy lifting here. I first create jobs and results channels. I then spawn workers. Each worker will be receiving port over jobs channel, scan that port and return result of the TCP call over results channels. I use net.DialTimeout so that worker does not wait for response for longer than 2 seconds. When last port is finished scanning I return all open ports to consumer.
You can already try interacting with this tool by running:
$ go run cmd/server/main.goAnd issuing curl commands for localhost:8080/open-ports
I hope you have learned something useful. In next post I’ll go through steps needed to containerise this app, deploy and scale using k8s. You can find the source code here.
