Introducing deepcopier, a Go library to make copying of structs a bit easier
-
-
13 April 2017
Context
We are currently refactoring our API at Ulule from our monolithic Python stack with django-tastypie to a separate Go microservice.
When working with models in Go, you don't want to expose all columns and also implement more methods without writing a lot of code, because everyone knows programmers are lazy ;)
deepcopier will help you in your daily job when you want to copy a struct into another one (think resource) or from another one (think payload).
Installation
Assuming you are already a Go developer, you have your environment up and ready, so run this command in your shell:
$ go get github.com/ulule/deepcopier
You are now ready to use this library.
Usage
To demonstrate why you should use this library, we will build a dead simple REST API in READ only.
We will use postgresql as database so I'm also assuming you already have postgresql installed on your laptop :)
Let's create the databass!
$ psql postgres psql (9.4.1) Type "help" for help. postgres=# create user dummy with password ''; CREATE ROLE postgres=# create database dummy with owner dummy; CREATE DATABASE postgres=# \d No relations found.
We now have a perfectly capable database with no tables, let's jump to the SQL schema.
CREATE TABLE account (
id serial PRIMARY KEY,
first_name VARCHAR(50),
last_name VARCHAR(50),
username VARCHAR (50) UNIQUE NOT NULL,
password VARCHAR (50) NOT NULL,
email VARCHAR (355) UNIQUE NOT NULL,
date_joined TIMESTAMP NOT NULL
);
Transfer this schema to postgresql.
$ psql -U dummy psql (9.4.1) Type "help" for help. dummy=# CREATE TABLE account( dummy(# id serial PRIMARY KEY, dummy(# first_name VARCHAR (50), dummy(# last_name VARCHAR (50), dummy(# username VARCHAR (50) UNIQUE NOT NULL, dummy(# password VARCHAR (50) NOT NULL, dummy(# email VARCHAR (355) UNIQUE NOT NULL, dummy(# date_joined TIMESTAMP NOT NULL dummy(# ); CREATE TABLE dummy=# \d List of relations Schema | Name | Type | Owner --------+----------------+----------+------- public | account | table | thoas public | account_id_seq | sequence | thoas (2 rows) dummy=#
First insertions incoming!
dummy=# INSERT INTO account (username, first_name, last_name, password, email, date_joined) VALUES ('thoas', 'Florent', 'Messa', '8d56e93bcc8d63a171b5630282264341', 'foo@bar.com', '2015-07-31 15:10:10');
At this point, we have a schema in a great database, we need to setup our REST API.
We will use:
- go-json-rest to handle requests
- gorm to manipulate the database as an ORM
In your shell, run this to install them
$ go get -u github.com/jinzhu/gorm $ go get github.com/ant0ine/go-json-rest/rest
We will define a first attempt of our API to retrieve user information based on its username.
We will rewrite our API three times so you need to focus.
// main.go
package main
import (
"fmt"
"github.com/ant0ine/go-json-rest/rest"
"github.com/jinzhu/gorm"
_ "github.com/lib/pq"
"log"
"net/http"
"os"
"time"
)
type Account struct {
ID uint `gorm:"primary_key"`
FirstName string
LastName string
Username string
Password string
Email string
DateJoined time.Time
}
type Accounts struct {
Db gorm.DB
}
func (a *Accounts) Detail(w rest.ResponseWriter, r *rest.Request) {
account := &Account{}
result := a.Db.First(&account, "username = ?", r.PathParam("username"))
if result.RecordNotFound() {
rest.NotFound(w, r)
return
}
w.WriteJson(&account)
}
func main() {
dsn := fmt.Sprintf("user=%s dbname=%s sslmode=disable",
os.Getenv("DATABASE_USER"),
os.Getenv("DATABASE_NAME"))
db, err := gorm.Open("postgres", dsn)
fmt.Println(dsn)
if err != nil {
panic(err)
}
db.DB()
db.DB().Ping()
db.DB().SetMaxIdleConns(10)
db.DB().SetMaxOpenConns(100)
db.SingularTable(true)
db.LogMode(true)
api := rest.NewApi()
api.Use(rest.DefaultDevStack...)
accounts := &Accounts{Db: db}
router, err := rest.MakeRouter(
rest.Get("/users/:username", accounts.Detail),
)
if err != nil {
log.Fatal(err)
}
api.SetApp(router)
log.Fatal(http.ListenAndServe(":8080", api.MakeHandler()))
}
Let's start the server then
$ DATABASE_USER=dummy DATABASE_NAME=dummy go run main.go
and retrieve the response.
$ curl http://localhost:8080/users/thoas { "ID": 1, "Username": "thoas", "FirstName": "Florent", "LastName": "Messa", "Password": "8d56e93bcc8d63a171b5630282264341", "Email": "foo@bar.com", "DateJoined": "2015-07-31T15:10:10Z" }
Wait a minute? You are exposing the user's password... this not what we are excepting... We want this specific format
{
"id": 1,
"username": "thoas",
"first_name": "Florent",
"last_name": "Messa",
"name": "Florent Messa",
"email": "foo@bar.com",
"date_joined": "2015-07-31T15:10:10Z",
"api_url": "http://localhost:8080/users/thoas"
}
Implement a separate struct named AccountResource
type AccountResource struct {
ID uint `json:"id"`
Username string `json:"username"`
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
Name string `json:"name"`
Email string `json:"email"`
DateJoined time.Time `json:"date_joined"`
}
func (a Account) Name() string {
return fmt.Sprintf("%s %s", a.FirstName, a.LastName)
}
and rewrite Accounts.Detail to use deepcopier
func (a *Accounts) Detail(w rest.ResponseWriter, r *rest.Request) {
account := &Account{}
result := a.Db.First(&account, "username = ?", r.PathParam("username"))
if result.RecordNotFound() {
rest.NotFound(w, r)
return
}
resource := &AccountResource{}
deepcopier.Copy(account).To(resource)
w.WriteJson(&resource)
}
We are good now, we can inspect our result
$ curl http://localhost:8080/users/thoas { "id": 1, "username": "thoas", "first_name": "Florent", "last_name": "Messa", "name": "Florent Messa", "email": "foo@bar.com", "date_joined": "2015-07-31T15:10:10Z" }
Easy, right?
We will now rewrite for the last time Accounts.Detail to provide some context to retrieve the base url in api_url attribute.
func (a *Accounts) Detail(w rest.ResponseWriter, r *rest.Request) {
account := &Account{}
result := a.Db.First(&account, "username = ?", r.PathParam("username"))
if result.RecordNotFound() {
rest.NotFound(w, r)
return
}
resource := &AccountResource{}
context := map[string]interface{}{"base_url": r.BaseUrl()}
deepcopier.Copy(account).WithContext(context).To(resource)
w.WriteJson(&resource)
}
We need to update AccountResource to implement the ApiUrl new method
type AccountResource struct {
ID uint `json:"id"`
Username string `json:"username"`
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
Name string `json:"name"`
Email string `json:"email"`
DateJoined time.Time `json:"date_joined"`
ApiUrl string `deepcopier:"context" json:"api_url"`
}
func (a Account) Name() string {
return fmt.Sprintf("%s %s", a.FirstName, a.LastName)
}
func (a Account) ApiUrl(context map[string]interface{}) string {
return fmt.Sprintf("%s/users/%s", context["base_url"], a.Username)
}
We have now the final result of what we excepted for the first time :)
$ curl http://localhost:8080/users/thoas { "id": 1, "username": "thoas", "first_name": "Florent", "last_name": "Messa", "name": "Florent Messa", "email": "foo@bar.com", "date_joined": "2015-07-31T15:10:10Z", "api_url": "http://localhost:8080/users/thoas" }
If you have reached to the bottom you belong to the brave!
It has been a long introduction, hope your enjoy it!
Contributing to deepcopier
- Ping us on twitter @oibafsellig, @thoas
- Fork the project
- Fix bugs
Don't hesitate ;)