Navigate back to the homepage

Managing data models in Go

Claas Störtenbecker
March 4th, 2021 · 1 min read

Once upon a time a developer created a new Go service. He started by defining the service interface as protobuf spec, generated code, and saw it was good. The need to store data creeped up to him. Being a smart hackernews reader he chooses sqlc to generate CRUD and model code from sql. Then the product owner reached out to implement new business logic. Striving for a clean code base the developer creates a new data structure once again, adding methods to the new struct. After awhile inconsistency in the code base started to show.

In how many places does code break if we touch files code is generated from? Many.

Let’s make some observations.

  1. The database schema is most likely the root of the datamodel.
  2. Protocol buffers exist mainly to define an interface, used for GRPC communication.
  3. Business logic doesn’t have to be a method of a struct, though it makes sense in many cases.

The goal should be to have a single struct for each data model which is used throughout the entire codebase. To achieve this we make use of struct embedding and methods that handle transcoding.

1// generated
2
3// /pkg/pb/account.pb.go
4
5type Account struct {
6 state protoimpl.MessageState
7 sizeCache protoimpl.SizeCache
8 unknownFields protoimpl.UnknownFields
9
10 Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
11 Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
12}
1// generated
2
3// /pkg/db/model.go
4
5type Account struct {
6 ID uuid.UUID `json:"id"`
7 Name string `json:"name"`
8}
1// /pkg/business/account.go
2
3import "pkg/db"
4import "pkg/pb"
5
6type Account struct {
7 db.Account
8}
9
10// This is pretty straight forward.
11// Generated query functions will return a db.Account and we can transcode swiftly.
12// Nothing can break here.
13func AccountFromDB(a db.Account) Account {
14 return Account{Account: a}
15}
16
17// Arguably the most complicated function.
18// We need to transcode a protobuf struct to our business logic struct.
19// Writing a generator for std data types shouldn't be to complicated.
20// How to extend it to custom types needs some thought.
21func AccountFromPB(a *pb.Account) (Account, error) {
22 id, err := uuid.FromStringOrNil(a.Id)
23 if err != nil {
24 return Account{}, err
25 }
26
27 account := Account{}
28 account.ID = id
29 account.Name = a.Name
30
31 return account, nil
32}
33
34// This is pretty simple again and could easily be generated.
35func (a Account) PB() *pb.Account {
36 return &pb.Account{
37 Id: a.ID.String(),
38 Name: a.Name,
39 }
40}

When investing the effort to write robust generators one time, maintenance headache in the code base is reduced significantly. Mentioned generator could even be used universally over services.

Join our email list and get notified about new content

Be the first to receive our latest content with the ability to opt-out at anytime. We promise to not spam your inbox or share your email with any third parties.

More articles from clstb.codes

Secure HA Kubernetes on bare metal using k3s

Walkthrough deploying secure HA Kubernetes on bare metal using k3s and embedded etcd.

December 21st, 2020 · 1 min read
© 2020–2021 clstb.codes
Link to $https://github.com/clstbLink to $https://linkedin.com/in/clstbLink to $mailto:[email protected]