kanna-mapper

kanna-mapper writes the functions between your domain types and the wire types you do not control. You declare their signatures; it writes the bodies, field by field, calling the conversions you list for the types Go cannot convert on its own.

Part of the kanna family of code generators. Requires Go 1.26.

Install

go get -tool github.com/go-kanna/mapper/cmd/kanna-mapper

There is no runtime to import: the generated code calls your types, the functions you list, and fmt, along with the constructors of your value types and enums.

Use

Declare the functions you want as the methods of an interface marked //kanna:mapper, and list the conversions Go cannot do by itself in a //kanna:converter list:

package mapper

//go:generate go tool kanna-mapper

import (
	"uuid"

	"google.golang.org/protobuf/types/known/timestamppb"

	"example.com/app/gen/employeev1"
	"example.com/app/model"
)

//kanna:mapper
type _ interface {
	EmployeeToProto(model.Employee) *employeev1.Employee
	EmployeeFromProto(*employeev1.Employee) (model.Employee, error)
}

//kanna:converter
var _ = []any{
	uuid.UUID.String,
	uuid.Parse,
	timestamppb.New,
	(*timestamppb.Timestamp).AsTime,
}

With these types on either side:

// package model: yours
type Employee struct {
	ID       uuid.UUID
	FullName string `map:"Name"`
	HiredAt  time.Time
	Address  Address
}

// package employeev1: generated by protoc-gen-go
type Employee struct {
	Id      string
	Name    string
	HiredAt *timestamppb.Timestamp
	Address *Address
	// ...
}

go generate ./... writes mapper_gen.go next to the declaration:

// Code generated by kanna-mapper. DO NOT EDIT.

// EmployeeToProto maps model.Employee to *employeev1.Employee.
func EmployeeToProto(src model.Employee) *employeev1.Employee {
	return &employeev1.Employee{
		Id:      src.ID.String(),
		Name:    src.FullName,
		HiredAt: timestamppb.New(src.HiredAt),
		Address: addressToEmployeev1(src.Address),
	}
}

// EmployeeFromProto maps *employeev1.Employee to model.Employee.
func EmployeeFromProto(src *employeev1.Employee) (model.Employee, error) {
	if src == nil {
		return model.Employee{}, nil
	}
	v, err := uuid.Parse(src.GetId())
	if err != nil {
		return model.Employee{}, fmt.Errorf("map model.Employee.ID: %w", err)
	}
	return model.Employee{
		ID:       v,
		FullName: src.GetName(),
		HiredAt:  src.GetHiredAt().AsTime(),
		Address:  addressToModel(src.GetAddress()),
	}, nil
}

// addressToEmployeev1 maps model.Address to *employeev1.Address.
func addressToEmployeev1(src model.Address) *employeev1.Address {
	// ...
}

The declaration is ordinary Go: the compiler and your editor check every type and function in it, and the imports it uses need no blank import to stay. The interface is never implemented, which is why it can be called _; any name works. (uuid is the standard library's, new in Go 1.27; github.com/google/uuid is listed the same way.)

A mapping that cannot be written stops generation, rather than the first request that meets it. Had the wire type carried a Revision the domain does not have, the domain's Status been an enum of its own, and EmployeeFromProto been declared without its error:

$ go generate ./...
./mapper.go:15:2: error: EmployeeToProto: no source in model.Employee for employeev1.Employee.Revision
  add map:"Revision" to the field of model.Employee that fills it, or leave it zero with //kanna:map Revision=-
./mapper.go:15:2: error: EmployeeToProto: cannot map model.Employee.Status (model.Status) to employeev1.Employee.Status (employeev1.Status)
  add a func(model.Status) employeev1.Status to a //kanna:converter list
./mapper.go:16:2: error: EmployeeFromProto has to return an error: ID is converted by uuid.Parse, which can fail
  declare it as EmployeeFromProto(*employeev1.Employee) (model.Employee, error)

Declaring functions

Each method of a //kanna:mapper interface is a function kanna-mapper writes, with the name and signature written there:

• one parameter, the source, and a result that is the destination or (destination, error);
• the source and the destination are struct types or pointers to them, and not generic;
• the parameter's name is used when it has one, src otherwise;
• the method's doc comment becomes the function's, //kanna: lines left out; without one, the function gets the sentence above;
• a method with an unexported name gives an unexported function, and a name the package already declares is an error.

A package may have several such interfaces; they all go into its one mapper_gen.go. An interface embedding another is not supported.

Code may call a declared function before the first run has written it, or as its declaration is about to change it. kanna-mapper type-checks the packages it loads once more with what it is about to write in place, and writes nothing while their code does not compile with it: a mistake the missing function hid is reported then, such as a field that what it returns does not have.

Whether a function returns an error is up to its declaration. When a conversion it needs can fail, a declaration without the error stops generation and says which field needs it, so a signature changes only when you change it. A declaration with an error that nothing can produce is fine, and returns nil: an API may expect to fail later.

A struct field whose types make a pair nobody declared, such as Address above, is mapped by an unexported function kanna-mapper writes for that pair and reuses wherever the pair appears. Declare the pair yourself to call it from elsewhere or to adjust its fields with //kanna:map. A problem inside such a function is reported at the declaration that led to it, with the path:

./mapper.go:15:2: error: EmployeeToProto: Address: no source in model.Address for employeev1.Address.Etag
  declare AddressToEmployeev1(model.Address) *employeev1.Address in the //kanna:mapper interface and add //kanna:map Etag=-

Conversions

A //kanna:converter list names the functions that convert one type to another when Go cannot:

• a function of any package, by name: uuid.Parse, timestamppb.New, or one of your own;
• a method, as a method expression: uuid.UUID.String, (*timestamppb.Timestamp).AsTime.

Each takes the source and returns the destination, optionally with an error: func(A) B or func(A) (B, error). A function is called as such and a method as a method, src.ID.String(). A function literal, a method value, or a generic function is rejected; give it a name.

A list is a variable of type []any initialized with a composite literal, and its name may be _. Every list in a package applies to every //kanna:mapper interface of that package. Nothing outside a list is ever called, but a function of the package that would fit a missing conversion is pointed out in the error.

Two entries of a package's lists for the same pair of types stop generation, used or not, so that adding uuid.MustParse next to uuid.Parse cannot change which one runs. So does an entry for a pair one of the package's declared functions already maps.

Sharing a list

A list with a name can be an entry of another list, which then holds everything in it. Write the conversions an application uses everywhere once:

package convert

//kanna:converter
var Defaults = []any{
	uuid.UUID.String,
	uuid.Parse,
	timestamppb.New,
	(*timestamppb.Timestamp).AsTime,
}

and use them in each package that maps, such as the handlers of one service:

package billing

//kanna:converter
var _ = []any{
	convert.Defaults,
	statusToProto,
}

An entry naming a list must name one marked //kanna:converter. A function a shared list brings in has to be callable from the package using it, so an unexported one is an error there. The same function arriving twice counts once. A package holding only lists, such as convert, needs no //go:generate line: the packages naming its lists read them.

When two different functions convert the same pair, the nearer one wins, as with fields promoted through embedding: the package's own lists and its declared functions first, then the lists they name, then the lists those name. Two at the same distance are an error. To read a missing timestamp as the zero time.Time in one package, where (*timestamppb.Timestamp).AsTime would make it the Unix epoch, list a function of your own there; it takes the place of the AsTime that convert.Defaults brings.

How fields are matched

Each destination field takes the first rule that matches.

A destination field with no source is an error, not a silent zero value. A field tagged map:"-" takes no part in mapping, in either direction. A tag on a promoted field keeps applying: a field excluded or renamed in an embedded struct is not matched by its name. Neither is a field promoted through an embedded pointer, which would panic to read when the pointer is nil, or through an embedded field tagged map:"-".

A struct mapped to several types may carry a tag naming a field only some of them have, so a tag naming none of a destination's fields is no error by itself. When a destination field has no source and a source field's tag nearly names it, the error points the tag out.

Where a source declares a getter, as protoc-gen-go writes them, it is read through it, src.GetName(), so that a nil message reads as empty. A getter that turns a nil pointer into a zero value is passed over for the field itself, unless its result is already the destination's type, so that a field left unset stays zero rather than being converted from an empty value.

Fields you cannot tag

The types protoc-gen-go writes are not yours to tag. A //kanna:map line above a method says what its destination fields read, for that function only:

//kanna:mapper
type _ interface {
	// EmployeeToProto leaves Revision and UpdateTime to the server.
	//
	//kanna:map Revision=- UpdateTime=-
	EmployeeToProto(model.Employee) *employeev1.Employee
}

Field=- leaves the destination field zero; Field=Source names the source field it reads, as a tag on the destination would. An entry naming no field of the destination is an error, with the field it was probably meant to name. An entry takes the place of the tags for its field, in that function only, so a struct filled from several types can read each under its own name. Field alone changes nothing about how the field is filled, but names it, as a strict mapping requires.

Strict mappings

A destination field with a source of the same name is filled without a word, which is the point within a service, but not where the destination is what the service shows the world: a field added to the domain and to a response type of the same name goes out unreviewed. Mark the interface of such functions //kanna:mapper strict, and each destination field has to be named for its function, by //kanna:map or by a tag, the destination's or the source's naming it:

//kanna:mapper strict
type _ interface {
	//kanna:map ID Name Email=ContactEmail Internal=-
	UserToPublic(model.User) api.User
}

A field matched by its name alone stops generation, and so does a pair of structs a strict function would call a written function for, whichever mapping wrote it, as that function fills its fields unnamed; declare it in the interface, and name its fields:

./mapper.go:15:2: error: UserToPublic: api.User.RecoveryToken is not named; a strict mapping fills only the fields //kanna:map or a tag names
  add RecoveryToken to the //kanna:map of UserToPublic, or RecoveryToken=- to leave it zero

How values are converted

Each field takes the first case that applies.

"Lossless" is meant strictly: int32 → int64 converts, int64 → int32 does not. Neither does int → int32 (int is 64 bits on some platforms), int → uint (sign), int64 → float64 (precision), or a number → string.

Two types declared with type never convert into each other on their own, even when one could be written as the other. A protobuf enum and a domain enum may both be int32, but on the wire 0 is STATUS_UNSPECIFIED, while in the domain it is likely a real status; converting the number would quietly change what it means. List a function between them, or make the domain's an enum of kanna-value, which maps by name. A declared type and a predeclared one still convert: model.Email → string is string(src.Email).

A nil source stays nil whenever the destination is a pointer too. That includes a conversion that takes the source pointer itself, such as (*timestamppb.Timestamp).AsTime filling a *time.Time: it is called only when the source is set. Filling a time.Time, it is called with whatever the source holds, and decides what nil becomes; AsTime makes it the Unix epoch. List a function of your own to decide otherwise.

The error of a conversion that fails names the destination field it was for: map model.Employee.ID: invalid uuid.

Copies

A slice or a map of the same type on both sides is copied, with slices.Clone or maps.Clone, so that changing what the destination holds leaves the source as it was, and the other way round; nil stays nil. What it holds is not copied in turn, nor is a slice inside a struct of the same type: only the field mapped is.

A field to hand over as it is, such as the bytes of a file, which the copy would double, is shared instead. Name it in a //kanna:share line of the function, a destination field, for a type you cannot tag, such as one protoc-gen-go writes; or tag a field of your own type map:",share", or map:"Name,share", for every function that maps it:

//kanna:mapper
type _ interface {
	//kanna:share Content
	DocumentToProto(model.Document) *docv1.Document
}

func DocumentToProto(src model.Document) *docv1.Document {
	return &docv1.Document{
		Tags:    slices.Clone(src.Tags),
		Content: src.Content,
	}
}

A //kanna:share naming no field of the destination, or one holding neither a slice nor a map, stops generation.

Value types

A value type that kanna-value generates is read with its accessor and made with its constructor, so that a value arriving from the wire or a database row goes through its rule like any other. What it holds is converted by the rules above. With timestamppb.New and (*timestamppb.Timestamp).AsTime listed, a *model.CompletedAt, holding a time.Time, maps to and from a *timestamppb.Timestamp:

// TodoToProto maps model.Todo to *todov1.Todo.
func TodoToProto(src model.Todo) *todov1.Todo {
	var v *timestamppb.Timestamp
	if v2 := src.CompletedAt; v2 != nil {
		v = timestamppb.New((*v2).Time())
	}
	return &todov1.Todo{
		CompletedAt: v,
	}
}

// TodoFromProto maps *todov1.Todo to model.Todo.
func TodoFromProto(src *todov1.Todo) (model.Todo, error) {
	if src == nil {
		return model.Todo{}, nil
	}
	var v *model.CompletedAt
	if v2 := src.GetCompletedAt(); v2 != nil {
		v3, err := model.NewCompletedAt(v2.AsTime())
		if err != nil {
			return model.Todo{}, fmt.Errorf("map model.Todo.CompletedAt: %w", err)
		}
		v = &v3
	}
	return model.Todo{
		CompletedAt: v,
	}, nil
}

A rule can reject the value, so a function making one has to return an error, and the rule's error is wrapped as any other is. A zero on the source side is checked like any other value rather than taken for the zero value type, which stands for one never given: map an optional field to a pointer, which stays nil. A value whose type stays the same is copied, not checked again, *model.CompletedAt from model.CompletedAt included.

A value type is known by the shape kanna-value gives it, not by reading its //kanna:value, as kanna-mapper may see only the types of the package declaring it: a struct with one unexported field, a method named after the field's type returning it (Time for a time.Time, String for a string), and NewX(T) (X, error) beside it. A function listed for the pair is called instead, as for any other type. An unexported value type of another package cannot be made here, as its constructor is unexported too, so the field is reported as one that cannot be mapped. So is a value type whose package this one sees only through another, whose export data carries the type without its constructor; list the constructor, such as deep.NewLevel, which brings the package in.

Enums

An enum of kanna-value, a string type whose values are its constants, maps to and from an enum of protobuf by the names of their constants: the Go constant's past its type's, Placed in OrderStatusPlaced, and the protobuf value's past its enum's, PLACED in ORDER_STATUS_PLACED, compared regardless of case and underscores. A function is written for each direction:

// orderStatusToOrderv1 maps model.OrderStatus to orderv1.OrderStatus.
func orderStatusToOrderv1(src model.OrderStatus) orderv1.OrderStatus {
	switch src {
	case model.OrderStatusPlaced:
		return orderv1.OrderStatus_ORDER_STATUS_PLACED
	case model.OrderStatusPaid:
		return orderv1.OrderStatus_ORDER_STATUS_PAID
	}
	return orderv1.OrderStatus_ORDER_STATUS_UNSPECIFIED
}

// orderStatusToModel maps orderv1.OrderStatus to model.OrderStatus.
func orderStatusToModel(src orderv1.OrderStatus) (model.OrderStatus, error) {
	switch src {
	case orderv1.OrderStatus_ORDER_STATUS_UNSPECIFIED:
		return "", nil
	case orderv1.OrderStatus_ORDER_STATUS_PLACED:
		return model.OrderStatusPlaced, nil
	case orderv1.OrderStatus_ORDER_STATUS_PAID:
		return model.OrderStatusPaid, nil
	}
	return "", fmt.Errorf("model.OrderStatus has no value for %v", src)
}

Protobuf's 0 and the domain's empty string both stand for no value, and map to each other, unless a constant matches the 0 value by name. A value protobuf has and the domain does not, or a number nobody declared, is an error coming back, so a function mapping an enum from protobuf has to return one. A constant the domain has and protobuf does not stops generation instead:

./mapper.go:15:2: error: OrderToProto: Status: model.OrderStatusRefunded has no counterpart in orderv1.OrderStatus
  add ORDER_STATUS_REFUNDED to it, or list a func(model.OrderStatus) orderv1.OrderStatus in a //kanna:converter list

A plain string, such as a field a message carries as string, becomes an enum of kanna-value through its constructor, NewOrderStatus, which rejects a string that is none of the constants, the empty one included: map an optional field to a pointer. The other way, the enum is a string already, and converts as it is.

An enum of protobuf is known by the shape protoc-gen-go gives it, an int32 with an Enum method, and an enum of kanna-value by a string type with constants beside it, as kanna-mapper may see only the types of the package declaring it. A function listed for the pair is called instead, as for any other type.

Protobuf oneofs

A oneof arrives as an interface-typed field whose implementations are single-field wrapper structs, which is how protoc-gen-go writes them. On the domain side it is a set of pointer fields of which at most one is set:

type Change struct { // domain
	Version    int64
	Membership *Membership
	Category   *Category
}

type Change struct { // wire, as protoc-gen-go wrote it
	Version int64
	Kind    isChange_Kind // *Change_Membership | *Change_Category
}

Each arm is matched to a domain field by the name of its wrapper's field, with the rules above, and converted the way that field would be. Coming back, a type switch on the oneof fills the field for the arm that is there. Going out, the first set field in the oneof's declaration order fills the arm; none set leaves the oneof nil:

if v := src.Membership; v != nil {
	dst.Kind = &syncv1.Change_Membership{Membership: membershipToSyncv1(*v)}
} else if v2 := src.Category; v2 != nil {
	dst.Kind = &syncv1.Change_Category{Category: categoryToSyncv1(*v2)}
}

The domain field has to be a pointer, slice, or map, because that is what says whether the arm is set. An arm with no domain field is an error like any other destination field: leave it out with //kanna:map Membership=-, or the whole oneof with //kanna:map Kind=-.

Sum types

Pointer fields cannot keep the domain from setting two arms at once, and going out, the first one set wins without a word. A domain that wants one and only one by its types holds a sum type instead: an interface with an unexported method, which no other package can implement, and the exported types of its package that implement it by value:

type Payment interface{ isPayment() }

type Bank struct{ Account string }
type Cash struct{ Office string }

func (Bank) isPayment() {}
func (Cash) isPayment() {}

type Employee struct {
	Payment Payment // wire: oneof payment { Bank bank = 1; Cash cash = 2; }
}

A field of a sum type maps to a oneof field of the same name, each variant to the arm whose field, or the message it holds, is named as the variant is, regardless of case and underscores. The value converts as any other would, here through functions written for Bank and Cash:

switch v := src.Payment.(type) {
case model.Bank:
	dst.Payment = &employeev1.Employee_Bank{Bank: bankToEmployeev1(v)}
case *model.Bank:
	if v != nil {
		dst.Payment = &employeev1.Employee_Bank{Bank: bankToEmployeev1(*v)}
	}
case model.Cash:
	dst.Payment = &employeev1.Employee_Cash{Cash: cashToEmployeev1(v)}
case *model.Cash:
	if v != nil {
		dst.Payment = &employeev1.Employee_Cash{Cash: cashToEmployeev1(*v)}
	}
}

A variant implements the sum type by value, so a pointer to it does too: it goes out as the variant would, and a nil one as none. Coming back, a type switch on the oneof sets the variant of the arm that is there, by value. A nil sum type and a oneof with no arm set are each other. A variant with no arm stops generation going out, as an arm with no variant does coming back.

Types generated with protobuf's Opaque API, which have no exported fields, are not supported.

Flags

kanna-mapper [flags] [packages]

The packages default to the one in the current directory, so //go:generate go tool kanna-mapper needs no argument. A run that succeeds prints nothing; one that finds no //kanna:mapper interface at all warns, naming the packages by import path. A package whose interfaces have all lost //kanna:mapper has its mapper_gen.go removed, unless code in the packages the run loads still calls its functions; a file of that name without the generated-code header is never touched. The go generate that removes it ends with open ...: no such file or directory, from go generate itself, which lists a package's files before it runs the generators; the file is gone as it should be (golang/go#36068).

Example

example maps a domain aggregate onto protobuf-shaped wire types and back, covering each way a field can be handled: converted with uuid.Parse and (*timestamppb.Timestamp).AsTime listed as they are, through a list shared between packages, renamed with a tag, excluded with map:"-" and with //kanna:map, mapped through a function written for a nested pair, and folded into a oneof. CI regenerates it and fails if the output would change.

Development

make test       # unit tests
make lint       # golangci-lint
make example    # regenerate the example, then build and run it

License

MIT