Caching Slow Database Queries 
This package is used to cache the results of slow database queries in memory or Redis.
It can be used to cache any form of data. A Redis and in-memory storage driver is provided.
See Article for further details including a tutorial.
The package is production ready and the API is stable. A variant of this package has been used in production for over 1.5 years.
⭐ the project to show your appreciation.
Installation
go get -u github.com/rocketlaunchr/remember-go
Create a Key
Let’s assume the query’s argument is an arbitrary search term and a page number for pagination.
CreateKeyStruct
CreateKeyStruct can generate a JSON based key by providing a struct.
type Key struct {
Search string
Page int `json:"page"`
}
var key string = remember.CreateKeyStruct(Key{"golang", 2})
CreateKey
CreateKey provides more flexibility to generate keys:
key := remember.CreateKey(false, "-", "search-x-y", "search", "golang", 2)
// Key will be "search-golang-2"
Initialize the Storage Driver
In-Memory
var ms = memory.NewMemoryStore(10 * time.Minute)
Redis
The Redis storage driver relies on Gary Burd’s excellent Redis client library.
var rs = red.NewRedisStore(&redis.Pool{
Dial: func() (redis.Conn, error) {
return redis.Dial("tcp", "localhost:6379")
},
})
Memcached
An experimental (and untested) memcached driver is provided.
It relies on Brad Fitzpatrick's memcache driver.
Ristretto
DGraph's Ristretto is a fast, fixed size, in-memory cache with a dual focus on throughput and hit ratio performance.
The API is potentially still in flux so no backward compatibility guarantee is provided for this driver.
Create a SlowRetrieve Function
The package initially checks if data exists in the cache. If it doesn’t, then it elegantly fetches the data directly from the database by calling the SlowRetrieve function. It then saves the data into the cache so that next time it doesn’t have to refetch it from the database.
type Result struct {
Title string
}
slowQuery := func(ctx context.Context) (interface{}, error) {
results := []Result{}
stmt := `
SELECT title
FROM books
WHERE title LIKE ?
ORDER BY title
LIMIT ?, 20
`
rows, err := db.QueryContext(ctx, stmt, search, (page-1)*20)
if err != nil {
return nil, err
}
defer rows.Close()
for rows.Next() {
var title string
if err := rows.Scan(&title); err != nil {
return nil, err
}
results = append(results, Result{title})
}
return results, nil
}
Usage
import (
"github.com/gomodule/redigo/redis"
"github.com/rocketlaunchr/remember-go"
"github.com/rocketlaunchr/remember-go/memory"
red "github.com/rocketlaunchr/remember-go/redis"
)
key := remember.CreateKeyStruct(Key{"golang", 2})
exp := 10*time.Minute
results, found, err := remember.Cache(ctx, ms, key, exp, slowQuery, remember.Options{GobRegister: false})
return results.([]Result) // Type assert in order to use
Gob Register Errors
The Redis storage driver stores the data in a gob encoded form. You have to register with the gob package the data type returned by the SlowRetrieve function. It can be done inside a func init(). Alternatively, you can set the GobRegister option to true. This will slightly impact concurrency performance however.
Other useful packages
- dataframe-go - Statistics and data manipulation
- dbq - Zero boilerplate database operations for Go
- igo - A Go transpiler with cool new syntax such as
fordefer(defer for for-loops) - mysql-go - Properly cancel slow MySQL queries
- react - Build front end applications using Go
Legal Information
The license is a modified MIT license. Refer to LICENSE file for more details.
© 2019 PJ Engineering and Business Solutions Pty. Ltd.
Final Notes
Feel free to enhance features by issuing pull-requests.