Introduction#

What is gRPC?#

gRPC is Google’s open-source RPC framework, that takes advantage of Protocol Buffers (protobufs) to provide low-latency HTTP/2-based APIs between microservices and clients.

gRPC has over 40k stars on GitHub for its popular C-based version, and has about 19.7k stars for its Go-based version. gRPC also supports C++, Java, Python, and many more popular languages.

What are RPCs?#

Remote Procedure Calls (RPCs) are simply a defined way to call a procedure (a.k.a. function or method) from another program over the internet. RPCs shine when two programs that were written in different languages need to communicate with each other.

What are microservices?#

A microservice is a type of small, independent server that communicates with other small servers to run an application. Microservices greatly benefit from RPCs, as they provide low-latency, language-independent interprocess communication (IPC).

In microservice architecture, microservices help split the work of an application into loosely coupled services. Each of these services are in charge of completing specific tasks, and can be built and deployed independently. Additionally, they can be built in any language, allowing the use of the best language for the job.

When used jointly with orchestration systems like Kubernetes, microservices can help build at scale efficiently.

Why should I use gRPC for building microservices?#

gRPC’s speed is one of its defining factors due to its use of protobuf for serializing data into a compressed binary format to transfer over HTTP/2. gRPC also takes advantage of the protobuf compiler to generate code for most popular languages using common .proto files.

In addition to its performance and easy compilation, gRPC also has extensive documentation and is constantly maintained by Google and the open source community on GitHub for new features and bug fixes.

Prerequisites#

While not required, gRPC services typically use the Protocol Buffers (protobuf) Compiler (Installation Instructions).

Important ❗️: This guide will assume you are using protobuf. Protobuf helps avoid manual creation of necessary boilerplate for gRPC microservices.

Tutorial#

Note 💡: This guide uses the Official gRPC Go QuickStart guide as a reference point, but intends to serve as a full walk-through of the gRPC setup process for Go.

1. Install gRPC protoc extensions#

Use the Installation Instructions to download gRPC for your language(s) of choice. For this tutorial, we will be using Go for both our server and client.

Note 💡: gRPC is supported by many languages, including Go, Python, Java, C++, and more. While this tutorial is for Go implementation, you can use gRPC with a server and client of any language.

Want more? 🤔 Please feel free to Contact Me if you would like to see more gRPC tutorials like these for other languages, like Java or Python.

For Go, you require the following:

# Extensions for protoc compiler (global-level installation)
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

2. Install gRPC package#

We will now install the gRPC Go package

# Create a folder for your project
mkdir grpc-project
cd grpc-project

# Intialize a go module and install gRPC
go mod init grpc-project

# Get gRPC package (project-level installation)
go get -u google.golang.org/grpc

3. Define the service using proto3#

Create the folders proto/ and (inside of proto/) my_service/ and create a file inside of it called my_service.proto.

// File:
// grpc-project/proto/my_service/my_service.proto
// ----------------------------------------------
syntax = "proto3";

package my_service;

option go_package = "grpc-project/proto/my_service";

service MyService {
    // Ping is a remote procedure call that will 
    // return the message "Pong"
    rpc Ping(PingRequest) returns (PingResponse) {}

    // PingExternal is similar to Ping(), but will 
    // Dial a server with the Ping()
    // remote procedure call and return its message
    rpc PingExternal(PingRequest) returns (PingResponse) {}
}

// Define PingRequest and PingResponse
message PingRequest {
}

message PingResponse {
    string message = 1;
}

4. Compile `my_service.proto` using protoc with `protoc-gen-go` and `protoc-gen-go-grpc` extensions#

cd proto/my_service
protoc --go_out=. \
    --go_opt=paths=source_relative \
    --go-grpc_out=. \
    --go-grpc_opt=paths=source_relative \
    my_service.proto

We are using the following flags for protoc:

Flag	Description
–go_out	Root path for outputting generated Go code for `message` types
–go_opt=paths	Option to specify how the output file paths are constructed from the root path `go_out`
–go-grpc_out	Root path for outputting generated Go code for gRPC `service` Go interfaces
–go-grpc_opt=paths	Option to specify how the output file paths for gRPC code are constructed from the root path `go-grpc_out`

After running this command, you should see a few generated files:

.
└── proto
    └── my_service
        ├── my_service.pb.go
        ├── my_service.proto
        └── my_service_grpc.pb.go

We should see in the my_service folder:

my_service.pb.go: This file contains the message types we defined in my_service.proto
my_service_grpc.pb.go: This file contains the service interface we defined in my_service.proto

Now, we are ready to dive into implementing the our gRPC service!

5. Setup a gRPC server#

Go back to the main directory (grpc-project/) and let’s create a server/ directory:

# Navigate to `grpc-project/`, assuming
# we are still in `grpc-project/proto/my_service`
cd ../../

# Create the `server/` folder
mkdir server

In that folder, let’s create the Go file called my_service_server.go.

// File:
// grpc-project/server/my_service_server.go
// ----------------------------------------
package main

import (
    "flag"
    "fmt"
    "log"
    "net"

    "google.golang.org/grpc"

    // TODO: Import our protobuf-generated files
    // =========================================
)

var (
    port = flag.String("port", "8000", "port to listen on")
)

// TODO: Implement our MyService gRPC server
// ==========================================

func main() {
    // Parse --port=<value> or
    // --port <value> from the terminal
    flag.Parse()

    // Create a listener on address 127.0.0.1:<port>
    // (127.0.0.1 can be ommited)
    address := fmt.Sprintf(":%s", *port)
    listener, err := net.Listen("tcp", address)
    if err != nil {
        log.Fatalf("failed to listen: %v", err)
    }

    // Create a new gRPC server
    s := grpc.NewServer()

    // TODO: Register our MyService gRPC server
    // =========================================

    // Serve the gRPC server with the listener
    log.Printf("Server listening on port %s", *port)
    if err := s.Serve(listener); err != nil {
        log.Fatalf("failed to serve: %v", err)
    }
}

The above is some sample boilerplate to start up a gRPC server. At the moment, the server isn’t running a service, so let’s implement ours!

6. Implement a `MyService` gRPC server#

Since we used protoc to generate a few Go files for us, most of the work has been done for us. We can import these files using the module name we created in step 2.

// File:
// grpc-project/server/my_service_server.go
// ----------------------------------------
package main

import (
    // ...

    // TODO: Import our protobuf-generated files
    // =========================================
    pb "grpc-project/proto/my_service"
)

// ...

This will import all of the necessary message structs and service interfaces we need to implement and register MyService.

To implement MyService, we will create a struct that embeds the pb.UnimplementedMyServiceServer struct.

// File:
// grpc-project/server/my_service_server.go
// ----------------------------------------
// ...

// TODO: Implement our MyService gRPC server
// ==========================================
type server struct {
    pb.UnimplementedMyServiceServer
}

// ...

Note 💡: You may see other examples where the server struct instead directly embeds the interface pb.MyServiceServer, or do not embed anything at all.

All of these methods work due to how interfaces work in Go, but the official gRPC docs recommend to embed the unimplemented struct to get compiler errors if not all RPC’s are implemented by a custom server struct.

Once we have a struct, we can now implement the server inteface generated by protobuf:

// From:
// grpc-project/proto/my_service/my_service_grpc.pb.go
// ===================================================
type MyServiceServer interface {
    Ping(context.Context, *PingRequest) (*PingResponse, error)
    PingExternal(context.Context, *PingRequest) (*PingResponse, error)
    mustEmbedUnimplementedMyServiceServer()
}

Let’s implement it for our server struct!

// File:
// grpc-project/server/my_service_server.go
// ----------------------------------------

import (
    // Import context
    "context"
    // ...
    "google.golang.org/grpc/credentials/insecure"
)

var (
    // ...
    externalAddress = flag.String("external_address", "127.0.0.1:8000", "external address to ping")
)

// TODO: Implement our MyService gRPC server
// ==========================================
type server struct {
    pb.UnimplementedMyServiceServer
}

// Ping - sends the message "Pong" back to any pings
func (s *server) Ping(ctx context.Context, req *pb.PingRequest) (*pb.PingResponse, error) {
    log.Println("Received PingRequest from Ping")
    return &pb.PingResponse{
        Message: "Pong",
    }, nil
}

// PingExternal - relays pings to a defined external address (by default pings 127.0.0.1:8000)
func (s *server) PingExternal(ctx context.Context, req *pb.PingRequest) (*pb.PingResponse, error) {
    log.Println("Received PingRequest from PingExternal")
    
    // Note: insecure credentials should only be used for development.
    // Use appropriate credentials in production.
    conn, err := grpc.NewClient(*externalAddress, grpc.WithTransportCredentials(insecure.NewCredentials()))
    if err != nil {
        log.Printf("PingExternal failed to connect to external server %v", *externalAddress)
        return nil, err
    }

    defer conn.Close()
    client := pb.NewMyServiceClient(conn)

    resp, err := client.Ping(ctx, req)
    if err != nil {
        log.Printf("ExternalPing failed on server-side: %v", err)
        return nil, err
    }
    // return resp, nil
    return &pb.PingResponse{
        Message: fmt.Sprintf("External says: %v", resp.Message),
    }, nil
}

// ...

7. Register `MyService` on the gRPC server.#

Now, we are ready to register MyService and run our server:

// File:
// grpc-project/server/my_service_server.go
// ----------------------------------------

// ...

func main() {
    // ...

    // Create a new gRPC server
    s := grpc.NewServer()

    // TODO: Register our MyService gRPC server
    // =========================================
    pb.RegisterMyServiceServer(s, &server{})

    // ...
}

8. Setup a `MyService` gRPC client#

We have now created a MyService server in gRPC! Next, we must create a client that calls the RPCs we have created.

Let’s go back to the root folder and let’s create a client/ directory:

# Navigate to `grpc-project/`, assuming
# we are still in `grpc-project/server/`
cd ../

# Create the `client/ folder
mkdir client

In that folder, let’s create a Go file called my_service_client.go.

// File:
// grpc-project/client/my_service_client.go
// ----------------------------------------

package main

import (
    "flag"
    "log"

    "google.golang.org/grpc"
    "google.golang.org/grpc/credentials/insecure"

    // TODO: Import our protobuf-generated files
    // =========================================
)

var (
    address = flag.String("address", "127.0.0.1:8000", "address to connect to")
)

func main() {
    conn, err := grpc.NewClient(*address, grpc.WithTransportCredentials(insecure.NewCredentials()))
    if err != nil {
        log.Fatalf("Failed to connect to %v", address)
    }

    defer conn.Close()

    // TODO: Call Ping and PingExternal RPCs
    // and output their response message
    // =====================================
}

Note 💡: Older examples for gRPC clients may use grpc.Dial(), but it is now deprecated as of v1.63.0.

The grpc.Dial() method will still be supported for versions v1.x, but new code should now use grpc.NewClient() to maintain support when v2.x is released.

9. Import the generated `MyService` gRPC client#

Since we used protobuf, the gRPC client code is generated for us. Now, we just need to import the code into our client.

// File:
// grpc-project/client/my_service_client.go
// ----------------------------------------

package main;

import (
    "context"
    // ...

    pb "grpc-project/proto/my_service"
)

func main() {
    // ...

    // TODO: Call Ping and PingExternal RPCs
    // and output their response message
    // =====================================
    client := pb.NewMyServiceClient(conn)

    resp, err := client.Ping(context.Background(), &pb.PingRequest{})
    if err != nil {
        log.Fatalf("failed to Ping: %v", err)
    }

    log.Printf("Received PingResponse: %+v", resp)

    resp, err = client.PingExternal(context.Background(), &pb.PingRequest{})
    if err != nil {
        log.Fatalf("failed to PingExternal: %v", err)
    }
    
    log.Printf("Received PingResponse: %+v", resp)
}

10. Run the server and client#

Now that we have all of our code setup, we can now build and run our server:

# This assumes that we are in the root directory
# `grpc-project/`. (If not, please `cd` there first)
#
# cd grpc-project

# Create a bin/ folder for the executables to be stored there
mkdir bin

# Build the Go server
cd server/
go build -o ../bin/my_service_server .

# Navigate to the bin folder
cd ../bin

# Run the server on Port 8000 (in MacOS/Linux)
./my_service_server --port 8000 --external_address "127.0.0.1:8000"

# or in Windows
my_service_server.exe --port 8000 --external_address "127.0.0.1:8000"

After running, you should see the following logs:

20XX/XX/XX HH:MM:SS Server listening on port 8000

Now that we have a working server, we can now open a new terminal to run the client.

# Navigate the new terminal to the grpc-project
# directory
cd grpc-project

# Navigate to the client code
cd client

# Build the Go client
go build -o ../bin/my_service_client .

# Navigate to the bin folder
cd ../bin

# Run the server with the address 127.0.0.1:8000
# MacOS/Linux:
./my_service_client --address "127.0.0.1:8000"

# or in Windows
my_service_client.exe --address "127.0.0.1:8000"

After running, you should get the following results:

20XX/XX/XX HH:MM:SS Received Ping response: message:"Pong"
20XX/XX/XX HH:MM:SS Received Ping response: message:"External says: Pong"

Congratulations, you’ve created your first microservice using gRPC!

Extra tips#

Other clients for gRPC servers#

In addition to building your own client, you can also use other programs to make requests to a server quickly.

Before being able to do so, you must enable reflection on the server so that these clients can know what methods are available without providing a proto file:

// File:
// grpc-project/server/my_service_server.go
// ----------------------------------------
package main

import (
    // ...
    "google.golang.org/grpc/reflection"
)

func main() {
    // ...

    // TODO: Register our MyService gRPC server
    // =========================================
    pb.RegisterMyServiceServer(s, &server{})
    reflection.Register(s)  // Add to enable reflection for easy proto access to clients
    
    // ...
}

Once enabled, you can use any of the following:

grpc_cli (Build Instructions)

Important ❗️: Only choose grpc_cli if you are comfortable with building from source (or have access to brew on MacOS/Linux). Otherwise, I would not recommend beginners to build it from source.

If you are using brew, it’s as simple as: brew install grpc_cli

Example:

# Run rpc Ping()
grpc_cli call 127.0.0.1:8000 my_service.MyService.Ping ""
# connecting to 127.0.0.1:8000
# message: "Pong"
# Rpc succeeded with OK status

# Run rpc PingExternal()
grpc_cli call 127.0.0.1:8000 my_service.MyService.Ping ""
# connecting to 127.0.0.1:8000
# message: "External says: Pong"
# Rpc succeeded with OK status

Postman Desktop (Lightweight API client) (Installation Guide)

Example:

Postman Screenshot

Important ❗️: When Using Postman, please select Reflection under the Service Definition section in the right dropdown next to the url field.

Next Steps#

Don’t stop learning about gRPC here. You can explore its documentation here and find out what gRPC offers.

If you would like me to write more blogs about gRPC, feel free to contact me and let me know you would like to see more of these.

Happy learning!

Introduction to Remote Procedure Calls with gRPC in Go