Example of how to integrate a client for another gRPC service

[auspicacious]

Feature Request

Motivation

I appreciate the effort that has been put into the examples and documentation for this project. However, I could not easily find an example of what to do when calling another gRPC service from inside a server.

Specifically, I have the protobuf service definitions for another service, and I need to generate a client from them and call it. There are namespace conflicts between it and the protobuf definition of my own service.

Right now, to work around this, I have created a subcrate that builds the client protobuf. Is there a better way to do this? The meaning of some of the functions in build.rs is often opaque, and it is very likely that I have missed something. Thank you.

[dfawley]

There are namespace conflicts between it and the protobuf definition of my own service.

Why is that the case? That seems like a problem in and of itself. Do you own both service definitions? You should be properly namespacing your proto files (and ideally so should the service you are trying to call, but sometimes you can't control that):

https://buf.build/docs/reference/protobuf-files-and-packages/#packages:~:text=A%20good%20rule%20of%20thumb%20is%20to%20use%20at%20least%20three%20components%3A

A good rule of thumb is to use at least three components:

• The first component should be the name of the author or owner, typically a company or organization name
  ...
• The second component should convey the purpose of the Protobuf schema or the purpose or name of the system or product that it defines
  ...
• The last component should be a version number.

[auspicacious]

@dfawley I am aware of Buf's guidelines and use buf lint in my project. In a more general sense, I am aware of the value of namespacing; more than a decade ago, I was responsible for writing and maintaining the RESTful service standards for a mid-sized company.

I could get into how difficult gRPC makes this by requiring full path references to entities everywhere -- my schemas are already painful to read -- but more to the point: in a properly designed gRPC implementation, it should be impossible for namespace collisions to occur. There is no reason to need to rely on everyone following the Buf guidelines and choosing a unique path within a global namespace.

Even if I didn't have conflicts that I knew about, I would still want to generate each client separately. Maybe they need different settings or a different version. Maybe I've missed something. So I think my question is still valid.

[dfawley]

Could you provide a more concrete example of the problem you're running into? I can't tell from the description if you have duplicate proto message names, service names, or what kinds of errors you're encountering when the namespaces collide. Ideally you could just pull the generated code into separate mods within your crate, but there might be limitations there preventing it, or flags you need to set (or flags we need to add?) when running protoc.

[auspicacious]

@dfawley Thanks -- I can't share the actual code right now (hoping to open-source it at some point) so I'll share what I think are the relevant parts with a little editing.

I have two schema repositories that I'm pulling in as submodules, one so that I can generate the server and one so that I can generate a client that the server needs. Both of them have a service called services.admin.v1.AdminService.

With a build.rs like this, I get errors (and, honestly, I have no idea what a "file descriptor set" is here, but it's in the Hello World example):

    tonic_prost_build::configure()
        .file_descriptor_set_path(out_dir.join("my_service_api.bin"))
        .compile_protos(
            &[
                "my_service_api/src/services/admin/v1/admin_service.proto",
                "other_service_api/src/services/admin/v1/admin_service.proto",
            ],
            &["my_service_api/src", "other_service_api/src"],
        )
        .unwrap_or_else(|e| panic!("Failed to compile protos {e:?}"));

I am short on time to do it now, but I can revert to this setup and generate an error later if you ask.

If I invoke tonic_prost_build::...compile_protos() twice, once for each service, the second call overwrites the first call.

Inside my application, I have a module which puts everything into the structure I want:

#[allow(clippy::all, clippy::pedantic)]
pub mod my_service {
    pub mod services {
        pub mod admin {
            pub mod v1 {
                tonic::include_proto!("services.admin.v1");
            }
        }
    }
}

As far as I can remember, it was not possible for me to just, for example, include_proto!("services") and automatically get the proper structure, although I may have missed something, so I have created this nested structure by hand. I also have a few other top-level items I need (types, units, etc.) that are shared across service definitions.

The client, of course, gets a similar module.