Skip to content

Commit

Permalink
adding logging module (#2)
Browse files Browse the repository at this point in the history
* adding logging module

* removed old references

* go mod cleanup

* api cleanup (removed reference to /gen/go mostly)

* dependency cleanup for pkg/

* fixed healthz imports

* more go mod stuff

* upgraded dctl deps to resolve incompatible module

* removed replace and fixed paths
  • Loading branch information
jgkawell authored Jan 21, 2024
1 parent 346fa63 commit b809853
Show file tree
Hide file tree
Showing 68 changed files with 4,199 additions and 3,519 deletions.
6 changes: 3 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,13 @@ clean:
blueprint: clean_blueprint blueprint_1 blueprint_2 blueprint_3

blueprint_1:
BOOTSTRAP_RAFT=true RAFT_IP=localhost RAFT_PORT=1111 SERVER_PORT=2221 RAFT_NODE_ID="node_1" go run services/blueprint/main.go
BOOTSTRAP_RAFT=true RAFT_IP=localhost RAFT_PORT=1111 SERVER_PORT=2221 RAFT_NODE_ID="node_1" go run services/core/blueprint/main.go

blueprint_2:
RAFT_PORT=1112 RAFT_IP=localhost SERVER_PORT=2222 RAFT_NODE_ID="node_2" go run services/blueprint/main.go
RAFT_PORT=1112 RAFT_IP=localhost SERVER_PORT=2222 RAFT_NODE_ID="node_2" go run services/core/blueprint/main.go

blueprint_3:
RAFT_PORT=1113 SERVER_PORT=2223 RAFT_IP=localhost RAFT_NODE_ID="node_3" go run services/blueprint/main.go
RAFT_PORT=1113 SERVER_PORT=2223 RAFT_IP=localhost RAFT_NODE_ID="node_3" go run services/core/blueprint/main.go

blueprint_register_leader:
go run pkg/blueprint-client/main.go
Expand Down
70 changes: 6 additions & 64 deletions api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,50 +3,19 @@ To serve as the home for type definitions, rpc service's, model's, validation of
All of the above are declared in [protocol buffer's]() and compiled using [buf]() inside of a docker container
to reduce protoc compiler, and compiler plugin environment dependencies.

# service-clients-internal

Contains all Fluid Truck defined protobuf files.

## Generating files for local testing

### Using `fctl`
### Using `dctl`

The easiest way to generate protos is using [`fctl`](https://github.com/fluidtruck/fctl):
The easiest way to generate protos is using [`dctl`](../tools/dctl/):

```shell
# you only need to run this the first time
fctl api init
fctl api build
dctl api init
dctl api build
```

Any time you want to regenerate your protos just run `fctl api build` again.

### Manually

`make install` needs to be run only once to install required go modules.

There are two targets to compiling code locally:

- `make go` will compile all protobuf files into go and place them in the `gen/go` directory.
- `make go-diff` will compile only added & changed protobuf files into go and place them in the `gen/go` directory.

To use these generated files locally use this `replace` in the project's go.mod:

`replace github.com/fluidtruck/service-clients-internal => ../protobufs/gen/go`

git will ignore all generated files.

The manual work flow becomes:

1. pull main & checkout new branch
2. `make go`
3. add `replace` to project's go.mod
4. add and modify protos
5. `make go-diff`

If a `.proto` is deleted, `make go` will be required.

Using `make go` exclusively works but `make go-diff` will be faster for normal development.
Any time you want to regenerate your protos just run `dctl api build` again.

## Using locally generated files

Expand All @@ -55,34 +24,7 @@ To use the generated files locally, you will need to add a `replace` to the proj
```go.mod
...
replace github.com/fluidtruck/service-clients-internal => ../protobufs/gen/go
replace github.com/steady-bytes/draft/api => ../../../api/gen/go
...
```

## Using a gRPC GUI

### Postman

To use Postman, you can simply use gRPC reflection to "import" the proto defintions. There are two ways to do this depending on if you are running the [proxy](https://github.com/fluidtruck/proxy) or not.

If you are using the proxy, for the URL in Postman you can set it to `localhost:10000` + the path of *any* gRPC service that is implemented by the microservice you wish to test. For example, if you are testing the [users](https://github.com/fluidtruck/users) microservice, it implements a few different gRPC services including `users.v1.PointsOfContactService` and `users.v1.UserPermissionsService`. To use reflection, simply choose *any* of the implemented services and use that as your URL. For the `users` microservice example we can use `localhost:10000/users.v1.UserPermissionsService` and this will pull in *all* gRPC services implemented by the `users` microservice.

If you are not using the proxy, for the URL in Postman you will need to find the exact port of the service you are testing. When you run the service it will log this out to the console. For example, if you are testing a service that is running on port `54879`, you will need to set your Postman URL to `localhost:54879`. This will also pull in *all* gRPC services implemented by the microservice.

### BloomRPC

To use BloomRPC, you will need to import the top-level of the `protobufs` repo as well as any vendor submodules that are used in the proto files you will be testing. The imports should look like this:

```text
<USER PATH TO GIT REPOS>/protobufs
<USER PATH TO GIT REPOS>/protobufs/vendor/googleapis
<USER PATH TO GIT REPOS>/protobufs/vendor/grpc-gateway
<USER PATH TO GIT REPOS>/protobufs/vendor/protoc-gen-gotag
<USER PATH TO GIT REPOS>/protobufs/vendor/protoc-gen-validate
...
```

Then you can import individual proto files into the GUI and test them.

NOTE: if you did not originally clone the repository with git submodules, you will need to run `git submodule update --init --recursive` to pull in the submodules.
2 changes: 1 addition & 1 deletion api/consensus/fsm/v1/fsm.proto
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ syntax = "proto3";

package consensus.fsm.v1;

option go_package = "github.com/steady-bytes/draft/api/gen/go/consensus/fsm/v1";
option go_package = "github.com/steady-bytes/draft/api/consensus/fsm/v1";

import "google/protobuf/any.proto";

Expand Down
2 changes: 1 addition & 1 deletion api/consensus/raft/v1/service.proto
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ syntax = "proto3";

package consensus.raft.v1;

option go_package = "github.com/steady-bytes/draft/api/gen/go/consensus/raft/v1";
option go_package = "github.com/steady-bytes/draft/api/consensus/raft/v1";

service RaftService {
// Join the raft cluster
Expand Down
8 changes: 4 additions & 4 deletions api/gen/go/consensus/fsm/v1/fsm.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 6 additions & 7 deletions api/gen/go/consensus/raft/v1/service.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api/gen/go/consensus/raft/v1/v1connect/service.connect.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 6 additions & 1 deletion api/gen/go/go.mod
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
module github.com/steady-bytes/draft/api/gen/go
module github.com/steady-bytes/draft/api

go 1.21.5

require (
connectrpc.com/connect v1.14.0
google.golang.org/protobuf v1.32.0
)

require (
github.com/google/go-cmp v0.6.0 // indirect
golang.org/x/net v0.19.0 // indirect
)
9 changes: 3 additions & 6 deletions api/gen/go/go.sum
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
connectrpc.com/connect v1.14.0 h1:PDS+J7uoz5Oui2VEOMcfz6Qft7opQM9hPiKvtGC01pA=
connectrpc.com/connect v1.14.0/go.mod h1:uoAq5bmhhn43TwhaKdGKN/bZcGtzPW1v+ngDTn5u+8s=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k=
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
golang.org/x/net v0.19.0 h1:zTwKpTd2XuCqf8huc7Fo2iSy+4RHPd10s4KzeTnVr1c=
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
google.golang.org/protobuf v1.32.0 h1:pPC6BG5ex8PDFnkbrGU3EixyhKcQ2aDuBS36lqK/C7I=
google.golang.org/protobuf v1.32.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
24 changes: 12 additions & 12 deletions api/gen/go/registry/key_value/v1/service.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit b809853

Please sign in to comment.