Smithy4s integration by Baccata · Pull Request #58 · neandertech/jsonrpclib

Baccata · 2023-02-21T11:28:55Z

Adds a smithy4s module, allowing to automatically derive endpoints and stubs from smithy definitions

Related PRs:

Bump jsonrpclib langoustine#210

Baccata · 2023-02-21T15:11:35Z

@armanbilge, good progress so far.

The examples currently hang at the end, but otherwise work as expected : there are two example applications involved, a client and a server. The build is wired in such a way that the client receives through the forkEnv the path to an assembly jar of the server when running mill -i examples.smithyClient.run. The client starts the server in a subprocess, wires the various streams and pipes in the relevant manner, and start talking to the server.

A test.smithy file defines the interfaces of both parties. The code is generated, and the new ClientStub and ServerEndpoints (temporary names) constructs are used to get the smithy4s code work with the jsonrpclib code.

Regarding the hanging : I presume the ChildProcess I've copied and adapted from Davenverse is not sound. I wonder whether it's a similar thing to what led you to run fs2.io.stdin on a dedicated pool ... I could use some help to understand how to troubleshoot it.

I mean, it's entirely possible/likely that I've messed up something else, but that subprocess stdin/stdout based communication is the only I/O code involved in the example ...

Baccata · 2023-02-21T15:20:10Z

@@ -0,0 +1,23 @@
+$version: "2.0"
+
+namespace jsonrpclib


this smithy file contains the protocol definition. It kind of echoes a question that @ckipp01 had on my BSP PR about where would be a good "central" for protocols.

I suppose DisneyStreaming could host it in https://github.com/disneystreaming/alloy/, but I'm a bit wary about putting it there and have the BSP depend on it ($work project etc etc).

armanbilge · 2023-02-21T16:28:02Z

+        def stdout: fs2.Stream[F, Byte] = fs2.io
+          .readInputStream[F](Sync[F].interruptible(p.getInputStream()), chunkSize = readBufferSize)
+
+        def stderr: fs2.Stream[F, Byte] = fs2.io
+          .readInputStream[F](Sync[F].blocking(p.getErrorStream()), chunkSize = readBufferSize)
+          // Avoids broken pipe - we cut off when the program ends.
+          // Users can decide what to do with the error logs using the exitCode value
+          .interruptWhen(done.void.attempt)


readInputStream is not using interruptible looks like.
https://github.com/typelevel/fs2/blob/56b86022d48d3caf0a9b5f00bf637cac62bb7f5e/io/shared/src/main/scala/fs2/io/io.scala#L46

So that would probably explain why it is hanging. We can try swapping that, but as you mentioned interruptibility might be broken like it is for stdin. In which case I suppose each Process would have to start its own dedicated blocking threads that it can kill 🤔

I published 3.7-ae0fc02-SNAPSHOT which swapped in interruptible, you can give that a try and see if it helps?

Can't resolve yet, are you sure it's the right version ?

https://s01.oss.sonatype.org/content/repositories/snapshots/co/fs2/fs2-io_3/3.7-ae0fc02-SNAPSHOT/

Doh, my bad. Will try in a bit

3.7-0ec6699-SNAPSHOT

Introduce Process API typelevel/fs2#3149

Mmm I only get the initial prints from both the client and server side, but then nothing ... It may be because chunks are not automatically flushed in your implementation. That's why I couldn't use the Davenverse lib directly :

https://github.com/neandertech/jsonrpclib/pull/58/files#diff-281c0b146d38f8e1fc9f7c77eb87773debe12d364454ef6766472a26563a0ee6R47-R59

Woops, I missed that. Thanks!

Ok, try this one 3.7-eacce62-SNAPSHOT.

It works 🎉 most things that should print actually print (except for the server termination message but that's some userland problem), and the program terminates as expected !

* Add method on FS2Channel: resources * Rewrite old stream variant in terms of resource --------- Co-authored-by: ghostbuster91 <ghostbuster91@users.noreply.github.com>

Fix smithy protocol setup

* feat: Add jsonPayload trait * Remove useless comments * Add tests for JsonPayloadValidator * Transform errors as well * Use traitValidator --------- Co-authored-by: ghostbuster91 <ghostbuster91@users.noreply.github.com>

Co-authored-by: Jakub Kozłowski <kubukoz@gmail.com>

* Create caches for document encoders * Remove unused imports * Remove redundant classes * Apply review comments * add client side error handling * Mark impl details package private * minor simplification --------- Co-authored-by: ghostbuster91 <ghostbuster91@users.noreply.github.com> Co-authored-by: Jakub Kozłowski <kubukoz@gmail.com>

Baccata · 2025-06-30T09:45:56Z

Can't approve my own PR, but LGTM (I'm mostly interested in what you did in the core, I trust you for the smithy4s and tests-related code)

Baccata added 7 commits February 21, 2023 10:21

Wrote server/client interpreters

b11823c

Example compiles

8b5795f

Fixed un-bound flatMap, tried a few other things

c18a814

Merge remote-tracking branch 'origin/main' into smithy4s-integration

619eb63

Fix comment

cce5d1d

Revert changes to FS2Channel

0c3251b

Bump remaining versions

2cfb589

Baccata commented Feb 21, 2023

View reviewed changes

Comment thread examples/smithyClient/src/examples/smithy/client/ChildProcess.scala Outdated

armanbilge reviewed Feb 21, 2023

View reviewed changes

ghostbuster91 and others added 9 commits May 3, 2025 23:19

Merge branch 'main' into smithy4s-integration

4b8c0db

Merge branch 'main' into smithy4s-integration

6f93323

Upgrade to latest smithy4s

53cee0a

Fix scala native part of the build

fe670de

Fix duplicated resources error

799e818

Cross publish smithy across all platforms

9523f61

Add versionScheme

67c49a5

Convert for-comp to flatMap

703e25a

Add method on FS2Channel: resource (#80)

0e7dd22

* Add method on FS2Channel: resources * Rewrite old stream variant in terms of resource --------- Co-authored-by: ghostbuster91 <ghostbuster91@users.noreply.github.com>

kubukoz mentioned this pull request May 6, 2025

Replace bsp codegen with smithy4s-bsp simple-scala-tooling/sls#18

Merged

kubukoz and others added 5 commits May 6, 2025 03:53

Fix smithy protocol wiring

7d1323c

Merge pull request #81 from neandertech/fix-smithy-setup

c3ce708

Fix smithy protocol setup

Generate java parts for smithy traits

6269973

Set minJdkVersion to 11

8aea9e6

Remove unused file

e45acf7

kubukoz reviewed May 7, 2025

View reviewed changes

Comment thread modules/examples/smithyClient/src/main/scala/examples/smithy/client/ChildProcess.scala Outdated

kubukoz reviewed May 7, 2025

View reviewed changes

Comment thread modules/smithy/src/main/resources/META-INF/smithy/jsonrpclib.smithy Outdated

kubukoz reviewed May 7, 2025

View reviewed changes

Comment thread modules/smithy4s/src/main/scala/jsonrpclib/smithy4sinterop/ClientStub.scala Outdated

Simplify smithy4s implementation, get rid of fs2 in it

97462b6

Add common smithy traits into protocol definition

6c4b437

kubukoz reviewed May 13, 2025

View reviewed changes

Comment thread modules/smithy4s/src/main/scala/jsonrpclib/smithy4sinterop/ClientStub.scala Outdated

ghostbuster91 and others added 13 commits May 14, 2025 22:06

Better way to coerce unit

830a6b0

Errors smithy4s (#86)

c070f65

Filter out optionals by default (#87)

bec1dca

Accessing endpoints from multiple servers

246ba77

chore: sort and group imports with scalafmt

903ab82

Add some scaladocs

3821de8

internal error when processing notification should not break the server

fdb23b4

Add smithy validations (#88)

9395f05

Fix RawMessage decoder

a76d283

feat: Add jsonPayload trait (#89)

c3546cc

* feat: Add jsonPayload trait * Remove useless comments * Add tests for JsonPayloadValidator * Transform errors as well * Use traitValidator --------- Co-authored-by: ghostbuster91 <ghostbuster91@users.noreply.github.com>

Rename traits to include Rpc in their names

e168783

Update readme

074fb87

Fix missing renames

8eee63e

ghostbuster91 marked this pull request as ready for review May 29, 2025 20:34

kubukoz approved these changes Jun 6, 2025

View reviewed changes

ghostbuster91 and others added 5 commits June 7, 2025 10:07

Update README.md

381b083

Co-authored-by: Jakub Kozłowski <kubukoz@gmail.com>

Update project/build.sbt

55e611e

Co-authored-by: Jakub Kozłowski <kubukoz@gmail.com>

Update project/plugins.sbt

6f2f70c

Co-authored-by: Jakub Kozłowski <kubukoz@gmail.com>

Apply review comments

1bc15ae

Replace target with release

9734e14

kubukoz reviewed Jun 14, 2025

View reviewed changes

Comment thread modules/smithy4s/src/main/scala/jsonrpclib/smithy4sinterop/ClientStub.scala Outdated

ghostbuster91 and others added 2 commits June 14, 2025 09:36

Fail when service does support jsonRPC protocol

9cbe713

Baccata commented Jun 30, 2025

View reviewed changes

Comment thread modules/core/src/main/scala/jsonrpclib/Monadic.scala

Mark extension classes as final

b63bcfe

ghostbuster91 merged commit 078fef7 into main Jun 30, 2025
2 checks passed

ghostbuster91 deleted the smithy4s-integration branch June 30, 2025 09:57

Uh oh!

Conversation

Baccata commented Feb 21, 2023 • edited by ghostbuster91 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Baccata commented Feb 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Baccata Feb 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Baccata commented Jun 30, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Baccata commented Feb 21, 2023 •

edited by ghostbuster91

Loading

Baccata commented Feb 21, 2023 •

edited

Loading

Baccata Feb 21, 2023 •

edited

Loading