Conversation
c6898a8 to
6f553aa
Compare
56d4530 to
2b4caf2
Compare
This switches capnp-rpc from Lwt to Eio. One particularly nice side effect of this is that `Service.return_lwt` has gone, as there is no distinction now between concurrent and non-concurrent service methods.
| | Ok (`Data data) -> | ||
| Log.debug (fun f -> f "Read %d bytes" (Cstruct.length data)); | ||
| Capnp.Codecs.FramedStream.add_fragment t.decoder (Cstruct.to_string data); | ||
| let buf = Cstruct.create 4096 in (* TODO: make this efficient *) |
There was a problem hiding this comment.
Note: there will be a follow-on PR replacing most of endpoint.ml with a buffered system.
|
|
||
| let run_client service = | ||
| let n = 100000 in | ||
| (* let n = 100000 in *) (* XXX: improve speed *) |
There was a problem hiding this comment.
Will be fixed in a follow-on PR.
| If an application forgets to release a resource and it gets GC'd then we want to | ||
| log a warning and clean up (so forgotten refs don't build up over time). | ||
|
|
||
| Because GC finalizers can run at any time and from any thread, |
There was a problem hiding this comment.
The Lwt version didn't handle the case of a finalizer running from a different thread. This is more important with multiple domains too.
| (* todo: we stop waiting and we send a finish message, but we don't currently | ||
| abort the service operation. *) |
There was a problem hiding this comment.
The Lwt version doesn't cancel service operations either, so this isn't a regression.
| Capability.await_settled_exn service >>= fun () -> | ||
| Capability.await_settled_exn service2 >>= fun () -> |
There was a problem hiding this comment.
Depending on timing, these could fail if the vat is shut down before returning the results. So, use the non-exn version.
| Lwt.wakeup set_service @@ Capnp_rpc_net.Restorer.grant @@ Echo.local (); | ||
| service >>= fun _ -> |
There was a problem hiding this comment.
Changed this to have the test wait for the disconnected error before letting the server continue to ensure it really is testing this case.
| val add_connection : t -> switch:Lwt_switch.t -> mode:[`Accept|`Connect] -> Endpoint.t -> CapTP.t Lwt.t | ||
| (** [add_connection t ~switch ~mode endpoint] runs the CapTP protocol over [endpoint], | ||
| val run_connection : t -> mode:[`Accept|`Connect] -> Endpoint.t -> (CapTP.t -> unit) -> unit | ||
| (** [run_connection t ~mode endpoint r] runs the protocol over [endpoint], |
There was a problem hiding this comment.
This simplifies FD management, since endpoint can always be closed when the function returns.
| Lwt_switch.add_hook switch (fun () -> | ||
| let ex = Capnp_rpc.Exception.v ~ty:`Disconnected "Vat shut down" in | ||
| ID_map.bindings t.connections |> Lwt_list.iter_p (fun (_, c) -> CapTP.disconnect c ex) >>= fun () -> | ||
| t.connections <- ID_map.empty; | ||
| Lwt_list.iter_p (fun c -> CapTP.disconnect c ex) t.anon_connections >|= fun () -> | ||
| t.anon_connections <- []; | ||
| ID_map.iter (fun _ th -> Lwt.cancel th) t.connecting; | ||
| t.connecting <- ID_map.empty; | ||
| ); |
There was a problem hiding this comment.
There's no need to clean up the connections now, because each connection has an owning daemon fiber that will close it when it ends.
| let connect_anon t addr ~service = | ||
| let switch = Lwt_switch.create () in | ||
| Network.connect t.network ~switch ~secret_key:t.secret_key addr >>= function | ||
| | Error (`Msg m) -> Lwt.return @@ Error (Capnp_rpc.Exception.v m) | ||
| | Ok ep -> | ||
| add_connection t ~switch ep ~mode:`Connect >|= fun conn -> | ||
| Ok (CapTP.bootstrap conn service) |
There was a problem hiding this comment.
This logic got merged into initiate_connection.
|
CHANGES:
capnp-rpc 2.0 switches from using Lwt to Eio.
The `capnp-rpc-lwt` package is now just Lwt wrappers around the Eio functions in `capnp-rpc`.
This allows libraries using the old Lwt API to be used with applications and libraries using the new Eio one,
in the same binary.
Because Lwt libraries (using `capnp-rpc-lwt`) can be used with Eio applications (using `capnp-rpc-unix`),
you should first upgrade your application and then upgrade the libraries afterwards.
It is recommended to upgrade in stages, checking your application still works after each step:
1. Upgrade to the latest Lwt version (`opam install capnp-rpc-unix.1.2.4`).
This uses the new version of mirage-crypto, tls, etc,
which may be incompatible with other libraries you are using.
Get that sorted out before switching to capnp-rpc 2.0.
2. Use [lwt_eio][] to allow using Eio and Lwt together in your application.
This means replacing your call to `Lwt_main.run` with code to run Eio and Lwt together,
as explained at the start of the `lwt_eio` README.
3. Upgrade your main application to capnp-rpc-unix 2.0.
Calls to functions in the `Capnp_rpc_unix` and `Capnp_rpc_net` modules will need minor updates.
They were previously called from Lwt context, so you'll need to wrap them with `Lwt_eio.run_eio`
(or remove a `Lwt_eio.run_lwt` if you already updated the surrounding code).
You should also use `mirage-crypto-rng-eio` to ensure randomness is available
(`capnp-rpc-unix` no longer does this, although some other library might).
4. Upgrade code and libraries using `Capnp_rpc_lwt`:
1. Replace `open Capnp_rpc_lwt` with `open Capnp_rpc.Std`.
2. Replace all other uses of `Capnp_rpc_lwt` with just `Capnp_rpc`.
3. In `dune` and `opam` files, replace `capnp-rpc-lwt` with `capnp-rpc`.
4. Some modules are in `Capnp_rpc` but not the `Capnp_rpc.Std` subset.
Those should now be fully qualified (e.g. replace `Persistence` with
`Capnp_rpc.Persistence`).
5. Replace `Service.return_lwt` with `Lwt_eio.run_lwt`.
Replace `Lwt.return (Ok x)` with `Service.return x`.
Once all Lwt code is gone, you can remove the dependencies on `lwt` and `lwt_eio`.
New features:
- Allow numeric IPv6 listen addresses (@talex5 mirage/capnp-rpc#296, requested by @BChabanne).
API changes:
- Eio port (@talex5 mirage/capnp-rpc#280 mirage/capnp-rpc#284 mirage/capnp-rpc#298 mirage/capnp-rpc#292 mirage/capnp-rpc#297 mirage/capnp-rpc#300 mirage/capnp-rpc#304).
This switches capnp-rpc from Lwt to Eio.
One particularly nice side effect of this is that `Service.return_lwt` has gone,
as there is no distinction now between concurrent and non-concurrent service methods.
Uses of `Capnp_rpc_lwt` should be replaced by uses of `Capnp_rpc`
(and the old `Capnp_rpc` is now an internal module, `Capnp_rpc_proto`).
The new `Capnp_rpc` now provides `Error`, `Exception` and `Debug` aliases to the same modules in `Capnp_rpc_proto`,
so that `Capnp_rpc_proto` doesn't need to be used directly.
`Capnp_rpc_lwt` now provides (deprecated) compatibility wrappers, using `lwt_eio`.
This also adds `Capnp_rpc.Std` with some common module aliases,
to reduce the need to `open Capnp_rpc` (which is rather large).
- Deprecate `Debug.failf` (@talex5 mirage/capnp-rpc#291).
Performance and bug fixes:
- Add buffering of outgoing messages (@talex5 mirage/capnp-rpc#287 mirage/capnp-rpc#303).
Sending each message in its own TCP packet isn't very efficient, and also interacts very badly with Nagle's algorithm.
See <https://roscidus.com/blog/blog/2024/07/22/performance/> for details.
- Only disconnect socket when sending is done (@talex5 mirage/capnp-rpc#295).
Allows sending the reason for the disconnection in some cases.
- Fix type of `Capability.call` (@talex5 mirage/capnp-rpc#293).
It was previously unusable.
- Work around FreeBSD `close` problem (@talex5 mirage/capnp-rpc#302).
Build:
- Update to capnp 3.6.0 (@talex5 mirage/capnp-rpc#285).
- Update to dune 3.16 and fix warnings (@talex5 mirage/capnp-rpc#282).
- Use `String.starts_with` instead of `Astring` (@talex5 mirage/capnp-rpc#290).
- Remove unused `tls-mirage` dependency (@talex5 mirage/capnp-rpc#289).
- Remove `asetmap` dependency (@talex5 mirage/capnp-rpc#288).
[lwt_eio]: https://github.com/ocaml-multicore/lwt_eio
CHANGES:
capnp-rpc 2.0 switches from using Lwt to Eio.
The `capnp-rpc-lwt` package is now just Lwt wrappers around the Eio functions in `capnp-rpc`.
This allows libraries using the old Lwt API to be used with applications and libraries using the new Eio one,
in the same binary.
Because Lwt libraries (using `capnp-rpc-lwt`) can be used with Eio applications (using `capnp-rpc-unix`),
you should first upgrade your application and then upgrade the libraries afterwards.
It is recommended to upgrade in stages, checking your application still works after each step:
1. Upgrade to the latest Lwt version (`opam install capnp-rpc-unix.1.2.4`).
This uses the new version of mirage-crypto, tls, etc,
which may be incompatible with other libraries you are using.
Get that sorted out before switching to capnp-rpc 2.0.
2. Use [lwt_eio][] to allow using Eio and Lwt together in your application.
This means replacing your call to `Lwt_main.run` with code to run Eio and Lwt together,
as explained at the start of the `lwt_eio` README.
3. Upgrade your main application to capnp-rpc-unix 2.0.
Calls to functions in the `Capnp_rpc_unix` and `Capnp_rpc_net` modules will need minor updates.
They were previously called from Lwt context, so you'll need to wrap them with `Lwt_eio.run_eio`
(or remove a `Lwt_eio.run_lwt` if you already updated the surrounding code).
You should also use `mirage-crypto-rng-eio` to ensure randomness is available
(`capnp-rpc-unix` no longer does this, although some other library might).
4. Upgrade code and libraries using `Capnp_rpc_lwt`:
1. Replace `open Capnp_rpc_lwt` with `open Capnp_rpc.Std`.
2. Replace all other uses of `Capnp_rpc_lwt` with just `Capnp_rpc`.
3. In `dune` and `opam` files, replace `capnp-rpc-lwt` with `capnp-rpc`.
4. Some modules are in `Capnp_rpc` but not the `Capnp_rpc.Std` subset.
Those should now be fully qualified (e.g. replace `Persistence` with
`Capnp_rpc.Persistence`).
5. Replace `Service.return_lwt` with `Lwt_eio.run_lwt`.
Replace `Lwt.return (Ok x)` with `Service.return x`.
Once all Lwt code is gone, you can remove the dependencies on `lwt` and `lwt_eio`.
New features:
- Allow numeric IPv6 listen addresses (@talex5 mirage/capnp-rpc#296, requested by @BChabanne).
API changes:
- Eio port (@talex5 mirage/capnp-rpc#280 mirage/capnp-rpc#284 mirage/capnp-rpc#298 mirage/capnp-rpc#292 mirage/capnp-rpc#297 mirage/capnp-rpc#300 mirage/capnp-rpc#304).
This switches capnp-rpc from Lwt to Eio.
One particularly nice side effect of this is that `Service.return_lwt` has gone,
as there is no distinction now between concurrent and non-concurrent service methods.
Uses of `Capnp_rpc_lwt` should be replaced by uses of `Capnp_rpc`
(and the old `Capnp_rpc` is now an internal module, `Capnp_rpc_proto`).
The new `Capnp_rpc` now provides `Error`, `Exception` and `Debug` aliases to the same modules in `Capnp_rpc_proto`,
so that `Capnp_rpc_proto` doesn't need to be used directly.
`Capnp_rpc_lwt` now provides (deprecated) compatibility wrappers, using `lwt_eio`.
This also adds `Capnp_rpc.Std` with some common module aliases,
to reduce the need to `open Capnp_rpc` (which is rather large).
- Deprecate `Debug.failf` (@talex5 mirage/capnp-rpc#291).
Performance and bug fixes:
- Add buffering of outgoing messages (@talex5 mirage/capnp-rpc#287 mirage/capnp-rpc#303).
Sending each message in its own TCP packet isn't very efficient, and also interacts very badly with Nagle's algorithm.
See <https://roscidus.com/blog/blog/2024/07/22/performance/> for details.
- Only disconnect socket when sending is done (@talex5 mirage/capnp-rpc#295).
Allows sending the reason for the disconnection in some cases.
- Fix type of `Capability.call` (@talex5 mirage/capnp-rpc#293).
It was previously unusable.
- Work around FreeBSD `close` problem (@talex5 mirage/capnp-rpc#302).
Build:
- Update to capnp 3.6.0 (@talex5 mirage/capnp-rpc#285).
- Update to dune 3.16 and fix warnings (@talex5 mirage/capnp-rpc#282).
- Use `String.starts_with` instead of `Astring` (@talex5 mirage/capnp-rpc#290).
- Remove unused `tls-mirage` dependency (@talex5 mirage/capnp-rpc#289).
- Remove `asetmap` dependency (@talex5 mirage/capnp-rpc#288).
[lwt_eio]: https://github.com/ocaml-multicore/lwt_eio
CHANGES:
capnp-rpc 2.0 switches from using Lwt to Eio.
The `capnp-rpc-lwt` package is now just Lwt wrappers around the Eio functions in `capnp-rpc`.
This allows libraries using the old Lwt API to be used with applications and libraries using the new Eio one,
in the same binary.
Because Lwt libraries (using `capnp-rpc-lwt`) can be used with Eio applications (using `capnp-rpc-unix`),
you should first upgrade your application and then upgrade the libraries afterwards.
It is recommended to upgrade in stages, checking your application still works after each step:
1. Upgrade to the latest Lwt version (`opam install capnp-rpc-unix.1.2.4`).
This uses the new version of mirage-crypto, tls, etc,
which may be incompatible with other libraries you are using.
Get that sorted out before switching to capnp-rpc 2.0.
2. Use [lwt_eio][] to allow using Eio and Lwt together in your application.
This means replacing your call to `Lwt_main.run` with code to run Eio and Lwt together,
as explained at the start of the `lwt_eio` README.
3. Upgrade your main application to capnp-rpc-unix 2.0.
Calls to functions in the `Capnp_rpc_unix` and `Capnp_rpc_net` modules will need minor updates.
They were previously called from Lwt context, so you'll need to wrap them with `Lwt_eio.run_eio`
(or remove a `Lwt_eio.run_lwt` if you already updated the surrounding code).
You should also use `mirage-crypto-rng-eio` to ensure randomness is available
(`capnp-rpc-unix` no longer does this, although some other library might).
4. Upgrade code and libraries using `Capnp_rpc_lwt`:
1. Replace `open Capnp_rpc_lwt` with `open Capnp_rpc.Std`.
2. Replace all other uses of `Capnp_rpc_lwt` with just `Capnp_rpc`.
3. In `dune` and `opam` files, replace `capnp-rpc-lwt` with `capnp-rpc`.
4. Some modules are in `Capnp_rpc` but not the `Capnp_rpc.Std` subset.
Those should now be fully qualified (e.g. replace `Persistence` with
`Capnp_rpc.Persistence`).
5. Replace `Service.return_lwt` with `Lwt_eio.run_lwt`.
Replace `Lwt.return (Ok x)` with `Service.return x`.
Once all Lwt code is gone, you can remove the dependencies on `lwt` and `lwt_eio`.
New features:
- Allow numeric IPv6 listen addresses (@talex5 mirage/capnp-rpc#296, requested by @BChabanne).
API changes:
- Eio port (@talex5 mirage/capnp-rpc#280 mirage/capnp-rpc#284 mirage/capnp-rpc#298 mirage/capnp-rpc#292 mirage/capnp-rpc#297 mirage/capnp-rpc#300 mirage/capnp-rpc#304).
This switches capnp-rpc from Lwt to Eio.
One particularly nice side effect of this is that `Service.return_lwt` has gone,
as there is no distinction now between concurrent and non-concurrent service methods.
Uses of `Capnp_rpc_lwt` should be replaced by uses of `Capnp_rpc`
(and the old `Capnp_rpc` is now an internal module, `Capnp_rpc_proto`).
The new `Capnp_rpc` now provides `Error`, `Exception` and `Debug` aliases to the same modules in `Capnp_rpc_proto`,
so that `Capnp_rpc_proto` doesn't need to be used directly.
`Capnp_rpc_lwt` now provides (deprecated) compatibility wrappers, using `lwt_eio`.
This also adds `Capnp_rpc.Std` with some common module aliases,
to reduce the need to `open Capnp_rpc` (which is rather large).
- Deprecate `Debug.failf` (@talex5 mirage/capnp-rpc#291).
Performance and bug fixes:
- Add buffering of outgoing messages (@talex5 mirage/capnp-rpc#287 mirage/capnp-rpc#303).
Sending each message in its own TCP packet isn't very efficient, and also interacts very badly with Nagle's algorithm.
See <https://roscidus.com/blog/blog/2024/07/22/performance/> for details.
- Only disconnect socket when sending is done (@talex5 mirage/capnp-rpc#295).
Allows sending the reason for the disconnection in some cases.
- Fix type of `Capability.call` (@talex5 mirage/capnp-rpc#293).
It was previously unusable.
- Work around FreeBSD `close` problem (@talex5 mirage/capnp-rpc#302).
Build:
- Update to capnp 3.6.0 (@talex5 mirage/capnp-rpc#285).
- Update to dune 3.16 and fix warnings (@talex5 mirage/capnp-rpc#282).
- Use `String.starts_with` instead of `Astring` (@talex5 mirage/capnp-rpc#290).
- Remove unused `tls-mirage` dependency (@talex5 mirage/capnp-rpc#289).
- Remove `asetmap` dependency (@talex5 mirage/capnp-rpc#288).
[lwt_eio]: https://github.com/ocaml-multicore/lwt_eio
CHANGES:
capnp-rpc 2.0 switches from using Lwt to Eio.
The `capnp-rpc-lwt` package is now just Lwt wrappers around the Eio functions in `capnp-rpc`.
This allows libraries using the old Lwt API to be used with applications and libraries using the new Eio one,
in the same binary.
Because Lwt libraries (using `capnp-rpc-lwt`) can be used with Eio applications (using `capnp-rpc-unix`),
you should first upgrade your application and then upgrade the libraries afterwards.
It is recommended to upgrade in stages, checking your application still works after each step:
1. Upgrade to the latest Lwt version (`opam install capnp-rpc-unix.1.2.4`).
This uses the new version of mirage-crypto, tls, etc,
which may be incompatible with other libraries you are using.
Get that sorted out before switching to capnp-rpc 2.0.
2. Use [lwt_eio][] to allow using Eio and Lwt together in your application.
This means replacing your call to `Lwt_main.run` with code to run Eio and Lwt together,
as explained at the start of the `lwt_eio` README.
3. Upgrade your main application to capnp-rpc-unix 2.0.
Calls to functions in the `Capnp_rpc_unix` and `Capnp_rpc_net` modules will need minor updates.
They were previously called from Lwt context, so you'll need to wrap them with `Lwt_eio.run_eio`
(or remove a `Lwt_eio.run_lwt` if you already updated the surrounding code).
You should also use `mirage-crypto-rng-eio` to ensure randomness is available
(`capnp-rpc-unix` no longer does this, although some other library might).
4. Upgrade code and libraries using `Capnp_rpc_lwt`:
1. Replace `open Capnp_rpc_lwt` with `open Capnp_rpc.Std`.
2. Replace all other uses of `Capnp_rpc_lwt` with just `Capnp_rpc`.
3. In `dune` and `opam` files, replace `capnp-rpc-lwt` with `capnp-rpc`.
4. Some modules are in `Capnp_rpc` but not the `Capnp_rpc.Std` subset.
Those should now be fully qualified (e.g. replace `Persistence` with
`Capnp_rpc.Persistence`).
5. Replace `Service.return_lwt` with `Lwt_eio.run_lwt`.
Replace `Lwt.return (Ok x)` with `Service.return x`.
Once all Lwt code is gone, you can remove the dependencies on `lwt` and `lwt_eio`.
New features:
- Allow numeric IPv6 listen addresses (@talex5 mirage/capnp-rpc#296, requested by @BChabanne).
API changes:
- Eio port (@talex5 mirage/capnp-rpc#280 mirage/capnp-rpc#284 mirage/capnp-rpc#298 mirage/capnp-rpc#292 mirage/capnp-rpc#297 mirage/capnp-rpc#300 mirage/capnp-rpc#304).
This switches capnp-rpc from Lwt to Eio.
One particularly nice side effect of this is that `Service.return_lwt` has gone,
as there is no distinction now between concurrent and non-concurrent service methods.
Uses of `Capnp_rpc_lwt` should be replaced by uses of `Capnp_rpc`
(and the old `Capnp_rpc` is now an internal module, `Capnp_rpc_proto`).
The new `Capnp_rpc` now provides `Error`, `Exception` and `Debug` aliases to the same modules in `Capnp_rpc_proto`,
so that `Capnp_rpc_proto` doesn't need to be used directly.
`Capnp_rpc_lwt` now provides (deprecated) compatibility wrappers, using `lwt_eio`.
This also adds `Capnp_rpc.Std` with some common module aliases,
to reduce the need to `open Capnp_rpc` (which is rather large).
- Deprecate `Debug.failf` (@talex5 mirage/capnp-rpc#291).
Performance and bug fixes:
- Add buffering of outgoing messages (@talex5 mirage/capnp-rpc#287 mirage/capnp-rpc#303).
Sending each message in its own TCP packet isn't very efficient, and also interacts very badly with Nagle's algorithm.
See <https://roscidus.com/blog/blog/2024/07/22/performance/> for details.
- Only disconnect socket when sending is done (@talex5 mirage/capnp-rpc#295).
Allows sending the reason for the disconnection in some cases.
- Fix type of `Capability.call` (@talex5 mirage/capnp-rpc#293).
It was previously unusable.
- Work around FreeBSD `close` problem (@talex5 mirage/capnp-rpc#302).
Build:
- Update to capnp 3.6.0 (@talex5 mirage/capnp-rpc#285).
- Update to dune 3.16 and fix warnings (@talex5 mirage/capnp-rpc#282).
- Use `String.starts_with` instead of `Astring` (@talex5 mirage/capnp-rpc#290).
- Remove unused `tls-mirage` dependency (@talex5 mirage/capnp-rpc#289).
- Remove `asetmap` dependency (@talex5 mirage/capnp-rpc#288).
[lwt_eio]: https://github.com/ocaml-multicore/lwt_eio
This switches capnp-rpc from Lwt to Eio. One particularly nice side effect of this is that
Service.return_lwthas gone, as there is no distinction now between concurrent and non-concurrent service methods.#256 was supposed to be the initial support, but extra things got added to it and now people are using that branch, so this is a new "minimal" PR.
Once this is merged, some follow-on PRs will be needed: