I reviewed and merged Anil's pseudoterminal support PR (started in 2023!), which allows running a subprocess with a virtual terminal, controlled by the parent:
The main issue here was making it work in harmony with Eio's inherit_fds feature.
I reviewed and merged chown support (similar to the chmod support added last time).
chown in eio_posix and eio_linux #781.
I failed to spot some important problems with the new code, but luckily Anil noticed them soon after the merge:
OCaml's Unix.getaddrinfo returns [] (no results) on any error, which can be confusing as it hides the real error.
For example, you can be told that a name doesn't exist when in fact the process has just reached its limit on the number of open FDs!
I updated an old PR from Christiano which raises a suitable exception instead (the problem with the old PR was that it was based on OCaml's code, but the OCaml developers weren't willing to let us use it so it needed to be reimplemented):
Disappointingly, it turned out that the underlying getaddrinfo implementation in glibc also
fails to propagate the FDs-limit error!
I updated Eio to use the safer Res.t result type we recently added to uring:
Uring.Res module for safer error handling #868.
We had some discussion about whether the code for reading string-valued socket options was correct, and made the code a bit stricter:
Path.load didn't work correctly with files in /proc, which report a size of 0 but do have content.
Worked with Patrick to get that fixed:
Eio.Path.load to handle files with zero size #869.
It turns out that you also have to be careful about buffer sizes when reading from /proc,
as Linux can give incorrect results for small buffers:
Path.load #878.
I learnt about Linux's /proc/pressure/io system, and investigated a report that Eio was causing excessive pressure (it turned out to be a known problem in Linux with uring):
Other minor PR reviews:
I also made a few minor cleanups in mirage/prometheus:
Thomas Gazagnaire noticed that it was formatting floats with %f, losing some precision:
%.17g #62.
Prometheus also had some dependencies that aren't needed with modern compiler versions:
At some point Prometheus needs some larger refactoring, to split off the Lwt parts from the core.
]]>We've wanted support for setting socket options in Eio for ages, and I finally got around to reviewing Anil's PR adding them. It was pretty big, so I split parts of it off into two smaller PRs and reviewed them separately (it's all merged now):
wayland-proxy-virtwl uses a FIFO (named piped) to accept debug commands
(asking it to dump the log ring to a file).
I'd noticed a while ago that it would spin using 100% CPU when run with EIO_BACKEND=posix
and finally got around to investigating:
In summary, opening a FIFO for reading normally waits for a writer to connect,
but in a concurrent application that would stop the whole program.
To fix that, we open them in non-blocking mode.
But that has a wierd behaviour: trying to read from the pipe immediately returns end-of-file
rather than EAGAIN, even though select says that it's not ready!
Other reviewed PRs:
Propagate errno back from fork_actions into Eio.Process #854.
This allows handling specific errors, rather than just getting a generic Failure with a string.
Support chmod in Eio_linux and Eio_posix #785.
Another of Patrick's PRs adding missing Unix functionality to Eio.
Relax return types of Eio.Process.Pipe #775.
This was an Outreachy PR that got abandoned in 2024; finishing it off was a quick job.
Talking with Shon Feder at the Tarides off-site reminded me I should try out
OCaml Package Management With Dune.
To build an OCaml program you normally first run opam install --deps-only -t . to get the dependencies
and then dune build to build it, but you can have dune get the required packages too by using dune build --pkg=enable.
It's still experimental, but looks promising. Some things I noticed:
Dune's progress indicator isn't really designed for long build times, such as downloading and building the OCaml compiler. It could do with showing more information.
It doesn't seem to support installing depexts (non-OCaml dependencies) yet, although it can tell you what they are.
But default it doesn't use a shared cache, so if you build two projects this way it will build the compiler twice.
You need to set DUNE_CACHE=enabled to turn that on.
I'm using it now to generate this blog, and that's working nicely. I also tried it with Eio, but hit a problem trying to generate the docs:
dune build --pkg=enable @doc-new #15290.
I took Patrick's PR for improved directory reading and finished it off.
As well as providing incremental reads, this PR also gives you the entry's type with the name,
which is much faster than calling stat on each one.
I improved the performance of the eio_posix version and tidied things up a bit:
After reviewing Anil's changes to Uring.Region.init
(#145),
we realised that it would be cleaner to move fixed-buffer regions to Eio:
Eio_linux.Low_level.Fixed #848.
Reviewing Anil's uring PR adding Linux specific errnos (#152), I realised that the code using them is probably unnecessary anyway and got rid of it:
I've found Eio's Path.open_dir function is confusingly named.
It's not about opening directories (that's just an implementation detail),
it's about restricting a path to a subtree.
I proposed a PR to rename it (discussion is on-going):
Path.open_dir to open_subtree #853.
I'm still intending to revive Anil's socket options PR (#575), but as a first step I sorted out an encoding issue that was preventing the tests diff from being shown:
Since Eio's very first Git commit, the scheduler has been special-casing fixed-buffer IO operations. Moving it out simplified things nicely:
The lintcstubs-arity package auto-generates C function declarations from OCaml code,
which avoids some common mistakes (like returning void instead of a value of Val_unit,
which has caused unpredictable segfaults in the past).
The package was previously only used if installed for some reason, so people often forget to run it:
It would be good to get the whole of lintcstubs working one day (currently it only works with old versions of OCaml).
Statx attribute supported check was inverted #142.
Which led us to revise the API for Attr.check.
Add shutdown, socket, renameat and symlinkat #147.
Fix debug messages in ocaml_uring_unregister_buffers #146.
caml_uring_wait_cqe should have the same null cqe check that the timeout variant has #148.
Add fallocate and ftruncate bindings #149.
Allow setting socket flags on socket creation #151.
I released uring 2.14.0 with the recent changes:
I also continued looking at using uring flags for better Eio performance. First, Eio now tells uring that each ring has a single submitting thread:
IORING_SETUP_SINGLE_ISSUER #841.
This allows uring to skip some locking apparently, but it didn't make any noticable difference to me.
However, it does mean you get a better error if you accidentally try to submit from two threads,
which can happen if something calls Unix.fork (eio/issues/801).
Except that Uring.submit failed to report errors:
Last week I tried using IORING_SETUP_COOP_TASKRUN and IORING_SETUP_DEFER_TASKRUN,
but noted that DEFER caused a test to hang.
The test has a fiber that's always runnable (and keeps yielding) and is checking that IO still gets handled.
With DEFER, Eio sees that there are no completions ready and so just gets on with the CPU-intensive fiber.
I didn't see this problem with COOP mode, but I'm not sure why not because we're not doing any syscalls.
In any case, the fix turned out to be simple: uring has a IORING_SETUP_TASKRUN_FLAG which tells the kernel
to set a flag in the shared memory if there are completion tasks that need to run,
and io_uring_peek_cqe checks this flag and enters the kernel automatically if needed.
So, Eio is now using DEFER mode:
I'd be interested to know if anyone else sees a speed-up with this.
According to io_uring_setup_flags(7), zero-copy receive and ring resizing also require DEFER mode.
I reviewed some more PRs (from Patrick, Anil and Mark):
COMMENTED Incremental directory reading #821.
Greatly improves the speed of scanning directories by returning the type of each entry as part of the read
(saving one stat per file) and by allowing you to process the listing incrementally.
Support chown in eio_posix and eio_linux #781.
docker: add ?os to Peek and Pull (default linux) #472.
Statx attribute supported check was inverted #142.
Uring.Statx.Attr.check had a logic error, but now I'm wondering if it's even useful.
Fix dio_offset_align statx getter returning wrong field #141.
I updated my MDX hang-detector (#476) to report accurate locations for cram tests too. The CI was failing, which turned out to be due to a recent change in dune:
I updated uring to use MSG_CMSG_CLOEXEC when receiving file descriptors,
and this week I made the equivalent change in eio_posix (for other systems):
MSG_CMSG_CLOEXEC when receiving file descriptors #843.
Disappointingly, it turns out that macos doesn't support this flag and I had to add a work-around for that.
The Eio benchmarks use a Dockerfile for the tests and it hadn't been updated for ages. I was hoping that using newer dependencies might show a speed improvement, but it didn't help:
I had a brief look at improving the eio-tar API, which has a number of problems currently:
Tar.t type that's used by the core to abstract over monads and effects,
which Eio users shouldn't need to care about.
Tar_eio.extract needs to take a switch argument because it doesn't actually do the extraction,
but just returns a computation that runs later, and so needs to hold on to the argument.
Patrick explained to me that extract needs to return a computation so that you can wrap it with Tar_gz.in_gzipped.
I think it would make a lot more sense to wrap the source file with a gzip decoder, rather than wrap the computation,
but changing it all looked like a fair bit of work and I didn't go any further.
I've got a new contract to do some Eio maintenance and related work for 3 months, and I intend to post rough notes about what I'm working on here.
I've been having a look at the backlog of PRs for ocaml-uring.
ocaml-uring#129 and
ocaml-uring#130
were two different attempts at dealing with file descriptors
(working around OCaml's lack of a safe way to convert between int
and Unix.file_descr).
After trying the proposed API out with Eio,
I went with a modified version of #129:
Uring.Res module to handle result values #131.
Ideally, Eio would use the new features to handle results more safely, but until the new version is released I just added a cast so that Eio will work with either version:
Uring.Res API #839.
MDX reads a markdown document and runs all the examples in it, checking that they give the expected result. However, if a test hangs then MDX doesn't tell you where the problem was, and if a test hangs in CI then you just get a message at the end of an hour saying that the tests timed out, which was making it difficult to find out why the uring tests were hanging on riscv.
I made a PR to MDX that shows the current test location on Ctrl-C, and also prints warning messages if a test is taking a long time:
Unfortunately, it doesn't work very well when used with dune yet (dune uses SIGKILL to terminate MDX, and hides output until the whole file completes by default).
The riscv hang turned out to be because the CI machines are running a very old version of Linux. The tests tried to write a byte to a pipe using uring and then read it back. The write failed because Linux was too old to support that API, but MDX continued anyway with the read.
The main problem here was getting the new C header file in the right place. The existing build process ended up with 4 copies of each header, which was pretty confusing. I simplified it a bit and documented the rest:
I tried using various flags to tell uring that it doesn't need to interrupt us whenever a completion is due, as we have an event loop and will check soon anyway:
Using IORING_SETUP_COOP_TASKRUN led to a big speed up on my machine for the HTTP tests
(from about 350k requests per second to 450k!).
However, it seemed to have no noticable effect on the CI benchmark machine.
Using IORING_SETUP_DEFER_TASKRUN had a similar effect, but caused one of the tests to hang
(my MDX hang-detector came in handy here!).
The problem is that if there are no pending completions from the kernel, but we
do have other work to do, then we do that first, but then we never enter the kernel
and the completions are never written.
Would probably be easy enough to change the scheduler to enter the kernel anyway
from time to time.
MSG_CMSG_CLOEXEC when receiving file descriptors #132.
write_fixed #133.
Reviewed and merged various PRs:
polling_timeout with GC in ocaml_uring_setup #136.
Eio_unix.Stdenv.override for updating environments #823.
send_zc and sendmsg_zc ops #139.
An ocaml-tar bug was found
that could cause it to write files outside of the target directory.
Anil Madhavapeddy made a quick fix for the tar-eio package and I reviewed it
(though I'm unfamiliar with tar-eio).
Interestingly, while the tar-unix and tar-lwt-unix versions are always vulnerable,
the unpatched version is not necessarily unsafe with tar-eio.
This is because tar-eio gets its ability to write files from its caller.
If the caller only gives it access to the output directory then the bug cannot be exploited.
However, you can forget to do that, so ensuring tar-eio does it in all cases is still a good idea.
I needed something simple to publish these work updates, without polluting my main blog, so I set up this sub-site for it. I copied the OCaml static site generator I wrote for the main blog and simplified it for this one.
]]>