Thomas Leonard's work notes

(see roscidus.com for my main blog)

(Eio and odoc)

Eio

I've been trying to update Eio's generated API documentation, and have been finding some rough edges in the tooling!

The first problem is that I was using DUNE_CACHE=enabled, which seems to cause dune build @doc to produce incorrect results. Setting it to DUNE_CACHE=disabled fixed that. This turned out to be a known bug:

  • dune build @doc does not rebuild pages #7720.

I then added a custom index.mld for the eio library, as the auto-generated default is very confusing (e.g. it includes private libraries like eio.core). The new index should make it much easier to find the module you want:

  • Add index.mld for eio #926.

Also, the search bar is only generated if you happen to have sherlodoc installed, so it's easy to remove it accidentally when making a new release. I added a hard dependency on it from my fork of dune-release to avoid that in future.

Next, I wanted to fix some warnings odoc was reporting about missing roots. dune build @doc doesn't handle links between packages, but the undocumented dune build @doc-new is supposed to. However, some cases are difficult. For example, some text in the eio package suggests using Eio_main.run, but eio doesn't depend on eio_main, so you get the error unresolvedroot(Eio_main). I found a dune issue that said you can mark a dependency as with-doc for this case:

  • Write odoc configuration files #13476.

When you do this, dune adds the dependency to the generated opam file and also auto-generates the odoc-config.sexp files for odoc. Eio already had manual versions of the odoc-config files, which dune prompted me to remove. Unfortunately, it looks like the auto-generated file can only depend on packages, not plain libraries, which means we can't reference the Unix module in the docs! I made an example repository to demonstrate the various problems and filed an issue:

  • odoc: can't link between packages #16363.

Also, @doc-new doesn't actually use the odoc-config files it creates, so even the link to eio_main doesn't work!

It turns out there's a third (undocumented) way to build the docs, which avoids some of these problems:

  • Document "dune mode" for odoc_driver #1492.

For now, I've added a make docs rule to Eio with the best known current solution:

  • Fix odoc problems #939.

Patrick had previously opened a PR adding environment variables to Eio's Stdenv:

  • Add Eio.Vars.t to Stdenv #923.

Reviewing that, I realised we needed a better way to use the returned environment with Eio.Process.spawn so I added a new module to deal with that:

  • Add Process.Env #930.

Environment variables are a bit tricky because although they're usually used as a mapping from names to values, technically the environment is just a list of strings and we don't want to mess up existing entries. Also, things are difficult on Windows (of course) because there names are case-insensitive (at least for ASCII characters).

Patrick also noticed that searching the environment array using OCaml is considerably slower than doing the same thing from C. Looks like C is doing some clever tricks to speed things up, but also OCaml's String.starts_with is a little inefficient.

Thomas Gazagnaire requested a ?follow argument to reject symlinks when opening paths, which I implemented:

  • Eio.Path: add ?follow arguments to open functions #938.

This turned up a bug in the Windows error reporting. It turns out that reporting caml_unix_error(ELOOP, "nofollow", Nothing) doesn't actually get you a Unix.ELOOP error. Instead, you need to report it as -WSAELOOP!

Thomas also noted, when working around this by opening the file with a C stub, that it was still awkward because you can't import a Unix.file_descr as an Eio.File, so I added support for that too:

  • Add Eio_unix.File module #937.

Anil has been working on Windows support for pipes, processes and paths, and I've been reviewing that:

  • eio_windows: fix anonymous pipes on Windows #929.
  • Fix NT path handling and stat in eio_windows #932.
  • Windows: add rename support #934.
  • Windows: process spawning support #936.

Anil noticed a flaky test in CI and I was able to reproduce it on my machine. It turned out to be a bug in the Linux kernel, and is due to be fixed in Linux 7.4:

  • eio_linux: spawn sometimes fails with Unix.EUNKNOWNERR 26 #935.

Minor PR reviews:

  • Eio.Fiber.List.partition_map #927.
  • posix: add device mknod support #911.

Other

Reviewed a PR from Jeong-Sik Yun fixing a nasty bug in cohttp-eio:

  • cohttp-eio: continue a partial body delivery from the position already delivered #1149.

Updated capnp-rpc to the latest crypto APIs:

  • Update to X.509 1.2.0 API #323.

I also had a look at unhandled (see Unhandled: a static checker for unhandled effects in OCaml 5, and a negative result). I think it would be good if there were a general tool to extract program flow information from OCaml code and output it as Datalog facts. Then you could write your own analyses like this quite easily, I think.