GHC activities report: April–May 2022

Ben Gamari, Andreas Klebinger, Matthew Pickering, Zubin Duggal, Sam Derbyshire, Adam Gundry, Douglas Wilson – Thursday, 23 June 2022

all community ghc ghc-activities-report well-typed  

This is the twelth edition of our GHC activities report, which describes the work on GHC and related projects that we are doing at Well-Typed. The current edition covers roughly the months of April and May 2022.

You can find the previous editions collected under the ghc-activities-report tag.

A bit of background: One aspect of our work at Well-Typed is to support GHC and the Haskell core infrastructure. Several companies, including IOHK and GitHub via the Haskell Foundation, are providing us with funding to do this work. We are also working with Hasura on better debugging tools. We are very grateful on behalf of the whole Haskell community for the support these companies provide.

If you are interested in also contributing funding to ensure we can continue or even scale up this kind of work, please get in touch.

Of course, GHC is a large community effort, and Well-Typed’s contributions are just a small part of this. This report does not aim to give an exhaustive picture of all GHC work that is ongoing, and there are many fantastic features currently being worked on that are omitted here simply because none of us are currently involved in them in any way. Furthermore, the aspects we do mention are still the work of many people. In many cases, we have just been helping with the last few steps of integration. We are immensely grateful to everyone contributing to GHC. Please keep doing so (or start)!

Team

The current GHC team consists of Ben Gamari, Andreas Klebinger, Matthew Pickering, Zubin Duggal and Sam Derbyshire.

Many others within Well-Typed, including Adam Gundry, Alfredo Di Napoli, Alp Mestanogullari, Douglas Wilson and Oleg Grenrus, are contributing to GHC more occasionally.

Releases

  • Ben, with help from Matt, did quite a bit of release wrangling on the 9.4 branch, producing two alpha releases with a third in the pipeline.

  • Matt has worked on the release management scripts so that all release artifacts are generated by a single CI pipeline which makes it easier to debug mistakes in the release process.

  • Zubin released GHC 9.2.3.

Typechecker

  • Sam helped new contributor CarrieMY improve the typechecking of record updates. This allows record updates involving existentials to work properly, fixing tickets #2595 #3632 #10808 #10856 #16501 #18311 #18802 #21158 and #21289.

  • Sam improved the typechecking of partially applied functions which have representation-polymorphic arguments, such as coerce, unboxed tuples and sums, and newtype constructors with -XUnliftedNewtypes, fixing #21544 and #21650.

  • Sam finished refactoring the representation-polymorphism checks in the typechecker, making use of a new flavour of metavariables (concrete metavariables). This simplifies the implementation and provides the final stepping stone to allow rewriting in RuntimeReps.

  • Matt fixed a compiler panic which was caused by not properly enforcing the Template Haskell level invariants for Typeable instances (!8264).

  • Matt debugged an apparent infinite loop in the compiler which just turned out to be the result of attempting to print out a very large coercion. Removing the print happily fixes the issue (!8263).

  • Matt wrote the Deep Subsumption proposal which aims to reduce the breakage from the simplified subsumption proposal.

Code generation

  • Ben finished his rework of the migration to a clang-based toolchain on Windows (#21019).

  • Ben fixed a calling-convention bug on Windows potentially leading to undefined behavior in programs using SIMD extensions on x86-64 (#21465).

  • Ben finished, committed, and backported his fix for #20959, a correctness bug in GHC’s CAF analysis.

  • Ben fixed the AArch64 NCG’s implementation of the IntMulMayOflo machop and extended the test-primops testsuite to exercise the operation (#21624).

  • Andreas changed dataToTag# code generation to take advantage of the tag inference. In short this means we can sometimes avoid a redundant eval on the argument to dataToTag#.

Core

  • Sam made several improvements to the treatment of casts in the simplifier and the simple optimiser. This was motivated by the work on allowing rewriting in RuntimeReps.

  • Sam fixed a long-standing bug in pushCoercionIntoLambda.

  • Ben continued work on his refactoring removing the somewhat-magical GHC.PrimOpWrappers module (#20155)

  • Matt and Andreas debugged a panic in the optimiser which was caused by doing an incorrect eta-expansion (#21694).

  • Andreas added logic to dump files in !7998 which means dumps for different ways will now be written to different files instead of silently overwriting each other. The old behaviour is still available by passing -fno-dump-with-ways.

  • Andreas fixed a bug where the SpecConstr pass could generate invalid core leading to segfaults in !8096.

  • Andreas worked on reworking some invariants around core unfoldings and call-by-value argument passing in #21497. This will fix #21472 and #21496 once implemented.

  • Andreas fixed #21685. A bug where shadowing at the core level caused the CSE pass to potentially produce invalid core or segfaults.

Runtime system

  • Ben fixed #21556, a bug in the Windows runtime system linker’s treatment of weak symbols.

  • Ben started debugging a bug in the Windows runtime system linker triggered by libc++ (#21618).

  • To aid in debugging #21618, Ben started introducing support for gdb’s JIT interface into GHC’s runtime linker (!8312).

  • Ben debugged and fixed a bug in the bytecode interpreter causing some programs having data constructors with strict fields to crash when run under the interpreter due to missing pointer tags (#21390).

  • Ben fixed a lurking bug in the runtime’s treatment of dead threads triggered when profiling is enabled (!8094, #21438).

  • Matt fixed the fallback path when clock_gettime is not available (!8424).

  • Andreas changed the runtime system to export symbols for utility macros useful for debugging in gdb (!8132).

Error messages

  • Andreas improved -dstg-lint in !7941 and !8012 which will now detect more uses of unsafeCoerce# which are guaranteed to cause segfaults.

Renamer

  • Sam fixed a bug with the computation of unused variables in mdo statements.

Driver

  • Ben introduced response file support (#16476) to GHC’s driver, eliminating a persistent source of headaches for users on Windows.

  • Ben removed GHC’s long-vestigal dependency on libtool and cleaned up some adjacent linking logic (#18826).

Foreign Function Interface

  • Sam strengthened the checks on the kind of Any appearing in foreign import and export declarations, avoiding GHC panics.

Template Haskell

  • Matt fixed the Template Haskell representation of OPAQUE pragmas to avoid a panic (!8133).

Profiling

  • Andreas added a new flag -fno-prof-manual which causes GHC to ignore user-written cost centre annotations. The intended use is to allow suppression of cost centre annotations in libraries far up the dependency chain which one might not care about.
  • Andreas fixed #21429, a bug where profiling cost centres in certain positions could cause segfaults through incorrect demand analysis.

Libraries

  • Ben introduced a meta-package, system-cxx-std-lib, giving users an easy way to bind to C++ dependencies by capturing the link dependencies necessary to link against the C++ standard library (#20010).

  • Ben fixed a long-standing but only recently-noticed bug in GHC’s treatment of file handle closure, allowing file flush failures to go unnoticed when run during handle finalization (#21336).

  • Ben fixed a memory soundness bug in the ghc-heap package’s treatment of weak pointer objects (#21622).

Compiler performance

  • Doug has been working on a “jsem” feature which will allow multiple instances of ghc --make to cooperate in sharing available cores (#19416).

  • Doug and Matt have been investigating parallelising the simplifier.

  • Andreas optimized a few cases where GHC used O(n^2) algorithms, replacing them with code running in O(n*log(n)). This should help in edge cases where modules have a very large number of exported functions.

Packaging

  • Matt has made a number of improvements to the creation of source distributions (!8371).

  • Zubin added CI jobs and modified the release script to generate and upload sources to aid in bootstrapping Hadrian to build GHC without a pre-existing cabal-install installation.

Runtime performance

  • Ben characterised the effect of !7986, a change to make IO performance more predictable in exchange for slightly higher overhead by dropping use of O_NONBLOCK on normal files. In his microbenchmark, he found the change to regress IO performance by roughly 5% (see #15153)

  • Inspired by his work on the O_NONBLOCK change, Ben rebased WJWH’s io_uring-based polling patch (!3794) and extended it to use io_uring for Handle read and write operations. This quickly ended up turning into a proof-of-concept io_uring-based IO manager implementation (!8073)

  • Ben spent some time thinking about a C– optimization to widen narrow (8- and 16-bit) integer expressions to avoid partial register stalls on x86-64.

Infrastructure

  • Ben investigated a bootstrapping failure of 9.4 with the make build system (#21188). Unfortunately, this ended up being attributable to a deficiency in the make build system’s handling of bootstrap dependencies, putting new urgency on the migration to Hadrian.

  • Matt refactored Hadrian to avoid the same bootstrap issues that have broken the make build system (!8339).

  • Ben refactored Hadrian’s treatment of build information from Cabal, fixing a good number of issues in the process (#20566, #20579).

  • Ben continued work on his branch removing the make build system from the tree in preparation for removal later this summer.

  • Ben investigated and fixed a rather perplexing set of Darwin CI failures involving bytestring’s use of ARM’s NEON extensions which ultimately were attributed to Rosetta’s unpredictable choice of execution platform (#21579).

  • Ben investigated another set of equally-bizarre CI failures on Linux, which ended up being due to a bug in Docker’s seccomp filters.