HackerPapers

arXiv:2004.11663cs.PL

Retrofitting Parallelism onto OCaml

KC Sivaramakrishnan, Stephen Dolan, Leo White, Sadiq Jaffer, Tom Kelly, Anmol Sahoo, Sudha Parimala, Atul Dhiman, Anil Madhavapeddy

Abstract

OCaml is an industrial-strength, multi-paradigm programming language, widely used in industry and academia. OCaml is also one of the few modern managed system programming languages to lack support for shared memory parallel programming. This paper describes the design, a full-fledged implementation and evaluation of a mostly-concurrent garbage collector (GC) for the multicore extension of the OCaml programming language. Given that we propose to add parallelism to a widely used programming language with millions of lines of existing code, we face the challenge of maintaining backwards compatibility--not just in terms of the language features but also the performance of single-threaded code running with the new GC. To this end, the paper presents a series of novel techniques and demonstrates that the new GC strikes a balance between performance and feature backwards compatibility for sequential programs and scales admirably on modern multicore processors.

arXiv comments
Accepted to ICFP 2020
Journal reference
DOI
doi:10.1145/3408995
Subjects
cs.PLProgramming Languages

On Hacker News

Linked by 3 identifiable commenters in 11 comments across 8 threads, from Jun 2020 to Dec 2022. Found in 4 HN submissions from Apr 2020 to May 2020. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments8 threads

  1. OCaml 5.0 Multicore is out 1 comment

    • Signals and Threads had an interview with Anil Madhavapeddy last year: https://signalsandthreads.com/what-is-an-operating-system/ He talked about the work to put a multicore-ready memory model[1] and GC[2] under OCaml. [1] https://anil.recoil.org/papers/2018-pldi-memorymodel.pdf…

      nequoView on HNParent context

  2. OCaml Multicore merged upstream 2 comments

    • It involved a fair amount of research and there were many technical challenges. Maintaining backwards compatibility in terms of language features and single-threaded performance puts constraints around what you can implement. Also from my personal experience, debugging segfaults…

      sadiqView on HNParent context

    • Along with the graphs from the PR in the sibling comment, there's also the extensive benchmarking from the ICFP2020 paper: https://arxiv.org/pdf/2004.11663.pdf Work on this is on-going via the sandmark benchmarking suite: https://github.com/ocaml-bench/sandmark In short the…

      sadiqView on HNParent context

  3. Adapting the OCaml Ecosystem for Multicore OCaml 1 comment

    • Our paper from ICFP last year has extensive benchmarks against stock OCaml: https://arxiv.org/abs/2004.11663 In short there should be very little performance difference against stock. As KC has linked in a sibling comment there's a project going to explore using effects to write…

      sadiqView on HNParent context

  4. Multicore OCaml: April 2021 1 comment

    • As the sibling comment mentions, the hard part is actually retrofitting multicore whilst maintaining compatibility _and_ performance. Our paper last year covers most of why this is tricky: https://arxiv.org/abs/2004.11663

      sadiqView on HNParent context

  5. Multicore OCaml: Feb 2021 with new preprint on Effect Handlers 1 comment

    • Multicore is almost entirely backwards compatible with stock OCaml. One issue is C libraries that use so-called naked pointers but these are discouraged anyway and 4.12 ships a detector to find these: https://discuss.ocaml.org/t/ann-a-dynamic-checker-for-detect... There are also…

      sadiqView on HNParent context

  6. Draft of OCaml Scientific Computing book 1 comment

    • Just to add to this good summary, OCaml already has some support for parallelism that would be useful for scientific computation. Essentially you can already have multi-threaded OCaml programs as long as only one thread is using the OCaml runtime at any point in time. For…

      sadiqView on HNParent context

  7. Parallel Programming in Multicore OCaml 3 comments

    • (Multicore OCaml hacker here - speaking for myself though) Nice to see people having a look at this. It's an early draft chapter that Sudha, one of the Multicore team, has been working on. If you have feedback or suggestions I'm sure she'd welcome it on the draft's PR…

      sadiqView on HNParent context

    • Better throughput, and almost no impact on latency overall. See https://arxiv.org/abs/2004.11663 section 6.2 for parallel performance evaluation.

      kcsrkView on HNParent context

    • No breaking changes with the parallel minor collector (ParMinor). This is the design which will be upstreamed. See https://arxiv.org/pdf/2004.11663.pdf for details.

      kcsrkView on HNParent context

  8. Multicore OCaml: May 2020 update 1 comment

    • We have a preprint up of our paper that details the design of Multicore OCaml along with a range of performance comparisons: https://arxiv.org/abs/2004.11663 In terms of performance it varies. The benchmarks in that paper it ranges from a 20% slowdown to a 20% speedup - most…

      sadiqView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.