commit e41c3d4bccf9b5cf584f1453e0a028fd066cd926
from: tb <tb@openbsd.org>
date: Sat May 17 20:13:13 2025 UTC

Default to a maximum of 4 threads for ld.lld

This is one of those typical 'more is better' defaults that are just bad.
Even on systems that are much better at parallelism than OpenBSD, people
noticed that 'let me use as many threads as CPUs' isn't helping anything:

https://github.com/llvm/llvm-project/commit/da68d2164efcc1f5e57f090e2ae2219056b120a0

This is the above LLVM commit (which was pointed out by mpi and others)
with s/16/4/g.

This reduces wall clock time and especially system time of builds
significantly on machines with far more CPUs than our beloved OS can
properly handle these days such as claudio's ampere box. On my 8 core
bulk builder I don't see a significant change in wall clock time but
a lot less spin during linking of monsters. Anything with more than 8
CPUs benefits from not calling sched_yield() until the cows come home.

tested by claudio, landry and myself

commit - 5d5504715671cd3636955b8ebd3cd34495f6646c
commit + e41c3d4bccf9b5cf584f1453e0a028fd066cd926
blob - eae1a4cc3eb565f88b097dd2cc553aabb1c9c777
blob + 05026ece5d8392304533269269a4aa7cf05b48a2
--- gnu/llvm/lld/ELF/Driver.cpp
+++ gnu/llvm/lld/ELF/Driver.cpp
@@ -1405,8 +1405,12 @@ static void readConfigs(opt::InputArgList &args) {
     config->mllvmOpts.emplace_back(arg->getValue());
   }
 
+  config->threadCount = parallel::strategy.compute_thread_count();
+
   // --threads= takes a positive integer and provides the default value for
-  // --thinlto-jobs=.
+  // --thinlto-jobs=. If unspecified, cap the number of threads since
+  // overhead outweighs optimization for used parallel algorithms for the
+  // non-LTO parts.
   if (auto *arg = args.getLastArg(OPT_threads)) {
     StringRef v(arg->getValue());
     unsigned threads = 0;
@@ -1415,10 +1419,12 @@ static void readConfigs(opt::InputArgList &args) {
             arg->getValue() + "'");
     parallel::strategy = hardware_concurrency(threads);
     config->thinLTOJobs = v;
+  } else if (config->threadCount > 4) {
+    log("set maximum concurrency to 4, specify --threads= to change");
+    parallel::strategy = hardware_concurrency(4);
   }
   if (auto *arg = args.getLastArg(OPT_thinlto_jobs_eq))
     config->thinLTOJobs = arg->getValue();
-  config->threadCount = parallel::strategy.compute_thread_count();
 
   if (config->ltoo > 3)
     error("invalid optimization level for LTO: " + Twine(config->ltoo));