WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: Setting up distcc  (Read 7360 times)

Online Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11639
Re: Setting up distcc
« Reply #15 on: January 25, 2022, 01:22:38 PM »
Hi polikuo
In that case, forget what I said and stick with your working version.

Offline polikuo

  • Hero Member
  • *****
  • Posts: 737
Re: Setting up distcc
« Reply #16 on: January 25, 2022, 11:33:12 PM »
Hi

Just wanna report that, with distcc, I can see some binaries increase in size.

Code: [Select]
tc@pi4:/tmp$ du */liblto_plugin.so.0.0.0
132K    distcc-build/liblto_plugin.so.0.0.0
112K    distcc-stripped/liblto_plugin.so.0.0.0
96K     local-build/liblto_plugin.so.0.0.0
76K     local-stripped/liblto_plugin.so.0.0.0

This is disappointing  :(

Online Juanito

  • Administrator
  • Hero Member
  • *****
  • Posts: 14823
Re: Setting up distcc
« Reply #17 on: January 26, 2022, 12:38:14 AM »
Indeed  :(

Online Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11639
Re: Setting up distcc
« Reply #18 on: January 26, 2022, 01:21:42 AM »
Hi polikuo
Maybe this is the reason for the size increase:
... I usually compile with LTO enabled.
When I remove that from the flags, pi3 receives the packages. ...

Online Juanito

  • Administrator
  • Hero Member
  • *****
  • Posts: 14823
Re: Setting up distcc
« Reply #19 on: January 26, 2022, 01:48:52 AM »
The last time I tried, gcc would not compile itself with -flto.

Offline polikuo

  • Hero Member
  • *****
  • Posts: 737
Re: Setting up distcc
« Reply #20 on: January 26, 2022, 02:29:44 AM »
No, just the regular flags
Code: [Select]
CFLAGS='-march=armv8-a+crc -mtune=cortex-a72 -Os -pipe'
CXXFLAGS='-march=armv8-a+crc -mtune=cortex-a72 -Os -pipe'
LDFLAGS='-Wl,-O1'

Offline polikuo

  • Hero Member
  • *****
  • Posts: 737
Re: Setting up distcc
« Reply #21 on: January 26, 2022, 02:37:36 AM »
OK I see the problem

You need to export the flags on the Sever machines as well.

Right before you start the daemon:
Code: [Select]
tc@pi3:~$ export CFLAGS="-march=armv8-a+crc -mtune=cortex-a72 -Os -pipe"
tc@pi3:~$ export CXXFLAGS="-march=armv8-a+crc -mtune=cortex-a72 -Os -pipe"
tc@pi3:~$ export LDFLAGS="-Wl,-O1"
tc@pi3:~$ distccd --daemon --allow 192.168.0.40 --log-file=/var/log/distccd.log

Results:
Code: [Select]
tc@pi4:10.2.0$ du liblto_plugin.so.0.0.0
96K     liblto_plugin.so.0.0.0
tc@pi4:10.2.0$ strip --strip-unneeded liblto_plugin.so.0.0.0
tc@pi4:10.2.0$ du liblto_plugin.so.0.0.0
76K     liblto_plugin.so.0.0.0

I'll add the flags to the startup script.
« Last Edit: January 26, 2022, 02:39:44 AM by polikuo »

Offline polikuo

  • Hero Member
  • *****
  • Posts: 737
Re: Setting up distcc
« Reply #22 on: February 12, 2022, 04:38:33 AM »
Hi Juanito

This is the first time I compile LLVM and I'm following your instructions here.

I have a few questions, but first I spot a typo here:
The comma is gone, so nothing is passed to the linker
Quote
find . -name link.txt -type f -exec sed -i 's/-Wl,-O3/-Wl -Os/g' {} \;

Anyway~
I can't find all the files listed above so I decide to "grep" my way through.
There are only 3 files containing "-O3" or "-g" flags.
Code: [Select]
sed -i.orig -e 's:-Wl,-O3:-Wl,-O1:g' -e 's: -O3 : :g' -e 's: -g : :g' build.ninja CMakeCache.txt compile_commands.json
Quote
[libexecdir needs setting]
[-DLLVM_ENABLE_LTO=Full]
I didn't touch libexecdir and I removed LTO so jobs can be distributed.

Distcc sure helps a lot with the compile time.
Code: [Select]
time ninja -j8
real    5h 38m 17s
user    16h 17m 23s
sys     44m 3.42s

BTW, you can specify DESTDIR with ninja
Code: [Select]
DESTDIR=/tmp/packing-llwm ninja install
I'm having troubles when packing things up.
What should I do with this ?
Quote
[links against ncursesw without ncursesw-dev]

What about clang ?
It looks like llvm should be packed into multiple packages and I'm not sure what are they.

P.S. I'm pretty sure -Wl,-O[1-9] is identical to -Wl,-O1 so you can just ignore that.
« Last Edit: February 12, 2022, 04:46:25 AM by polikuo »

Online Juanito

  • Administrator
  • Hero Member
  • *****
  • Posts: 14823
Re: Setting up distcc
« Reply #23 on: February 12, 2022, 05:07:06 AM »
The comma is gone, so nothing is passed to the linker
Quote
find . -name link.txt -type f -exec sed -i 's/-Wl,-O3/-Wl -Os/g' {} \;
Well spotted  :)

Quote
What should I do with this ?
Quote
[links against ncursesw without ncursesw-dev]
That was a note to remind me to add ncursesw to the dep file

Quote
What about clang ?
It looks like llvm should be packed into multiple packages and I'm not sure what are they.
See llvm-bin, llvm-dev, llvm-lib and clang

Offline polikuo

  • Hero Member
  • *****
  • Posts: 737
Re: Setting up distcc
« Reply #24 on: February 12, 2022, 08:47:39 AM »
Hi Juanito
I'm not quite sure which binaries should be in llvm-bin, llvm-dev or clang
It's pretty obvious for those with "llvm-" or "clang-" prefix, but the rest is messy.
Different port also has different layout. (comparing aarch64 and x86_64)
Any suggestions ?
Code: [Select]
llvm-root/usr/local/bin/analyze-build
llvm-root/usr/local/bin/bugpoint
llvm-root/usr/local/bin/c-index-test
llvm-root/usr/local/bin/diagtool
llvm-root/usr/local/bin/dsymutil
llvm-root/usr/local/bin/hmaptool
llvm-root/usr/local/bin/intercept-build
llvm-root/usr/local/bin/llc
llvm-root/usr/local/bin/lli
llvm-root/usr/local/bin/opt
llvm-root/usr/local/bin/sancov
llvm-root/usr/local/bin/sanstats
llvm-root/usr/local/bin/scan-build
llvm-root/usr/local/bin/scan-build-py
llvm-root/usr/local/bin/scan-view
llvm-root/usr/local/bin/split-file
llvm-root/usr/local/bin/verify-uselistorder

Online Juanito

  • Administrator
  • Hero Member
  • *****
  • Posts: 14823
Re: Setting up distcc
« Reply #25 on: February 12, 2022, 09:07:06 AM »
I think I originally looked what debian did - since several utilities have been added subsequently I should have checked back before now…

Offline polikuo

  • Hero Member
  • *****
  • Posts: 737
Re: Setting up distcc
« Reply #26 on: February 14, 2022, 12:02:30 AM »
Hi Juanito

Just mailed the extensions for you to check.
I use distcc this time, see the instructions for more details.

Not sure what's causing the size difference

Code: [Select]
$ du *tcz
106M    clang.tcz
14M     llvm-bin.tcz
61M     llvm-dev.tcz
19M     llvm-lib.tcz

Code: [Select]
tc@pi4:/tmp/repo$ grep Size *info
clang.tcz.info:Size: 86MB
llvm-bin.tcz.info:Size: 11.7MB
llvm-dev.tcz.info:Size: 48MB
llvm-lib.tcz.info:Size: 15.4MB

It could be the files cause I could see many new ones not presented in the old packages

Online Juanito

  • Administrator
  • Hero Member
  • *****
  • Posts: 14823
Re: Setting up distcc
« Reply #27 on: February 15, 2022, 06:06:33 AM »
I'd say the size difference is due to:

* updates are almost always bigger (librsvg with rust 10x bigger)
* lto compiles 5-10% smaller binaries/dynamic libs

Offline polikuo

  • Hero Member
  • *****
  • Posts: 737
Re: Setting up distcc
« Reply #28 on: February 15, 2022, 07:19:27 AM »
I use distcc this time, see the instructions for more details.
Not sure what's causing the size difference
I'd say the size difference is due to:

* updates are almost always bigger (librsvg with rust 10x bigger)
* lto compiles 5-10% smaller binaries/dynamic libs
I have verified with a local build, it produces the same result as a distcc build.
Not sure why, but -DLLVM_ENABLE_LTO=Full does not compile for me.