• 作成:
  • 更新:

FirefoxがGCCとClangのどちらでビルドした方が速いか私のGentoo Linux環境で測定してみました

結論

あまり変わらないのでClangを使うことにしました.

FirefoxがGCCではなくClangでビルドされるようになりました

次のナイトリーからTier-1プラットフォームのFirefoxすべてがLLVM Clangでビルドされることになると伝えた。

Firefox全プラットフォームでLLVM Clang利用、性能向上を確認 | マイナビニュース

そしてFirefox 64からは正式にClangで全てのプラットフォームに提供されるようになりました.

Improved performance for Mac and Linux users, by enabling link time optimization (Clang LTO). (Clang LTO was enabled for Windows users in Firefox 63.)

Firefox 64.0, See All New Features, Updates and Fixes

しかし実はGentooのデフォルト設定だとGCCでビルドされるままなんですよね.Gentooの特性上コンパイラ依存を増やしたくないことはわかるのですが.

そこで私は実際Clangでビルドして性能が向上するか測定してみてから移行することにしました.

Linuxユーザはあまり重要視されないのでGCCの方が速いままという可能性も十分ありうるとも思ったので.

また,私が使っている拡張機能を有効にした状態の実際の速度も見てみたいと思いました.

Firefoxのバージョン

Mozilla/5.0 (X11; Linux x86_64; rv:64.0) Gecko/20100101 Firefox/64.0

FirefoxのUSEフラグ

clangを有効にする前は

custom-cflags custom-optimization dbus gmp-autoupdate lto pulseaudio screenshot startup-notification -bindist -clang -debug -eme-free -geckodriver -hardened -hwaccel -jack -neon -selinux -system-harfbuzz -system-icu -system-jpeg -system-libevent -system-libvpx -system-sqlite -test -wifi

about:buildconfig

gcc

target
x86_64-pc-linux-gnu

Build tools

Compiler Version Compiler flags
/usr/bin/x86_64-pc-linux-gnu-gcc -std=gnu99 8.2.0 -pipe -fomit-frame-pointer -march=native -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-strict-aliasing -ffunction-sections -fdata-sections -fno-math-errno -pthread -pipe
/usr/bin/x86_64-pc-linux-gnu-g++ 8.2.0 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wsign-compare -Wtype-limits -Wunreachable-code -Wwrite-strings -Wno-invalid-offsetof -Wc++1z-compat -Wduplicated-cond -Wimplicit-fallthrough -Wno-error=maybe-uninitialized -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=free-nonheap-object -Wno-error=multistatement-macros -Wno-error=class-memaccess -Wformat -Wformat-security -Wformat-overflow=2 -fno-sized-deallocation -pipe -fomit-frame-pointer -march=native -fno-delete-null-pointer-checks -fno-lifetime-dse -fno-schedule-insns -fno-schedule-insns2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -O2 -fomit-frame-pointer -funwind-tables

Configure options

PYTHON3=/usr/bin/python3.6 --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu --enable-application=browser --disable-tests PKG_CONFIG=x86_64-pc-linux-gnu-pkg-config --enable-optimize=-O2 CC=x86_64-pc-linux-gnu-gcc LD=x86_64-pc-linux-gnu-ld CXX=x86_64-pc-linux-gnu-g++ HOST_CC=x86_64-pc-linux-gnu-gcc HOST_CXX=x86_64-pc-linux-gnu-g++ --disable-debug-symbols --enable-lto=thin --enable-linker=gold --enable-system-ffi --enable-default-toolkit=cairo-gtk3 --with-google-api-keyfile=/var/tmp/portage/www-client/firefox-64.0/work/firefox-64.0/google-api-key MAKE=/usr/bin/gmake XARGS=/usr/bin/xargs --disable-crashreporter --enable-dbus --enable-extensions=default --disable-gconf --disable-install-strip --disable-necko-wifi --enable-official-branding --enable-startup-notification --disable-strip --enable-system-pixman --disable-system-sqlite --disable-updater --libdir=/usr/lib64 --prefix=/usr --with-intl-api --with-nspr-prefix=/usr --with-nss-prefix=/usr --with-system-bz2 --without-system-icu --without-system-jpeg --without-system-libvpx --with-system-nspr --with-system-nss --with-system-png --with-system-zlib --x-includes=/usr/include --x-libraries=/usr/lib64

clang

target
x86_64-pc-linux-gnu

Build tools

Compiler Version Compiler flags
/usr/lib/llvm/7/bin/x86_64-pc-linux-gnu-clang -std=gnu99 7.0.1 -pipe -fomit-frame-pointer -march=native -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-strict-aliasing -ffunction-sections -fdata-sections -fno-math-errno -pthread -pipe
/usr/lib/llvm/7/bin/x86_64-pc-linux-gnu-clang++ 7.0.1 -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Qunused-arguments -Wall -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wwrite-strings -Wno-invalid-offsetof -Wclass-varargs -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wc++1z-compat -Wc++2a-compat -Wcomma -Wimplicit-fallthrough -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-enum-zero-compare -Wtautological-unsigned-zero-compare -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=return-std-move -Wno-error=atomic-alignment -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -Wno-unknown-warning-option -Wno-return-type-c-linkage -fno-sized-deallocation -pipe -fomit-frame-pointer -march=native -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -O2 -fomit-frame-pointer -funwind-tables

Configure options

PYTHON3=/usr/bin/python3.6 --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu --enable-application=browser --disable-tests PKG_CONFIG=x86_64-pc-linux-gnu-pkg-config --enable-optimize=-O2 CC=x86_64-pc-linux-gnu-clang LD=x86_64-pc-linux-gnu-ld CXX=x86_64-pc-linux-gnu-clang++ HOST_CC=x86_64-pc-linux-gnu-gcc HOST_CXX=x86_64-pc-linux-gnu-g++ --disable-debug-symbols --enable-lto=thin --enable-linker=lld --enable-system-ffi --enable-default-toolkit=cairo-gtk3 --with-google-api-keyfile=/var/tmp/portage/www-client/firefox-64.0/work/firefox-64.0/google-api-key --disable-elf-hack MAKE=/usr/bin/gmake XARGS=/usr/bin/xargs --disable-crashreporter --enable-dbus --enable-extensions=default --disable-gconf --disable-install-strip --disable-necko-wifi --enable-official-branding --enable-startup-notification --disable-strip --enable-system-pixman --disable-system-sqlite --disable-updater --libdir=/usr/lib64 --prefix=/usr --with-intl-api --with-nspr-prefix=/usr --with-nss-prefix=/usr --with-system-bz2 --without-system-icu --without-system-jpeg --without-system-libvpx --with-system-nspr --with-system-nss --with-system-png --with-system-zlib --x-includes=/usr/include --x-libraries=/usr/lib64

ベンチマークの選択参考

Firefox 60の性能は1年前とは別物 Chromeを視界に捉える - Mozilla Fluxから取得しました.

Performance sheriffing/Talos/Tests - MozillaWikiを使うことも考えたのですが,結構セットアップが面倒そうなのでやめました.

色々走らせてみたのですが内容とかデータ出力とか考えてKrakenとSpeedometerだけで良いと判断しました.

Kraken

Kraken JavaScript Benchmark (version 1.1)

Mozilla製のJavaScriptベンチマーク.

gcc

https://krakenbenchmark.mozilla.org/kraken-1.1/results.html?%7B%22v%22:%20%22kraken-1.1%22,%20%22ai-astar%22:%5B66,64,56,65,67,66,64,69,65,70%5D,%22audio-beat-detection%22:%5B85,82,84,84,86,80,78,78,85,92%5D,%22audio-dft%22:%5B77,80,82,81,69,65,67,67,90,90%5D,%22audio-fft%22:%5B60,57,58,56,57,56,56,59,65,61%5D,%22audio-oscillator%22:%5B53,53,54,53,190,51,53,54,55,54%5D,%22imaging-gaussian-blur%22:%5B67,73,66,65,64,67,67,67,68,65%5D,%22imaging-darkroom%22:%5B93,97,92,91,93,93,94,94,95,94%5D,%22imaging-desaturate%22:%5B54,55,52,52,51,52,52,54,53,53%5D,%22json-parse-financial%22:%5B31,38,33,32,34,32,33,33,32,31%5D,%22json-stringify-tinderbox%22:%5B26,27,26,27,28,278,25,33,27,25%5D,%22stanford-crypto-aes%22:%5B63,56,54,63,61,53,56,58,55,62%5D,%22stanford-crypto-ccm%22:%5B178,120,75,74,75,76,68,92,77,76%5D,%22stanford-crypto-pbkdf2%22:%5B104,104,103,101,99,102,108,141,106,103%5D,%22stanford-crypto-sha256-iterative%22:%5B39,40,38,38,39,41,39,40,41,39%5D%7D

===============================================
RESULTS (means and 95% confidence intervals)
-----------------------------------------------
Total:                        945.0ms +/- 5.8%
-----------------------------------------------

  ai:                          65.2ms +/- 4.2%
    astar:                     65.2ms +/- 4.2%

  audio:                      285.7ms +/- 10.9%
    beat-detection:            83.4ms +/- 3.6%
    dft:                       76.8ms +/- 8.7%
    fft:                       58.5ms +/- 3.5%
    oscillator:                67.0ms +/- 46.1%

  imaging:                    213.3ms +/- 1.6%
    gaussian-blur:             66.9ms +/- 2.6%
    darkroom:                  93.6ms +/- 1.3%
    desaturate:                52.8ms +/- 1.7%

  json:                        85.1ms +/- 66.4%
    parse-financial:           32.9ms +/- 4.4%
    stringify-tinderbox:       52.2ms +/- 108.7%

  stanford:                   295.7ms +/- 9.1%
    crypto-aes:                58.1ms +/- 4.7%
    crypto-ccm:                91.1ms +/- 26.7%
    crypto-pbkdf2:            107.1ms +/- 8.1%
    crypto-sha256-iterative:   39.4ms +/- 1.9%

clang

https://krakenbenchmark.mozilla.org/kraken-1.1/results.html?%7B%22v%22:%20%22kraken-1.1%22,%20%22ai-astar%22:%5B67,68,65,63,63,88,62,63,61,62%5D,%22audio-beat-detection%22:%5B80,83,82,85,85,82,83,80,78,85%5D,%22audio-dft%22:%5B86,123,91,78,83,71,75,81,79,79%5D,%22audio-fft%22:%5B70,58,58,56,57,58,65,57,56,57%5D,%22audio-oscillator%22:%5B77,269,53,53,52,51,53,52,63,52%5D,%22imaging-gaussian-blur%22:%5B66,67,66,66,92,69,67,67,68,68%5D,%22imaging-darkroom%22:%5B93,94,91,95,93,92,93,93,92,93%5D,%22imaging-desaturate%22:%5B53,52,52,55,52,54,53,52,53,53%5D,%22json-parse-financial%22:%5B30,33,30,33,29,213,28,30,29,28%5D,%22json-stringify-tinderbox%22:%5B26,27,26,27,27,27,27,26,27,26%5D,%22stanford-crypto-aes%22:%5B54,51,61,60,62,65,56,62,69,56%5D,%22stanford-crypto-ccm%22:%5B70,77,74,77,71,78,72,86,82,71%5D,%22stanford-crypto-pbkdf2%22:%5B98,102,106,100,97,100,99,101,101,98%5D,%22stanford-crypto-sha256-iterative%22:%5B40,41,46,40,39,39,38,43,38,38%5D%7D

===============================================
RESULTS (means and 95% confidence intervals)
-----------------------------------------------
Total:                        935.9ms +/- 7.4%
-----------------------------------------------

  ai:                          66.2ms +/- 8.6%
    astar:                     66.2ms +/- 8.6%

  audio:                      303.6ms +/- 19.2%
    beat-detection:            82.3ms +/- 2.1%
    dft:                       84.6ms +/- 12.3%
    fft:                       59.2ms +/- 5.5%
    oscillator:                77.5ms +/- 62.5%

  imaging:                    215.4ms +/- 2.6%
    gaussian-blur:             69.6ms +/- 8.1%
    darkroom:                  92.9ms +/- 0.8%
    desaturate:                52.9ms +/- 1.3%

  json:                        74.9ms +/- 55.4%
    parse-financial:           48.3ms +/- 85.7%
    stringify-tinderbox:       26.6ms +/- 1.4%

  stanford:                   275.8ms +/- 3.0%
    crypto-aes:                59.6ms +/- 6.5%
    crypto-ccm:                75.8ms +/- 4.9%
    crypto-pbkdf2:            100.2ms +/- 1.8%
    crypto-sha256-iterative:   40.2ms +/- 4.6%

総評

総合では微妙に速くなってますが誤差レベル.

Speedometer

Speedometer 2.0

React, Vueなどを使ったTodoアプリを走らせてDOMのベンチマークをしてくれるようです.これ実行するにはublock originを無効にする必要がありました.

gcc

Iteration 1    56.23 runs/min
Iteration 2     54.60 runs/min
Iteration 3     55.77 runs/min
Iteration 4     52.69 runs/min
Iteration 5     54.60 runs/min
Iteration 6     47.14 runs/min
Iteration 7     54.33 runs/min
Iteration 8     53.36 runs/min
Iteration 9     52.80 runs/min
Iteration 10    50.55 runs/min
Arithmetic Mean:53.2 ± 1.9 (3.6%)

clang

Iteration 1    60.68 runs/min
Iteration 2     58.18 runs/min
Iteration 3     57.04 runs/min
Iteration 4     56.94 runs/min
Iteration 5     57.46 runs/min
Iteration 6     54.97 runs/min
Iteration 7     56.74 runs/min
Iteration 8     57.46 runs/min
Iteration 9     56.86 runs/min
Iteration 10    57.38 runs/min
Arithmetic Mean:57.4 ± 1.0 (1.8%)

総評

速くなっています.誤差レベルではありますが.

感想と決定

私の環境ではどちらでもあまり変わらないことがわかりました.

どちらでも変わらないならClangでビルドすることにします.これからのFirefoxの開発はClangでビルドすることが前提になることが予想できるので,今変わらなくてもこれからどっち向けの最適化をするかを考えればClangだと容易に予測できるためです.