Contents
概要
機械学習機能は、Elasticsearch5.x 系ではベータ、6.1 で追加、7.x では既定で有効になっているので最新のハードウェアを持っている方は気にしなくても良いのですが古いハードウェアで Linux を動かしているような場合、出ばなをくじかれることになります。
環境
- Ubuntu 18.04 LTS
インストールメモ
$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - $ sudo apt -y install apt-transport-https $ echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list $ sudo apt update $ sudo apt upgrade $ sudo apt -y install elasticsearch $ sudo vi /etc/elasticsearch/elasticsearch.yml $ sudo vi /etc/elasticsearch/jvm.options $ sudo ufw allow 9200 $ sudo ufw allow 9300 $ sudo systemctl enable elasticsearch.service $ sudo systemctl restart elasticsearch.service $ curl http://1.2.3.1:9200/_cluster/health?pretty
問題
エラーログ
[2019-07-19T23:08:40,569][ERROR][o.e.b.Bootstrap ] [master-node1] Exception org.elasticsearch.ElasticsearchException: Failed to create native process factories for Machine Learning at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:455) ~[?:?] at org.elasticsearch.node.Node.lambda$new$9(Node.java:438) ~[elasticsearch-7.2.0.jar:7.2.0] at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?] at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?] at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?] at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?] at org.elasticsearch.node.Node.(Node.java:441) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.node.Node.(Node.java:251) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:221) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) [elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-7.2.0.jar:7.2.0] at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-7.2.0.jar:7.2.0] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) [elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-7.2.0.jar:7.2.0] Caused by: java.io.FileNotFoundException: /tmp/elasticsearch-4767962642349373813/controller_log_12106 (No such file or directory) at java.io.FileInputStream.open0(Native Method) ~[?:?] at java.io.FileInputStream.open(FileInputStream.java:213) ~[?:?] at java.io.FileInputStream.(FileInputStream.java:155) ~[?:?] at java.io.FileInputStream.(FileInputStream.java:110) ~[?:?] at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:288) ~[?:?] at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:277) ~[?:?] at java.security.AccessController.doPrivileged(AccessController.java:310) ~[?:?] at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:130) ~[?:?] at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:97) ~[?:?] at org.elasticsearch.xpack.ml.process.ProcessPipes.connectStreams(ProcessPipes.java:131) ~[?:?] at org.elasticsearch.xpack.ml.process.NativeController.(NativeController.java:62) ~[?:?] at org.elasticsearch.xpack.ml.process.NativeControllerHolder.getNativeController(NativeControllerHolder.java:40) ~[?:?] at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:440) ~[?:?] ... 20 more
libMlCore.so ライブラリ参照時にコアを吐く
controller プロセスのバージョンチェックをすると・・・
# /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller --version si_signo 4, si_code: 2, si_errno: 0, address: 0x7fe9978399a2, library: /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/../lib/libMlCore.so, base: 0x7fe99778e000, normalized address: 0xab9a2 Illegal instruction (core dumped)
制限事項について
Elasticsearch の機械学習( Machine learning )は SSE ( Streaming SIMD Extensions ) 4.2 命令を利用するため、ML ノードの CPU は SSE4.2 をサポートしていることがハードウェア要件となります。
古いハードウェアで Elasticsearch 7.2 を実行したい場合は、xpack.ml.enabled を false に設定し機械学習を無効にする必要があります。
CPU が対応する命令セットの確認方法
利用している CPU が SSE 4.2 命令セットに対応しているかは以下のコマンドを実行してみると確認できます。何も返されなければ対応していないということになります。
# cat /proc/cpuinfo | grep sse4_2
xpack.ml.enabled
Elasticsearch クラスタで機械学習機能を利用したい場合、少なくとも master-eligible ノードでは xpack.ml.enabled にする必要があります。
対応策
/etc/elasticsearch/elasticsearch.yml に以下のようなエントリを追記しました。
# ---------------------------------- X-Pack ----------------------------------- # Machine learning settings xpack.ml.enabled: false
参考
- Machine Learning Limitations https://www.elastic.co/guide/en/elastic-stack-overview/master/ml-limitations.html
- Elasticsearch 7.2.0 – Unable to run xpack ml libraries https://discuss.elastic.co/t/elasticsearch-7-2-0-unable-to-run-xpack-ml-libraries/190383/4