Greenplum是一款全球领先的开源大数据平台,为全球各行各业提供具备实时处理、弹性扩容、弹性计算、混合负载、云原生和集成数据分析能力的强大的大数据引擎,目前广泛的应用于包括金融、保险、证券、通信、航空、物流、零售、媒体、政府、医疗、制造、能源等行业。
目前Greenplum的二进制发行版本只能运行在X86服务器。无论是Greenplum的官方开发商Pivotal公司,还是其他Greenplum发行商,都没有提供Greenplum的ARM发行版。不过,Greenplum是开源软件,我们可以通过编译Greenplum源代码自行构建Greenplum的ARM版本。
本文将详细讲述如何在ARM服务器上编译并运行开源版Greenplum。
一 环境准备
# yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
# yum makecache
- 关闭防火墙
# systemctl stop firewalld
# systemctl disable firewalld
- 配置Linux内核,修改/etc/sysctl.conf文件,添加下列内容
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 250 512000 100 2048
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_forward = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.defalut.arp_filter = 1
net.ipv4.ip_local_port_range = 1025 65535
net.core.netdev_max_backlog = 10000
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
vm.overcommit_memory = 2
- 修改/etc/security/limits.conf文件
* soft nofile 524288
* hard nofile 524288
* soft nproc 131072
* hard nproc 131072
- 关闭SELinux,编辑/etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
- 创建gpadmin用户
# useradd gpadmin
# passwd gpadmin
- 安装编译所需的工具及软件包
# yum groupinstall 'Development Tools' # GCC, libtools etc
# yum install curl-devel bzip2-devel python-devel openssl-devel readline-devel perl-ExtUtils-Embed libxml2-devel perl-devel zstd git
# wget https://bootstrap.pypa.io/get-pip.py
# python get-pip.py
# pip install psutil lockfile paramiko setuptools epydoc conan
- 重启服务器,是配置生效
# reboot
- 使用gpadmin登入,配置ssh免密码登录
$ ssh-keygen -t rsa
$ ssh-copy-id -i ~/.ssh/id_rsa gpadmin@localhost
编译Greenplum源代码
- 下载Greenplum源代码
$ git clone https://github.com/greenplum-db/gpdb
- 编译
$ cd gpdb
$ git checkout 6X_STABLE #假设编译6.X版本
$ CFLAGS="-O0 -g3 -ggdb3" \
./configure --with-perl --with-python --with-libxml --enable-debug --enable-cassert \
--disable-orca --disable-gpcloud --disable-gpfdist \
--disable-gpfdist
$ make
$ sudo make install
编译安装完成
总结
由此可见,Greenplum作为一款为通用平台设计的MPP数据库,在ARM服务器上的编译安装流程和在普通X86服务器上几乎没有区别,感兴趣的小伙伴可以在ARM服务器上尝试一下Greenplum。