316 lines
7.1 KiB
Markdown
316 lines
7.1 KiB
Markdown
# CDH部署
|
||
|
||
## Kerberos部署
|
||
|
||
### 系统环境
|
||
|
||
- Centos7.7
|
||
|
||
### CDH版本
|
||
|
||
- 6.3.2
|
||
|
||
### 部署用户权限
|
||
|
||
- root权限
|
||
|
||
### KDC服务安装及配置
|
||
|
||
#### 在Cloudera Manager服务器上安装KDC服务
|
||
|
||
```bash
|
||
yum install krb5-server krb5-libs krb5-auth-dialog krb5-workstation openldap-clients -y
|
||
```
|
||
|
||
- 会生成/etc/krb5.conf、/var/kerberos/krb5kdc/kadm5.acl、/var/kerberos/krb5kdc/kdc.conf三个文件。
|
||
|
||
#### 修改/etc/krb5.conf配置
|
||
|
||
```bash\
|
||
vim /etc/krb5.conf
|
||
```
|
||
|
||
- 配置文件内容
|
||
|
||
```bash
|
||
# Configuration snippets may be placed in this directory as well
|
||
includedir /etc/krb5.conf.d/
|
||
|
||
[logging]
|
||
default = FILE:/var/log/krb5libs.log
|
||
kdc = FILE:/var/log/krb5kdc.log
|
||
admin_server = FILE:/var/log/kadmind.log
|
||
|
||
[libdefaults]
|
||
dns_lookup_realm = false
|
||
ticket_lifetime = 24h
|
||
renew_lifetime = 7d
|
||
forwardable = true
|
||
rdns = false
|
||
pkinit_anchors = FILE:/etc/pki/tls/certs/ca-bundle.crt
|
||
default_realm = YLDEV.NET
|
||
#default_ccache_name = KEYRING:persistent:%{uid}
|
||
|
||
[realms]
|
||
YLDEV.NET = {
|
||
kdc = cdh-node-1
|
||
admin_server = cdh-node-1
|
||
default_domain = YLDEV.NET
|
||
}
|
||
|
||
|
||
[domain_realm]
|
||
.example.com = YLDEV.NET
|
||
example.com = YLDEV.NET
|
||
```
|
||
|
||
##### 配置参数
|
||
|
||
- default_realm: 默认realm，例如当我们使用kinit而不指定principal时使用的就是这里配置的默认realm
|
||
|
||
- dns_lookup_realm：是否可以通过DNS查找使用什么样的realm
|
||
|
||
- ticket_lifetime： 设定ticket的有效期
|
||
|
||
- forwardable： 用于指定ticket是否可以被转发，转发的含义是：如果一个用户已经有了一个TGT，当他登入到另一个远程系统，KDC会为他自动重新创建一个TGT，而不需要让用户重新进行身份认证。
|
||
|
||
- 然后是[realms]部分，这一部分会列出所有的realm，kdc和admin_server两个配置是在告诉客户端哪台服务器在运行KDC以及kadmin进程。这两项配置可以在服务器上追加端口，如果不指定，则使用默认端口，KDC是88，admin server是749.
|
||
|
||
- 最后一部分[domain_realm]是配置DNS名称和Kerberos Realm映射的。
|
||
|
||
|
||
|
||
#### 修改/var/kerberos/krb5kdc/kadm5.acl配置
|
||
|
||
将 */admin@YLDEV.NET 后缀的设为管理员
|
||
|
||
```bash
|
||
vim /var/kerberos/krb5kdc/kadm5.acl
|
||
```
|
||
|
||
```bash
|
||
*/admin@YLDEV.NET *
|
||
```
|
||
|
||
#### 修改/var/kerberos/krb5kdc/kdc.conf配置
|
||
|
||
```bash
|
||
vim /var/kerberos/krb5kdc/kdc.conf
|
||
```
|
||
|
||
```bash
|
||
[kdcdefaults]
|
||
kdc_ports = 88
|
||
kdc_tcp_ports = 88
|
||
|
||
|
||
[realms]
|
||
YLDEV.NET = {
|
||
max_life = 24h # 单个票据的最大有效期
|
||
|
||
max_renewable_life= 7d 0h 0m 0s
|
||
renewable = true
|
||
#master_key_type = aes256-cts
|
||
acl_file = /var/kerberos/krb5kdc/kadm5.acl
|
||
dict_file = /usr/share/dict/words
|
||
admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
|
||
supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
|
||
}
|
||
```
|
||
|
||
#### 创建Kerberos数据库
|
||
|
||
```bash
|
||
kdb5_util create -r YLDEV.NET -s
|
||
```
|
||
|
||
- 密码 hadoop
|
||
|
||
#### 创建Kerberos的管理账号
|
||
|
||
admin/admin@YLDEV.NET
|
||
|
||
```bash
|
||
kadmin.local
|
||
addprinc admin/admin@YLDEV.NET
|
||
```
|
||
|
||
- 密码/账号 admin/admin@YLDEV.NET
|
||
|
||
#### 将Kerberos服务添加到自启动服务，并启动krb5kdc和kadmin服务
|
||
|
||
```bash
|
||
systemctl enable krb5kdc
|
||
systemctl enable kadmin
|
||
systemctl start krb5kdc
|
||
systemctl start kadmin
|
||
```
|
||
|
||
#### 测试Kerberos的管理员账号
|
||
|
||
```bash
|
||
kinit admin/admin@YLDEV.NET
|
||
```
|
||
|
||
### 为集群安装所有Kerberos客户端，包括Cloudera Manager
|
||
|
||
```bash
|
||
yum -y install krb5-libs krb5-workstation
|
||
|
||
```
|
||
|
||
### 将KDC Server上的krb5.conf文件拷贝到所有Kerberos客户端
|
||
|
||
```bash
|
||
scp /etc/krb5.conf root@node-2:/etc/
|
||
scp /etc/krb5.conf root@node-3:/etc/
|
||
```
|
||
|
||
## CDH集群启用Kerberos
|
||
|
||
### 在KDC中给Cloudera Manager添加管理员账号
|
||
|
||
cloudera-scm/admin@YLDEV.NET
|
||
|
||
执行命令
|
||
|
||
```bash
|
||
kadmin.local
|
||
# 添加cloudera-scm 密码admin
|
||
addprinc cloudera-scm/admin
|
||
# 查询已有的用户
|
||
list_principals
|
||
```
|
||
|
||
### 进入Cloudera Manager的“管理”->“安全”界面
|
||
|
||
|
||
# Kerberos 相关命令
|
||
|
||
## 生成kerberos密钥
|
||
|
||
- hdfs 账号生成秘钥
|
||
|
||
```bash
|
||
kadmin.local -q "xst -k /tmp/hdfs.keytab hdfs@YLDEV.NET -norandkey"
|
||
kadmin.local -q "xst -k /tmp/hdfs.keytab hdfs@YLDEV.NET"
|
||
#生成密钥文件之后，密码失效，不想密码失效，加 '-norandkey’参数
|
||
ktadd -k /tmp/hdfs.keytab -norandkey hdfs@YLDEV.NET
|
||
# 或者执行上面的上面命令
|
||
```
|
||
|
||
## 更新Kerberos票据credentials信息
|
||
|
||
```bash
|
||
# 票据过期，无法正常登录，执行命令
|
||
kinit -R
|
||
```
|
||
|
||
## 测试Keytable是否可用
|
||
|
||
```bash
|
||
kinit -k -t /root/admin.keytab admin/admin@YLDEV.NET
|
||
kinit -k -t /tmp/admin.keytab admin/admin@YLDEV.NET
|
||
```
|
||
|
||
## 修改用户的密码
|
||
|
||
```bash
|
||
# 修改用户hdfs的密码为‘admin’
|
||
kadmin.local -q "cpw -pw admin hdfs "
|
||
```
|
||
|
||
## 查询证书状态
|
||
|
||
```bash
|
||
# 获取hdfs@A.COM账号的信息
|
||
getprinc hdfs@YLDEV.NET
|
||
|
||
modprinc -maxlife 720days -maxrenewlife 1000days +allow_renewable krbtgt/A.COM
|
||
```
|
||
|
||
## 修改过期参数
|
||
|
||
### ticket lifetime
|
||
|
||
```bash
|
||
ticket lifetime取决于以下5项设置中的最小值：
|
||
Kerberos server上/var/kerberos/krb5kdc/kdc.conf中max_life
|
||
内置principal krbtgt的maximum ticket life，可在kadmin命令行下用getprinc命令查看
|
||
principal的maximum ticket life，可在kadmin命令行下用getprinc命令查看
|
||
Kerberos client上/etc/krb5.conf的ticket_lifetime
|
||
kinit -l 参数后面指定的时间
|
||
```
|
||
|
||
|
||
|
||
### ticket renew lifetime
|
||
|
||
```bash
|
||
ticket renew lifetime取决于以下5项设置中的最小值：
|
||
Kerberos server上/var/kerberos/krb5kdc/kdc.conf中max_renewable_life
|
||
内置principal krbtgt的maximum renewable life，可在kadmin命令行下用getprinc命令查看
|
||
你的principal的maximum renewable life，可在kadmin命令行下用getprinc命令查看
|
||
Kerberos client上/etc/krb5.conf的renew_lifetime
|
||
kinit -r 参数后面指定的时间
|
||
```
|
||
|
||
|
||
|
||
# hive的Ldap配置
|
||
|
||
## 无LDAP配置
|
||
|
||
```bash
|
||
<property>
|
||
<name>hive.server2.authentication</name>
|
||
<value>NONE</value>
|
||
<description>客户端身份认证方式</description>
|
||
</property>
|
||
```
|
||
|
||
## LDAP配置
|
||
|
||
``` xml
|
||
<property>
|
||
<name>hive.server2.authentication</name>
|
||
<value>LDAP</value>
|
||
<description>客户端身份认证方式</description>
|
||
</property>
|
||
<property>
|
||
<name>hive.server2.authentication.ldap.url</name>
|
||
<value>ldaps://ldap.yldev.net:636</value>
|
||
<description>LDAP Url</description>
|
||
</property>
|
||
<property>
|
||
<name>hive.server2.authentication.ldap.baseDN</name>
|
||
<value>ou=project,dc=yldev,dc=net</value>
|
||
<description>LDAP搜索的baseDN</description>
|
||
</property>
|
||
|
||
```
|
||
|
||
- hive的权限认证的方式 hive.server2.authentication
|
||
- **NONE**:不做认证;
|
||
- **LDAP**: 使用基于 LDAP/AD 的用户身份校验；
|
||
- **KERBEROS**: 使用 Kerberos/GSSAPI 做身份校验；
|
||
- LDAP认证的服务器URL：hive.server2.authentication.ldap.url
|
||
- 协议
|
||
- 未做ssl的ldap协议
|
||
- **ldap**://ldap.yldev.net
|
||
- ssl的ldaps协议
|
||
- **ldaps**://ldap.yldev.net:636
|
||
- baseDN 登录用户组
|
||
- project用户组可以登录hive
|
||
- **ou=project**,dc=yldev,dc=net
|
||
|
||
|
||
|
||
|
||
|
||
|
||
## 获取特权keytab
|
||
|
||
创建 hdfs@YLDEV.NET
|
||
获取密钥 然后认证 即可使用特权hdfs账号 操作 hdfs 文件系统
|