postgresql 高可用 etcd + patroni 之二 patroni

os: centos 7.4
postgresql: 9.6.9
etcd: 3.2.18
patroni: 1.4.4

patroni + etcd 是在一个postgrsql 开源大会上 亚信的一个哥们讲解的高可用方案。
依然是基于 postgreql stream replication。

ip规划
192.168.56.101 node1 master
192.168.56.102 node2 slave
192.168.56.103 node3 slave

安装postgresql并配置好stream

node1、node2、node3 节点上注意设置如下几个参数

synchronous_commit = on
full_page_writes = on
wal_log_hints = on
synchronous_standby_names = '*'
max_replication_slots = 10

主要是为了使用 pg_rewind,尽量不用 synchronous 方式复制数据，性能影响太大。

node1上创建复制槽，至关重要，patroni 用到了这个玩意

postgres=# create user replicator replication login encrypted password '1qaz2wsx';postgres=# select * from pg_create_physical_replication_slot('pgsql96_node1');
postgres=# select * from pg_create_physical_replication_slot('pgsql96_node2');
postgres=# select * from pg_create_physical_replication_slot('pgsql96_node3');

node2、node3 配置stream replication

$ /usr/pgsql-9.6/bin/pg_ctl stop -m fast -D /var/lib/pgsql/9.6/main$ cd /var/lib/pgsql/9.6/main
$ rm -rf ./*
$ /usr/pgsql-9.6/bin/pg_basebackup -h 192.168.56.101 -D /var/lib/pgsql/9.6/main -U replicator -v -P -R$ vi recovery.conf
recovery_target_timeline = 'latest'
standby_mode = 'on'
primary_conninfo = 'host=192.168.56.101 port=5432 user=replicator password=1qaz2wsx'
primary_slot_name = 'pgsql96_node1'
trigger_file = '/tmp/postgresql.trigger.5432'$ /usr/pgsql-9.6/bin/pg_ctl start -D /var/lib/pgsql/9.6/main -o "-c config_file=/etc/postgresql/9.6/main/postgresql.conf"

注意 recovery.conf 的 primary_slot_name 在不同节点值会不同。

添加复制功能条目 pg_hba.conf

$ vi pg_hba.conf# Database administrative login by Unix domain socket
local   all             postgres                                peer# TYPE  DATABASE        USER            ADDRESS                 METHOD# "local" is for Unix domain socket connections only
local   all             all                                     peer# IPv4 local connections:
host    all             postgres        127.0.0.1/32            trust
host    all             all             127.0.0.1/32            md5
host    all             all             192.168.56.0/24         md5# IPv6 local connections:
host    all             all             ::1/128                 md5# Allow replication connections from localhost, by a user with the
# replication privilege.
local   replication     replicator                              peer
host    replication     replicator      127.0.0.1/32            md5
host    replication     replicator      ::1/128                 md5host	replication	    replicator	    192.168.56.101/32	         md5
host	replication	    replicator	    192.168.56.102/32	         md5
host	replication	    replicator	    192.168.56.103/32	         md5$ psql -c "select pg_reload_conf();"

查看复制状态

postgres=#  select client_addr,pg_xlog_location_diff(sent_location, write_location) as write_delay,pg_xlog_location_diff(sent_location, flush_location) as flush_delay,pg_xlog_location_diff(sent_location, replay_location) as replay_delay from pg_stat_replication;client_addr   | write_delay | flush_delay | replay_delay 
----------------+-------------+-------------+--------------192.168.56.102 |           0 |           0 |            0192.168.56.103 |           0 |           0 |            0
(2 row)

安装etcd

参考上一篇blog安装好etcd

下载、安装 patroni

用户也可以参考 https://www.linode.com/docs/databases/postgresql/create-a-highly-available-postgresql-cluster-using-patroni-and-haproxy/

个人觉得上篇文章中 etcd 做成单点不太合适，当然作为参考完全没有问题。

# cd /tmp
# curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
# python get-pip.py
# pip install patroni[dependencies]

patroni的一些依赖

urllib3>=1.19.1,!=1.21
boto
psycopg2>=2.5.4
PyYAML
requests
six>=1.7
kazoo>=1.3.1
python-etcd>=0.4.3,<0.5
python-consul>=0.7.0
click>=4.1
prettytable>=0.7
tzlocal
python-dateutil
psutil
cdiff
kubernetes>=2.0.0,<=6.0.0,!=4.0.*,!=5.0.*

patroni 的配置

# which patroni
/usr/bin/patroni# patroni --help
/usr/lib64/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.""")
Usage: /usr/bin/patroni config.ymlPatroni may also read the configuration from the PATRONI_CONFIGURATION environment variable

错误提示 please use “pip install psycopg2-binary” instead

# pip install psycopg2-binary

patroni 配置文件

# mkdir -p /usr/patroni/conf
# cd /usr/patroni/conf/# vi patroni_postgresql.ymlscope: pgsql96
namespace: /pgsql/
name: pgsql96_node1restapi:listen: 192.168.56.101:8008connect_address: 192.168.56.101:8008etcd:host: 192.168.56.101:2379bootstrap:# this section will be written into Etcd:/<namespace>/<scope>/config after initializing new cluster# and all other cluster members will use it as a `global configuration`dcs:ttl: 30loop_wait: 10retry_timeout: 10maximum_lag_on_failover: 1048576master_start_timeout: 300
#    synchronous_mode: falsepostgresql:use_pg_rewind: trueuse_slots: trueparameters:listen_addresses: "*"port: 5432wal_level: logicalhot_standby: "on"wal_keep_segments: 1000max_wal_senders: 10max_replication_slots: 10wal_log_hints: "on"
#        archive_mode: "on"
#        archive_timeout: 1800s
#        archive_command: gzip < %p > /data/backup/pgwalarchive/%f.gz
#      recovery_conf:
#        restore_command: gunzip < /data/backup/pgwalarchive/%f.gz > %ppostgresql:listen: 0.0.0.0:5432connect_address: 192.168.56.101:5432data_dir: /var/lib/pgsql/9.6/databin_dir: /usr/pgsql-9.6/bin
#  config_dir:authentication:replication:username: replicatorpassword: 1qaz2wsxsuperuser:username: postgrespassword: 1qaz2wsx#watchdog:
#  mode: automatic # Allowed values: off, automatic, required
#  device: /dev/watchdog
#  safety_margin: 5tags:nofailover: falsenoloadbalance: falseclonefrom: falsenosync: false

上面的配置和后面的输出信息有细微差异，是因为当时实验完成后又对这个配置文件做了好几次修正，方便大家直接copy使用。

手动启动 patroni

参数将按以下顺序应用(运行时被赋予最高优先级)：

1、从文件加载参数postgresql.base.conf(或从自定义conf文件(如果已设置)
2、从文件加载参数postgresql.conf
3、从文件加载参数postgresql.auto.conf
4、运行时参数使用-o-name=value

node1、node2、node3 三个节点依次启动

$ patroni /usr/patroni/conf/patroni_postgresql.yml

node1 的日志如下

2018-07-11 18:17:22,402 INFO: Lock owner: pg96_101; I am pg96_101
2018-07-11 18:17:22,430 INFO: no action.  i am the leader with the lock
2018-07-11 18:17:32,403 INFO: Lock owner: pg96_101; I am pg96_101
2018-07-11 18:17:32,432 INFO: no action.  i am the leader with the lock

node2 的日志如下

2018-07-11 18:17:22,421 INFO: Lock owner: pg96_101; I am pg96_102
2018-07-11 18:17:22,421 INFO: does not have lock
2018-07-11 18:17:22,435 INFO: no action.  i am a secondary and i am following a leader
2018-07-11 18:17:32,426 INFO: Lock owner: pg96_101; I am pg96_102
2018-07-11 18:17:32,426 INFO: does not have lock
2018-07-11 18:17:32,436 INFO: no action.  i am a secondary and i am following a leader

node3 的日志如下

2018-07-11 18:17:22,409 INFO: Lock owner: pg96_101; I am pg96_103
2018-07-11 18:17:22,410 INFO: does not have lock
2018-07-11 18:17:22,423 INFO: no action.  i am a secondary and i am following a leader
2018-07-11 18:17:32,415 INFO: Lock owner: pg96_101; I am pg96_103
2018-07-11 18:17:32,415 INFO: does not have lock
2018-07-11 18:17:32,425 INFO: no action.  i am a secondary and i am following a leader

查看集群状态

查看 patroni 集群状态

$ patronictl -c /usr/patroni/conf/patroni_postgresql.yml list pg96
+---------+----------+----------------+--------+---------+-----------+
| Cluster |  Member  |      Host      |  Role  |  State  | Lag in MB |
+---------+----------+----------------+--------+---------+-----------+
|   pg96  | pg96_101 | 192.168.56.101 | Leader | running |       0.0 |
|   pg96  | pg96_102 | 192.168.56.102 |        | running |       0.0 |
|   pg96  | pg96_103 | 192.168.56.103 |        | running |       0.0 |
+---------+----------+----------------+--------+---------+-----------+$ patronictl -c /usr/patroni/conf/patroni_postgresql.yml show-config pg96
loop_wait: 10
maximum_lag_on_failover: 1048576
postgresql:parameters:listen_addresses: '*'port: 5432use_pg_rewind: true
retry_timeout: 10
ttl: 30

查看 etcd 的 信息

$ etcdctl ls /pg96/pg96/
/pg96/pg96/members
/pg96/pg96/initialize
/pg96/pg96/leader
/pg96/pg96/config
/pg96/pg96/optime

$ etcdctl get /pg96/pg96/members/pg96_101
{"conn_url":"postgres://192.168.56.101:5432/postgres","api_url":"http://127.0.0.1:8008/patroni","timeline":1,"state":"running","role":"master","xlog_location":50378640}
$ etcdctl get /pg96/pg96/members/pg96_102
{"conn_url":"postgres://192.168.56.102:5432/postgres","api_url":"http://127.0.0.1:8008/patroni","timeline":1,"state":"running","role":"replica","xlog_location":50378640}
$ etcdctl get /pg96/pg96/members/pg96_103
{"conn_url":"postgres://192.168.56.103:5432/postgres","api_url":"http://127.0.0.1:8008/patroni","timeline":1,"state":"running","role":"replica","xlog_location":50378640}

$ etcdctl get /pg96/pg96/initialize
6576484813966394513

$ etcdctl get /pg96/pg96/leader
pg96_101

$ etcdctl get /pg96/pg96/config
{"ttl":30,"maximum_lag_on_failover":1048576,"retry_timeout":10,"postgresql":{"use_pg_rewind":true,"parameters":{"listen_addresses":"*","port":5432}},"loop_wait":10}
$ etcdctl get /pg96/pg96/optime/leader
50378640

connction

using jdbc:

jdbc:postgresql://node1,node2,node3/postgres?targetServerType=master

libpq starting from postgresql 10:

postgresql://node1:port,node2:port,node3:port/?target_session_attrs=read-write

配置随OS启动

# vi /etc/rc.local
su - postgres -c "/usr/bin/patroni /usr/patroni/conf/patroni_postgresql.yml >> /var/log/postgresql/patroni.log  2>&1 &"

或者配置成 patroni.service

# vi /etc/systemd/system/patroni.service[Unit]
Description=Runners to orchestrate a high-availability PostgreSQL
After=syslog.target network.target etcd.target[Service]
Type=simpleUser=postgres
Group=postgresExecStart=/usr/bin/patroni /usr/patroni/conf/patroni_postgresql.ymlKillMode=processTimeoutSec=30Restart=no[Install]
WantedBy=multi-user.targ

# systemctl status patroni
# systemctl start patroni
# systemctl enable patroni# systemctl status postgresql
# systemctl disable postgresql# systemctl status etcd
# systemctl enable etcd

禁止 postgresql 的自启动，通过 patroni 来管理 postgresql。

总结：
个人感觉 etcd + patroni 还是相当不错的，会继续对patroni 研究下。

参考：
https://github.com/zalando/patroni
https://patroni.readthedocs.io/en/latest/
https://pypi.org/project/patroni/

https://github.com/zalando/patroni/blob/master/docs/replication_modes.rst
https://postgresconf.org/system/events/document/000/000/228/Patroni_tutorial_4x3-2.pdf

转载于:https://www.cnblogs.com/ctypyb2002/p/9792939.html

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.mzph.cn/news/249972.shtml

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈email:809451989@qq.com，一经查实，立即删除！

相关文章

vue对象侦测

vue对象侦测

http://blog.csdn.net/yihanzhi/article/details/74200618 数组&#xff1a;this.$set(this.arr,index,value) 转载于:https://www.cnblogs.com/smzd/p/8390626.html
阅读更多...
Laravel 5.4 migrate时报错: Specified key was too long error

Laravel 5.4 migrate时报错: Specified key was too long error

Laravel 5.4默认使用utf8mb4字符编码&#xff0c;而不是之前的utf8编码。因此运行php artisan migrate 会出现如下错误&#xff1a; [Illuminate\Database\QueryException] SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key leng…
阅读更多...
springboot工具类

springboot工具类

ClassPathResource 在类路径下读取资源 public final String getPath() public boolean exists() public InputStream getInputStream() WebUtils 获取web资源工具类 public static String getRealPath(ServletContext servletContext, String path) public static Object g…
阅读更多...
MySQL中事物的详解

MySQL中事物的详解

1. 事物的定义及特性 事务是一组操作数据库的SQL语句组成的工作单元&#xff0c;该工作单元中所有操作要么同时成功&#xff0c;要么同时失败。事物有如下四个特性&#xff0c;ACID简称“酸性”。 1&#xff09;原子性&#xff1a;工作单元中所有的操作要么都成功&#xff0c;要…
阅读更多...
记了老是忘记那就写下来吧宏任务微任务

记了老是忘记那就写下来吧宏任务微任务

宏任务&#xff1a;script 定时器 微任务&#xff1a;promiss process.nexttick new Promise(function(resolve){console.log(3);//此为同步程序resolve();//同步 是否异步 由内部函数决定console.log(4); }).then(function(){ //。then 异步console.log(5); });async function…
阅读更多...
SPRING自定义注入CONTROLLER变量

SPRING自定义注入CONTROLLER变量

问题描述 在SpringMVC中默认可以注入Model&#xff0c;ModelAndView&#xff0c;RequestParam&#xff0c;PathVariable 等&#xff0c;那么这个是怎么实现的&#xff0c;以及怎么注入一个自定义的参数呢 HandlerMethodArgumentResolver 在SpringMVC中有一个接口HandlerMethod…
阅读更多...
进程，线程

进程，线程

import os, timeif __name__ __main__:print(the calling process id:%d % os.getpid())# 创建进程pid os.fork()if pid 0:# 子进程print(the child pid is %d % os.getpid())time.sleep(3)elif pid > 0:# 父进程os.wait() # 等待子进程终止print([%d]bye-bye % os.getpi…
阅读更多...
livebos--iframe使用

livebos--iframe使用

新建一个方法。建一个参数&#xff0c;iframe控件&#xff0c;虚拟列。然后使用以下信息 <% livebos languagejavascript %>var url LB_ObjURI("Lb_lbOrganization",0,[],["NoTitle"]);var v {"edit" : "url ", "view"…
阅读更多...
单行溢出 和多行溢出

单行溢出 和多行溢出

/*单行溢出*/.one_txt_cut{overflow: hidden;white-space: nowrap;text-overflow: ellipsis;}.txt_cut{overflow : hidden;text-overflow: ellipsis;display: -webkit-box;-webkit-line-clamp: 2;-webkit-box-orient: vertical;}转载于:https://www.cnblogs.com/smzd/p/8491583…
阅读更多...
Spring方法注入 @Lookup注解使用

Spring方法注入 @Lookup注解使用

情景分析 在Spring的诸多应用场景中bean都是单例形式&#xff0c;当一个单利bean需要和一个非单利bean组合使用或者一个非单利bean和另一个非单利bean组合使用时&#xff0c;我们通常都是将依赖以属性的方式放到bean中来引用&#xff0c;然后以Autowired来标记需要注入的属性。…
阅读更多...
Jupyter配置步骤

Jupyter配置步骤

Jupyter是基于浏览器的可交互式开发工具&#xff0c;在数据科学界非常受欢迎&#xff0c;它功能齐全&#xff0c;使用方便&#xff0c;是一款数据分析和建模挖掘的利器。 本文简介Jupyter的配置和使用过程 一、修改添加国内镜像 通常我会先安装Anaconda&#xff0c;再安装Jupyt…
阅读更多...
edittext 属性

edittext 属性

1.去掉edittext的底线&#xff0c;设置&#xff0c;不管是edittext&#xff0c;还是appcompatEdittext都是这个属性 转载于:https://www.cnblogs.com/hechangshou/p/9301004.html
阅读更多...
定义高亮颜色

定义高亮颜色

/*怎么定义高亮的颜色*/-webkit-tap-highlight-color: transparent;/*透明 其实就是不显示颜色*/-webkit-tap-highlight-color: red; 转载于:https://www.cnblogs.com/smzd/p/8491587.html
阅读更多...
springboot 配置webservice接口

springboot 配置webservice接口

导入依赖的jar <!-- webservice cxf --><dependency><groupId>org.apache.cxf</groupId><artifactId>cxf-rt-frontend-jaxws</artifactId><version>3.1.6</version></dependency><dependency><groupId>org…
阅读更多...
【Django】认证系统

【Django】认证系统

目录 #. auth模块1. 认证 authenticate()2. 登陆 login(HttpRequest, user)3. 注销 logout(request)4. 认证判断 is_authenticated()5. 登陆校验 login_requierd()6. 创建普通用户 create_user()7. 创建超级用户 create_superuser()8. 密码校验 check_password(password)9. 修改…
阅读更多...
学习的目的是什么？

学习的目的是什么？

学习的目的是为了掌握生存的常识和技能&#xff0c;以便独立地面对世界&#xff1b; 学习的目的是为了遵从生活的规范和律则&#xff0c;以便和谐地与人相处&#xff1b; 学习的目的是为了探索生命的价值和意义&#xff0c;以便有尊严地立于天地之间。 你觉得为什么要学习呢&am…
阅读更多...
span里面插入文字

span里面插入文字

.text-box span::before{ content:attr(data-text);} 转载于:https://www.cnblogs.com/smzd/p/8491664.html
阅读更多...
Spring Boot 动态注入的两种方式

Spring Boot 动态注入的两种方式

通过Profilespring.profiles.active spring.profiles.active&#xff1a;官方解释是激活不同环境下的配置文件&#xff0c;但是实际测试发现没有对应的配置文件也是可以正常执行的。那就可以把这个key当作一个参数来使用 Profile&#xff1a;spring.profiles.active中激活某配…
阅读更多...
kernel devel 安装与卸载

kernel devel 安装与卸载

1、查看系统内核 uname -r 2、查看已安装kernel-devel uname -a ; rpm -qa kernel\* | sort 3、下载对应的rpm wget xxx/kernel-devel-2.6.32-754.el6.x86_64.rpm 或者 $ sudo yum install "kernel-devel-uname-r $(uname -r)" 4、卸载已安装的内核 yum remove ker…
阅读更多...
弹性布局

弹性布局

/* 开启弹性布局的换行 */ flex-wrap: wrap;/* 变为多行了 无法使用 align-items 进行位置设置 align-content 在多行的时候 设置属性 跟 justify-content 一模一样如果只有 一行时 无法生效 *//* 调整元素 在主轴上的 排布方式flex-end 到主轴的末尾flex-start 默认值center…
阅读更多...
最新文章

Copyright @ 2022~2023