PostgreSQL的Slony-I數據同步實踐
官方文檔:
slony-I
系統要求:http://slony.info/documentation/requirements.html
配置與安裝:http://slony.info/documentation/administration.html#INSTALLATION
使用配置:http://slony.info/documentation/tutorial.html#FIRSTDB
環境:
主庫:centos linux 32bit虛擬機,ip為192.168.100.240
PostgreSQL9.2.13
Slony-I 2.2
備庫: centos linux 32bit虛擬機, ip為192.168.100.241
PostgreSQL9.2.13
Slony-I 2.2
1.源碼編譯安裝PostgreSQL
在主庫和備庫都,進行源碼編譯、安裝、配置PostgreSQL數據庫如下:
源碼編譯安裝PostgbreSQL9.2。
安裝目錄:/opt/pgsql2
源碼安裝完畢后,要配置pg_hba.conf和postgresql.conf確保主、備庫可以遠程訪問。
注意:PG版本要必須是Slony支持的版本,詳見 http://slony.info/documentation/requirements.html 。
2.準備slony-I復制的主、備庫
2.1聲明環境變量
在主庫和備庫都,執行:
su - postgres export CLUSTERNAME=lyy_cluster1 export MASTERDBNAME=masterdb export SLAVEDBNAME=slavedb export MASTERHOST=192.168.13.128 export SLAVEHOST=192.168.100.236 export REPLICATIONUSER=postgres
注意:
1.REPLICATIONUSER通常是PostgreSQL數據庫的超級用戶。
2.修改MASTERHOST和SLAVEHOST時,盡量不要使用localhost,因為可能會導致錯誤:ERROR remoteListenThread_1: db_getLocalNodeId() returned 2 - wrong database?。
2.2根據環境變量準備數據庫
在主庫和備庫都,根據環境變量來創建相應對象:
--在主服務器中 cd /opt/pgsql2/bin ./createuser -p 5432 -U postgres -SRD -P $PGBENCHUSER --若已創建則無需再創建 input password for new user :(此處輸入新用戶的密碼即可) input password again:(此處輸入新用戶的密碼即可) password:(此處輸入超級用戶postgres的密碼即可) ./createdb -p 5432 -U postgres -O $PGBENCHUSER -h $MASTERHOST $MASTERDBNAME password:(此處輸入超級用戶postgres的密碼即可)
--在備用服務器中 cd /opt/pgsql2/bin ./createuser -p 5432 -U postgres -SRD -P $PGBENCHUSER --若已創建則無需再創建 input password for new user :(此處輸入新用戶的密碼即可) input password again:(此處輸入新用戶的密碼即可) password:(此處輸入超級用戶postgres的密碼即可) ./createdb -p 5432 -U postgres -O $PGBENCHUSER -h $SLAVEHOST $SLAVEDBNAME password:(此處輸入超級用戶postgres的密碼即可)
--在主服務器,創建要同步的數據表(數據表必須有主鍵,才能通過slony-i實現數據同步) [postgres@localhost bin]$ ./psql -U $PGBENCHUSER -h $MASTERHOST -d $MASTERDBNAME psql (9.2.13) Type "help" for help.. masterdb=# CREATE TABLE lyy(id int primary key, name varchar); CREATE TABLE
2.3創建pl/pgsql過程語言
Slony-I 需要數據庫有 pl/pgSQL 過程語言,如果模板數據 template1已經安裝了pl/pgSQL,那么新建的$MASTERDBNAME也就也有了pl/pgSQL,如果已經存在了,則不需要執行以下語句:
在主庫和備庫都執行:
--在bin目錄下 ./createlang -h $MASTERHOST plpgsql $MASTERDBNAME
2.4手動從主庫將表定義導入備庫
當備庫Slony-I 訂閱主庫之后,Slony-I不能自動從主庫拷貝表定義到備庫,所以我們需要把表定義從主庫導入備庫,我們通過 pg_dump來實現表定義的主、備同步:
--把在pgbench數據庫建好的表同步到pgbenchslave數據庫中: --在主服務器中執行dump,然后將執行結果拷貝到備用服務器中執行,即可實現同步 --在bin目錄下 [postgres@localhost bin]$ ./pg_dump -s -U postgres -h 192.168.100.240 masterdb | ./psql -U postgres -h 192.168.100.241 slavedb
3.源碼編譯安裝Slony-I
在主庫和備庫都,按照以下步驟安裝slony-i。
slony下載地址:http://slony.info/downloads/2.2/source/
下載并解壓slony-2.2,然后進行編譯配置安裝,要確保每一步安裝正確后再進行下一步。
cd ../slony-2.2 ./configure --with-pgconfigdir=/opt/pgsql944/bin --with-perltools gmake all gamke install
注意:如果slony-I后面的配置過程使用altperl腳本,則configure時必須指定--with-perltools。
4. 配置slony并啟動同步復制
4.0配置并啟用slony同步復制的理論基礎(僅供理解):
Configuring the Database For Replication.
Creating the configuration tables, stored procedures, triggers and configuration is all done through the slonik tool. It is a specialized scripting aid that mostly calls stored procedures in the master/slave (node) databases.
The example that follows uses slonik directly (or embedded directly into scripts). This is not necessarily the most pleasant way to get started; there exist tools for building slonik scripts under the tools directory, including:
-
Section 6.1.1 - a set of Perl scripts that build slonik scripts based on a single slon_tools.conf file.
-
Section 6.1.2 - a shell script (e.g. - works with Bash) which, based either on self-contained configuration or on shell environment variables, generates a set of slonik scripts ??to configure a whole cluster.
以下兩種方法省略詳細介紹,詳見文檔http://slony.info/documentation/tutorial.html#FIRSTDB。
方法一:Using slonik Command Directly
方法二:Using the altperl Scripts
擴展至:http://file:///C:/Users/Yuanyuan/Desktop/slony1-2.2.4/doc/adminguide/additionalutils.html#ALTPERL
4.1直接使用slonik命令(以上方法一)
在主庫配置slony cluster
在主庫,創建并執行slony_setup.sh文件,實現slony cluster的配置:
[postgres@localhost data]$ vi slony_setup.sh
CLUSTERNAME=lyy_cluster1 MASTERDBNAME=masterdb SLAVEDBNAME=slavedb MASTERHOST=192.168.100.240 SLAVEHOST=192.168.100.241 REPLICATIONUSER=postgres /opt/pgsql92/bin/slonik <<_EOF_ #-- # define the namespace the replication system # uses in our example it is slony_example #-- cluster name = $CLUSTERNAME; #-- # admin conninfo's are used by slonik to connect to # the nodes one for eachnode on each side of the cluster, # the syntax is that of PQconnectdb in # the C-API # -- node 1 admin conninfo = 'dbname=$MASTERDBNAME \ host=$MASTERHOST user=$REPLICATIONUSER'; node 2 admin conninfo = 'dbname=$SLAVEDBNAME \ host=$SLAVEHOST user=$REPLICATIONUSER'; #-- # init the first node. Its id MUST be 1. This creates # the schema _$CLUSTERNAME containing all replication # system specific database objects. #-- init cluster ( id=1, comment = 'Master Node'); #-- # Slony-I organizes tables into sets. The smallest unit # a node can subscribe is a set. The master or origin of # the set is node 1. #-- create set (id=1, origin=1, comment='All masterdb tables'); set add table (set id=1, origin=1, id=1, fully qualified name = 'public.lyy', comment='lyy table'); # set add sequence (set id=1, origin = 1, id = 1, # fully qualified name = 'public.t1_id_seq', # comment = 't1 id sequence'); #-- # Create the second node (the slave) tell the 2 nodes how # to connect to each other and how they should listen for events. #-- store node (id=2, comment = 'Slave Node', event node=1); store path (server = 1, client = 2, conninfo='dbname=$MASTERDBNAME \ host=$MASTERHOST user=$REPLICATIONUSER'); store path (server = 2, client = 1, conninfo='dbname=$SLAVEDBNAME \ host=$SLAVEHOST user=$REPLICATIONUSER'); _EOF_
[postgres@localhost data]$sh slony_setup.sh
注釋:本步執行完畢后,初始化完畢一個名為lyy_cluster1的slony集群。
并相應的產生一個名為_lyy_cluster1的模式,里面含有slony運行所需的配置表、序列、函數、觸發器等(主要是通過slony-i安裝過程中在/opt/pgsql92/share下生成的slony1_base.2.2.4.sql和slony1_funcs.2.2.4.sql)。
還會在同步表lyy下創建相應的觸發器。
在主庫:
masterdb=# \d lyy
Table "public.lyy"
Column | Type | Modifiers
--------+-------------------+-----------
id | integer | not null
name | character varying |
Indexes:
"lyy_pkey" PRIMARY KEY, btree (id)
Triggers:
_lyy_cluster1_logtrigger AFTER INSERT OR DELETE OR UPDATE ON lyy FOR EACH ROW EXECUTE PROCEDURE _lyy_cluster1.logtrigger('_lyy_cluster1', '1', 'k')
_lyy_cluster1_truncatetrigger BEFORE TRUNCATE ON lyy FOR EACH STATEMENT EXECUTE PROCEDURE _lyy_cluster1.log_truncate('1')
_lyy_cluster_logtrigger AFTER INSERT OR DELETE OR UPDATE ON lyy FOR EACH ROW EXECUTE PROCEDURE _lyy_cluster.logtrigger('_lyy_cluster', '1', 'k')
_lyy_cluster_truncatetrigger BEFORE TRUNCATE ON lyy FOR EACH STATEMENT EXECUTE PROCEDURE _lyy_cluster.log_truncate('1')
Disabled triggers:
_lyy_cluster1_denyaccess BEFORE INSERT OR DELETE OR UPDATE ON lyy FOR EACH ROW EXECUTE PROCEDURE _lyy_cluster1.denyaccess('_lyy_cluster1')
_lyy_cluster1_truncatedeny BEFORE TRUNCATE ON lyy FOR EACH STATEMENT EXECUTE PROCEDURE _lyy_cluster1.deny_truncate()
_lyy_cluster_denyaccess BEFORE INSERT OR DELETE OR UPDATE ON lyy FOR EACH ROW EXECUTE PROCEDURE _lyy_cluster.denyaccess('_lyy_cluster')
_lyy_cluster_truncatedeny BEFORE TRUNCATE ON lyy FOR EACH STATEMENT EXECUTE PROCEDURE _lyy_cluster.deny_truncate() 在備庫:
slavedb=# \d lyy
Table "public.lyy"
Column | Type | Modifiers
--------+-------------------+-----------
id | integer | not null
name | character varying |
Indexes:
"lyy_pkey" PRIMARY KEY, btree (id)
Triggers:
_lyy_cluster1_denyaccess BEFORE INSERT OR DELETE OR UPDATE ON lyy FOR EACH ROW EXECUTE PROCEDURE _lyy_cluster1.denyaccess('_lyy_cluster1')
_lyy_cluster1_truncatedeny BEFORE TRUNCATE ON lyy FOR EACH STATEMENT EXECUTE PROCEDURE _lyy_cluster1.deny_truncate()
Disabled triggers:
_lyy_cluster1_logtrigger AFTER INSERT OR DELETE OR UPDATE ON lyy FOR EACH ROW EXECUTE PROCEDURE _lyy_cluster1.logtrigger('_lyy_cluster1', '1', 'k')
_lyy_cluster1_truncatetrigger BEFORE TRUNCATE ON lyy FOR EACH STATEMENT EXECUTE PROCEDURE _lyy_cluster1.log_truncate('1')
在主庫啟動主庫節點及監視器
在主庫,執行以下命令啟動slon daemon:
[postgres@localhost bin]$./slon lyy_cluster1 "dbname=masterdb user=postgres host=192.168.100.240"&
執行當前命令的終端會不斷地返回檢測信息,所以該終端不要關閉也不要再執行其他操作。
在備庫啟動備庫節點及監視器
在備庫。執行以下命令啟動slon deamon:
[postgres@localhost bin]$./slon slony_example "dbname=slavedb user=postgres host=192.168.100.241" &
執行當前命令的終端會不斷地返回檢測信息,所以該終端不要關閉也不要再執行其他操作。
在主庫執行備庫訂閱主庫
在主庫,執行訂閱過程, 通過腳本文件subscribe_set.sh來執行訂閱過程:
[postgres@localhost data]$ vi subscribe_set.sh
CLUSTERNAME=lyy_cluster1 MASTERDBNAME=masterdb SLAVEDBNAME=slavedb MASTERHOST=192.168.100.240 SLAVEHOST=192.168.100.241 REPLICATIONUSER=postgres /opt/pgsql92/bin/slonik <<_EOF_ # ---- # This defines which namespace the replication system uses # ---- cluster name = $CLUSTERNAME; # ---- # Admin conninfo's are used by the slonik program to connect # to the node databases. So these are the PQconnectdb arguments # that connect from the administrators workstation (where # slonik is executed). # ---- node 1 admin conninfo = 'dbname=$MASTERDBNAME host=$MASTERHOST \ user=$REPLICATIONUSER'; node 2 admin conninfo = 'dbname=$SLAVEDBNAME host=$SLAVEHOST \ user=$REPLICATIONUSER'; # ---- # Node 2 subscribes set 1 # ---- subscribe set ( id = 1, provider = 1, receiver = 2, forward = no); _EOF_
[postgres@localhost data]$sh subscribe_set.sh
目前,本次數據庫的slony的同步復制已經配置完畢。
驗證slony-I同步復制生效
在主庫,向表lyy插入數據:
[postgres@localhost bin]$ ./psql -U postgres -d masterdb psql (9.2.13) Type "help" for help. masterdb=# insert into lyy values(1,'lyy'); INSERT 0 1
在備庫,查詢表lyy中的數據情況:
postgres@localhost bin]$ ./psql -U postgres -d slavedb psql (9.2.13) Type "help" for help. slavedb=# select * from lyy; id | name ----+------ (0 rows) slavedb=# select * from lyy; id | name ----+------ 1 | lyy (1 row)
可以在主庫執行增刪操作,然后在備庫執行查詢操作,進行比對。
4.2使用altperl腳本配置并啟動(以上方法二):
配置slony_tool.conf并初始化slony集群
默認情況下,slony的配置文件樣本已默認安裝至/usr/local/etc/slon_tools.conf-sample。
cd /usr/local/etc --復制一個slon_tools.conf-sample,命名為slon_tools.conf [root@localhost etc]#cp slon_tools.conf-sample /usr/local/etc/slon_tools.conf --開始編輯slon_tools.conf [root@localhost etc]#vi slon_tools.conf
配置詳情:
#修改CLUSTER_NAME為前面我們設置的
# The name of the replication cluster. This will be used to
# create a schema named _$CLUSTER_NAME in the database which will
# contain Slony-related data.
$CLUSTER_NAME = 'lyy_cluster1';
#slony的pid文件目錄。如果沒有,則根據提示創建,并授予要求的權限。
# The directory where Slony store PID files. This
# directory will need to be writable by the user that invokes
# Slony.
$PIDFILE_DIR = '/var/run/slony1';
#日志文件存儲目錄,如果沒有,則修改配置或者根據配置創建目錄。
# The directory where Slony should record log messages. This
# directory will need to be writable by the user that invokes
# Slony.
$LOGDIR = '/var/log/slony1';
#修改節點信息為我們的主、備節點的信息,樣本中提供的多出的節點可以注釋或者刪除掉。
# Include add_node lines for each node in the cluster. Be sure to
# use host names that will resolve properly on all nodes
# (i.e. only use 'localhost' if all nodes are on the same host).
# Also, note that the user must be a superuser account.
add_node(node => 1,
host => '162.168.100.240',
dbname => 'masterdb',
port => 5432,
user => 'postgres',
password => 'postgres');
add_node(node => 2,
host => '192.168.100.241',
dbname => 'slavedb',
port => 5432,
user => 'postgres',
password => 'postgres');
#修改要同步的表或者序列
# This array contains a list of tables that already have primary keys.
"pkeyedtables" => [
'public.lyy',
],
#如果沒有序列和無主鍵表需要同步,則將以下示例注釋掉。
# For tables that have unique not null keys, but no primary
# key, enter their names and indexes here.
#lyy comment this
# "keyedtables" => {
# 'table3' => 'index_on_table3',
# 'table4' => 'index_on_table4',
# },
# Sequences that need to be replicated should be entered here.
# lyy comment this
# "sequences" => ['sequence1',
# 'sequence2',
# ],
},
#在主庫,初始化slony cluster [root@localhost etc]# slonik_init_cluster | /opt/pgsql92/bin/slonik <stdin>:10: Set up replication nodes <stdin>:13: Next: configure paths for each node/origin <stdin>:16: Replication nodes prepared <stdin>:17: Please start a slon replication daemon for each node
注釋:同樣的,本步執行完畢后,初始化完畢一個名為lyy_cluster1的slony集群。
并相應的產生一個名為_lyy_cluster1的模式,里面含有slony運行所需的配置表、序列、函數、觸發器等(主要是通過slony-i安裝過程中在/opt/pgsql92/share下生成的slony1_base.2.2.4.sql和slony1_funcs.2.2.4.sql)。
啟動slon并進行數據集合訂閱:
# 在主庫,啟動節點1的slon [root@localhost etc]# slon_start 1 Invoke slon for node 1 - /opt/pgsql92/bin//slon -p /var/run/slony1/lyy_cluster2_node1.pid -s 1000 -d2 lyy_cluster1 'host=192.168.100.240 dbname=master user=postgres port=5432 password=postgres' > /var/log/slony1/node1/master-2015-09-15.log 2>&1 & Slon successfully started for cluster lyy_cluster1, node node1 PID [7839] Start the watchdog process as well...
# 在備庫,啟動節點2的slon [root@localhost etc]# slon_start 2 Invoke slon for node 2 - /opt/pgsql92/bin//slon -p /var/run/slony1/cluster1_node2.pid -s 1000 -d2 cluster1 'host=192.168.100.241 dbname=slavedb user=postgres port=5432 password=postgres' > /var/log/slony1/node2/slavedb-2015-09-15.log 2>&1 & Slon successfully started for cluster lyy_cluster1, node node2 PID [7613] Start the watchdog process as well...
# 在主庫,創建數據集合 (此處 1 是一個 set 集合號) [root@localhost etc]# slonik_create_set 1 | /opt/pgsql92/bin/slonik <stdin>:11: Subscription set 1 (set1_name) created <stdin>:12: Adding tables to the subscription set <stdin>:16: Add primary keyed table public.lyy <stdin>:19: Adding sequences to the subscription set <stdin>:20: All tables added
# 在主庫,訂閱 集合1 到 節點2 (1= set ID, 2= node ID) [root@localhost etc]# slonik_subscribe_set 1 2 | /opt/pgsql92/bin/slonik <stdin>:6: Subscribed nodes to set 1
目前,本次數據庫的slony的同步復制已經配置完畢。
驗證slony-I同步復制生效
在主庫,向表lyy插入數據:
[postgres@localhost bin]$ ./psql -U postgres -d masterdb psql (9.2.13) Type "help" for help. masterdb=# insert into lyy values(1,'lyy'); INSERT 0 1
在備庫,查詢表lyy中的數據情況:
postgres@localhost bin]$ ./psql -U postgres -d slavedb psql (9.2.13) Type "help" for help. slavedb=# select * from lyy; id | name ----+------ (0 rows) slavedb=# select * from lyy; id | name ----+------ 1 | lyy (1 row)
可以在主庫執行增刪操作,然后在備庫執行查詢操作,進行比對。
5.Slony-I的其他操作
slony的switchover操作(也就是把主節點改成從節點,從節點升級為主節點):
slonik_move_set set1 node1 node2 | /opt/pgsql92/bin/slonik
slony的failver操作:
slonik_failover node1 node2 | /opt/pgsql92/bin/slonik
參考資料:
http://www.cnblogs.com/gaojian/p/3196244.html
http://blog.chinaunix.net/uid-15145533-id-2775796.html
也可以只用pgbench來測試slony-i的數據同步。
pgbench使用:http://www.postgresql.org/docs/9.2/static/pgbench.html
來自:http://my.oschina.net/liuyuanyuangogo/blog/504495