- zcbus-8.1-16-20240608.tar.gz版本支持逻辑slot模式,逻辑插件只支持mppdb_decoding
- ddl目前不支持,因为gaussdb不支持EVENT TRIGGERS
- 可通过调大pg_slot_fetch_record_once参数,加快分析速度,注意源端库系统资源消耗,设置超过262144(256k)解码性能无明显提升
1 源库配置
1.1 修改配置文件
- 修改postgresql.conf文件
#逻辑级别日志
wal_level = logical
#如果需要远程读日志,需要设置,至少为1,就接受1个读日志线程
max_wal_senders = 10 #建议大于等于10
#设置日志保留个数,设置太少了,日志量太大了日志自动删除后,容易读不到日志
wal_keep_segments = 10 #建议大于等于10
# wal_keep_size:要为日志复制进程保留多少日志量。旧版本使用的是wal_keep_segments参数(要为日志复制进程保留多少个日志文件),在pg 13中,该参数已被移除。wal_keep_size = wal_keep_segments * wal_segment_size
# 修改完成后需要重启数据库:pg_ctl -D /var/lib/postgresql/data -l logfile restart
# wal_level=logical 需要重启数据库
- 验证是否成功
postgres=# select name,setting,category from pg_settings where name in ('wal_level','wal_keep_segments','max_wal_senders','wal_keep_size');
name | setting | category
-------------------+---------+-------------------------------
max_wal_senders | 10 | Replication / Sending Servers
wal_keep_segments | 10 | Replication / Sending Servers
wal_level | logical | Write-Ahead Log / Settings
(3 rows)
- 如果远程读日志,需要设置读日志的权限,修改pg_hba.conf
host replication zcbus 0.0.0.0/0 sha256
- 修改pg_hba.conf执行gs_ctl命令生效
pg_ctl reload -D 指定gs_hba.conf所在目录
示例:gs_ctl reload -D /opt/postgresql/data
1.2 逻辑slot模式
1.2.1 创建逻辑slot
SELECT * FROM pg_create_logical_replication_slot('zcbus_slot', 'mppdb_decoding');
删除逻辑slot:
SELECT pg_drop_replication_slot('slot_name');
1.2.2 开启表REPLICA IDENTITY参数
\c database_name;
ALTER TABLE schemaname.tabname REPLICA IDENTITY FULL;
---批量生成打开表补充日志sql语句参考
SELECT 'ALTER TABLE '||nspname||'.'||relname||' REPLICA IDENTITY FULL;' FROM pg_class c,pg_namespace n where relnamespace = n.oid and c.relkind='r' and nspname='public';
---注1:表所属用户或者超级用户才有开启REPLICA IDENTITY权限,否则报错:must be owner of table
---注2:关于分区表开启补充日志以及添加新的同步表,请查看本页2.1,2.2章节说明
1.3 创建用户并授权
create user zcbus with password '********';
grant connect on database 同步库名 to zcbus;
ALTER USER zcbus replication; #逻辑slot模式,需要replication role
grant select on all tables in schema 同步schema名 to zcbus;
grant usage on schema 同步schema名 to zcbus;
2 补充说明
2.1 添加新的同步表
添加新表,根据schemaname.tablename,确认postgres账号开启表的补充日志模式状态开启成功,可参考1.2.2章节
select relreplident from pg_class c , pg_tables tab where tab.schemaname = 'schemaname' and c.relname='tablename';
查询结果说明
d = 默认(主键)
n = 无
f = 所有列
i = 索引的indisreplident被设置或者默认
查询结果为f,则此表满足同步条件,开启表的补充日志模式成功
注:每次添加完新的同步表,需要再次对同步用户执行如下权限
grant select on all tables in schema 同步schema名 to zcbus;
grant usage on schema 同步schema名 to zcbus;
2.2 源表为分区表开启补充日志
SELECT
parent.relname AS partitioned_table_name,
parent.relkind AS partitioned_table_type,
parent.relpartbound AS partition_bound,
child.relname,
child.relkind,
child.relreplident,
'ALTER TABLE ' || nspname || '.' || child.relname || ' REPLICA IDENTITY FULL' rif
FROM
pg_class parent
LEFT JOIN pg_inherits ON pg_inherits.inhparent = parent.oid
LEFT JOIN pg_class child ON pg_inherits.inhrelid = child.oid
LEFT JOIN pg_namespace pns ON child.relnamespace = pns.oid
WHERE
parent.relkind = 'p'
AND parent.relname = 'tabname';
---针对分区表,需要单独为表中的分区开启补充日志后才能满足同步条件
---若分区表没有开启补充日志,可通过上述语句生成的批量SQL开启分区表补充日志
文档更新时间: 2024-06-13 23:24 作者:程少波