• zcbus-8.1-16-20240608.tar.gz版本支持逻辑slot模式,逻辑插件只支持mppdb_decoding
  • ddl目前不支持,因为gaussdb不支持EVENT TRIGGERS
  • 可通过调大pg_slot_fetch_record_once参数,加快分析速度,注意源端库系统资源消耗,设置超过262144(256k)解码性能无明显提升

1 源库配置

1.1 修改配置文件

  • 修改postgresql.conf文件
#逻辑级别日志
wal_level = logical
#如果需要远程读日志,需要设置,至少为1,就接受1个读日志线程
max_wal_senders = 10   #建议大于等于10 
#设置日志保留个数,设置太少了,日志量太大了日志自动删除后,容易读不到日志
wal_keep_segments = 10   #建议大于等于10

# wal_keep_size:要为日志复制进程保留多少日志量。旧版本使用的是wal_keep_segments参数(要为日志复制进程保留多少个日志文件),在pg 13中,该参数已被移除。wal_keep_size = wal_keep_segments * wal_segment_size
# 修改完成后需要重启数据库:pg_ctl -D /var/lib/postgresql/data -l logfile restart
# wal_level=logical 需要重启数据库
  • 验证是否成功
postgres=# select name,setting,category from pg_settings where name in ('wal_level','wal_keep_segments','max_wal_senders','wal_keep_size');
       name        | setting |           category
-------------------+---------+-------------------------------
 max_wal_senders   | 10      | Replication / Sending Servers
 wal_keep_segments | 10      | Replication / Sending Servers
 wal_level         | logical | Write-Ahead Log / Settings
(3 rows)
  • 如果远程读日志,需要设置读日志的权限,修改pg_hba.conf
host    replication     zcbus    0.0.0.0/0               sha256
  • 修改pg_hba.conf执行gs_ctl命令生效
pg_ctl reload -D 指定gs_hba.conf所在目录
示例:gs_ctl reload -D /opt/postgresql/data

1.2 逻辑slot模式

1.2.1 创建逻辑slot

SELECT * FROM pg_create_logical_replication_slot('zcbus_slot', 'mppdb_decoding');

删除逻辑slot:
SELECT pg_drop_replication_slot('slot_name');

1.2.2 开启表REPLICA IDENTITY参数

\c database_name;
ALTER TABLE schemaname.tabname REPLICA IDENTITY FULL;

---批量生成打开表补充日志sql语句参考
SELECT 'ALTER TABLE '||nspname||'.'||relname||' REPLICA IDENTITY FULL;' FROM pg_class c,pg_namespace n where relnamespace = n.oid and c.relkind='r' and nspname='public';

---注1:表所属用户或者超级用户才有开启REPLICA IDENTITY权限,否则报错:must be owner of table
---注2:关于分区表开启补充日志以及添加新的同步表,请查看本页2.1,2.2章节说明

1.3 创建用户并授权

create user zcbus with password  '********';
grant connect on database 同步库名 to zcbus;
ALTER USER zcbus replication; #逻辑slot模式,需要replication role
grant select on all tables in schema 同步schema名 to zcbus;
grant usage on schema 同步schema名 to zcbus;

2 补充说明

2.1 添加新的同步表

添加新表,根据schemaname.tablename,确认postgres账号开启表的补充日志模式状态开启成功,可参考1.2.2章节

select relreplident from pg_class c , pg_tables tab where tab.schemaname = 'schemaname' and c.relname='tablename';

查询结果说明

d = 默认(主键)
n = 无
f = 所有列
i = 索引的indisreplident被设置或者默认
查询结果为f,则此表满足同步条件,开启表的补充日志模式成功

注:每次添加完新的同步表,需要再次对同步用户执行如下权限

grant select on all tables in schema 同步schemato zcbus;
grant usage on schema 同步schemato zcbus;

2.2 源表为分区表开启补充日志

SELECT
   parent.relname AS partitioned_table_name,
   parent.relkind AS partitioned_table_type,
   parent.relpartbound AS partition_bound,
   child.relname,
   child.relkind,
   child.relreplident,
   'ALTER TABLE ' || nspname || '.' || child.relname || ' REPLICA IDENTITY FULL' rif
FROM
 pg_class parent
LEFT JOIN pg_inherits ON pg_inherits.inhparent = parent.oid
LEFT JOIN pg_class child ON pg_inherits.inhrelid = child.oid
LEFT JOIN pg_namespace pns ON child.relnamespace = pns.oid
WHERE
 parent.relkind = 'p'
AND parent.relname = 'tabname';

---针对分区表,需要单独为表中的分区开启补充日志后才能满足同步条件
---若分区表没有开启补充日志,可通过上述语句生成的批量SQL开启分区表补充日志
文档更新时间: 2024-06-13 23:24   作者:程少波