实时同步

本文提供MySQL实时同步场景的实施思路、配置示例和验证方式,可作为实际项目配置参考。

实现MySQL中结果表向目标表中实时同步数据

案例

MySQL中将cdc_source中的数据实时同步到cdc_target中

前提

  • 需要开启binlog服务
  • 用户授权
GRANT SELECT, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'ispong'@'%';  
GRANT RELOAD ON *.* TO 'ispong'@'%';  
FLUSH PRIVILEGES;

解决方案

创建Flink SQL作业类型,添加flink-sql-connector-MySQL-cdc依赖

20241230183511

CREATE TABLE from_table
(
    username STRING,
    age      INT
) WITH ( 'connector' = 'mysql-cdc',
      'hostname' = 'localhost',
      'port' = '30306',
      'username' = 'ispong',
      'password' = 'ispong123',
      'database-name' = 'isxcode_db',
      'table-name' = 'cdc_source',
      'scan.incremental.snapshot.enabled' = 'false',
      'server-time-zone' = 'UTC',
      'scan.startup.mode' = 'latest-offset'
      );

CREATE TABLE target_table
(
    username STRING,
    age      INT,
    PRIMARY KEY (username) NOT ENFORCED
) WITH ( 'connector' = 'jdbc',
      'url' = 'jdbc:mysql://localhost:30306/isxcode_db?useSSL=false&allowPublicKeyRetrieval=true',
      'driver' = 'com.mysql.cj.jdbc.Driver',
      'table-name' = 'cdc_target',
      'username' = 'ispong',
      'password' = 'ispong123');

INSERT INTO target_table
select *
from from_table;
配置项说明
connector连接方式
url数据库连接url
driver数据源驱动
table-name表名
username数据源账号
password账号密码
server-time-zone服务器的时区,两个表时区不一致会报错
scan.startup.mode从什么地方开始同步