Doris offline synchronization

This article provides implementation ideas, configuration examples and verification methods for Doris offline synchronization scenarios, which can be used as a reference for actual project configuration.

Implement offline synchronization of data from the result table to the target table in Doris

Case

Synchronize the data in t_source to t_target offline in Doris

Note

The relevant port number of doris needs to be open, otherwise the synchronization will fail.

Instance nameport nameDefault portcommunication directionDescription
FEhttp_port8030Client <-> FE, FE <-> FEHTTP server port, used to submit tasks, obtain metadata, etc. CDC tasks primarily rely on this port.
query_port9030Client <-> FEMySQL server port, used to connect to FE through the MySQL protocol to execute SQL commands (such as creating tables and queries).
rpc_port9020BE -> FE, FE <-> FEThrift server port, used for internal communication between FE and BE.
edit_log_port9010FE <-> FEFE cluster internal communication port (bdbje).
BEwebserver_port8040Client <-> BE, BE <-> BEHTTP server port, used for data import and Tablet reporting, etc. The CDC task relies on this port to write data.
be_port9060FE -> BEThrift server port used to receive requests from FE.
brpc_port8060FE <-> BE, BE <-> BEBRPC communication port, used for internal communication between BEs.
heartbeat_service_port9050FE -> BEHeartbeat service port.

solution

Create a Flink SQL job type and add the following dependencies

20250124154311

CREATE TABLE from_table
(
    username STRING,
    age      INT
) WITH (
      'connector' = 'doris',
      'fenodes' = '127.0.0.1:8030',
      'table.identifier' = 'ispong_db.t_source',
      'username' = 'root',
      'password' = 'root123'
      );

CREATE TABLE target_table
(
    username STRING,
    age      INT
) WITH (
      'connector' = 'doris',
      'fenodes' = '127.0.0.1:8030',
      'table.identifier' = 'ispong_db.t_target',
      'username' = 'root',
      'password' = 'root123'
      );

INSERT INTO target_table
select *
from from_table;
Configuration itemsillustrate
connectorConnection method
fenodesDoris's fe connection, default port 8030
table.identifierLibrary name.Table name
usernameData source account
passwordAccount password