Configure a SelectDB data source

Last updated 10/03/2026

SelectDB Cloud is a new-generation, multi-cloud-native real-time data warehouse built on Apache Doris.

Supported versions

SelectDB 4.0+

Limitations

Currently, only the following configuration is supported:

• Writing from AE Built-in Warehouse workspace tables to SelectDB (offline write)

Note:

• Reading from SelectDB data sources (offline read) isn't supported yet

Supported field types

The SelectDB engine determines whether a source field can be written to the target field correctly. If it can't, the value is Null.

Integration plans in the AE DataOps Platform don't force data type conversion; the engine handles it.

SelectDB type	Offline write (Writer)	Remarks
Numeric types
TINYINT	Supported	1-byte signed integer, range [-128, 127]
SMALLINT	Supported	2-byte signed integer, range [-32768, 32767]
INT	Supported	4-byte signed integer, range [-2147483648, 2147483647]
BIGINT	Supported	8-byte signed integer, range [-9223372036854775808, 9223372036854775807]
DECIMAL	Supported	DECIMAL(P [, S]) High-precision fixed-point number. P is the total number of significant digits (precision), and S is the maximum number of digits after the decimal point (scale). In version 1.19.0 and later, the (P, S) of the decimal type has a default value of decimal(10, 0)
DOUBLE	Supported	8-byte floating-point number
BOOLEAN	Supported	BOOL, BOOLEAN Same as TINYINT: 0 means false and 1 means true
LARGEINT	Supported	16-byte signed integer, range [-2^127 + 1 ~ 2^127 - 1]
FLOAT	Supported	4-byte floating-point number.
String types
CHAR	Supported	CHAR(M) Fixed-length string. M is the length of the fixed-length string, and its range is 1~255.
VARCHAR	Supported	VARCHAR(M) Variable-length string. M is the length of the variable-length string, in bytes. The default value is 1.
STRING	Supported	String with a maximum length of 65533 bytes
Time types
DATE	Supported	Date type. The current value range is ['0000-01-01', '9999-12-31']. The default print format is 'YYYY-MM-DD'.
DATETIME	Supported	Datetime type. The value range is ['0000-01-01 00:00:00', '9999-12-31 23:59:59']. The print format is 'YYYY-MM-DD HH:MM:SS'
Semi-structured types
JSON	Supported
ARRAY	Supported
MAP	Supported
STRUCT	Supported
VARIANT	Supported	The VARIANT type is especially suitable for complex nested structures that may change at any time. During writing, it can automatically infer column information from the structure and type of the columns, dynamically merge the written schema, and store JSON keys and their values as columns and dynamic subcolumns.
Aggregation types
HLL	Not supported
BITMAP	Not supported
QUANTILE_STATE	Not supported
AGG_STATE	Not supported

Table types

Table type	Aggregate (aggregate table)	Unique (unique key table)	Duplicate (detail table)
Data model	Pre-aggregation model	Unique primary key model	Detail data model
Storage characteristics	Automatic aggregation on the same dimensions (SUM/MAX/MIN, etc.)	Overwrite on the same primary key (new versions overwrite old versions)	Full storage of raw data (no aggregation, no overwriting)
Core features	Metric columns must define an aggregate function (SUM/MAX/MIN/REPLACE) Queries automatically aggregate by dimension columns, with no need for GROUP BY	Rows with the same UNIQUE KEY are overwritten Supports real-time updates	Stores raw detail data (duplicates allowed) No aggregation; queries need an explicit GROUP BY
SQL example	CREATE TABLE agg_sales ( dt DATE, -- Dimension column (grouped automatically) product_id INT, -- Dimension column total_sales BIGINT SUM, -- Metric column (aggregate function) max_price INT MAX -- Metric column ) ENGINE=OLAP AGGREGATE KEY(dt, product_id) -- Specify the aggregate key DISTRIBUTED BY HASH(product_id);	CREATE TABLE user_profile ( user_id BIGINT, -- Primary key column name VARCHAR REPLACE, -- Updatable column age INT REPLACE, last_login DATETIME REPLACE ) ENGINE=OLAP UNIQUE KEY(user_id) -- Specify the unique key DISTRIBUTED BY HASH(user_id);	CREATE TABLE click_logs ( log_time DATETIME, -- No primary key user_id BIGINT, url VARCHAR, ip VARCHAR ) ENGINE=OLAP DUPLICATE KEY(log_time, user_id) -- Sorting only, not unique DISTRIBUTED BY HASH(user_id);

Aggregation types

Aggregation type	Purpose	Applicable column types	Typical scenarios
SUM	Sums the values for the same dimension columns	Numeric types (INT/BIGINT/DECIMAL)	Cumulative metrics such as sales and visits
MIN	Takes the minimum value for the same dimension columns	Numeric types/date types	Lowest values, earliest login time
MAX	Takes the maximum value for the same dimension columns	Numeric types/date types	Highest values, latest login time
REPLACE	Data written later completely overwrites the previous value (whether or not it's NULL)	Any type	A user's latest address, an order's final status
REPLACE_IF_NOT_NULL	Only non-NULL values overwrite the previous value (NULL values keep the old value) Requires the field's default value to be NULL (if the default value is an empty string '', it's treated as a valid value and overwrites the previous value).	Any type	Incremental updates to user information (existing fields are kept)

Create a SelectDB data source

In the DataOps Platform - Integration module, you can choose to add a SelectDB data source.

Data source configuration parameters

Fill in the basic information and data source configuration on the same page and pass the connectivity test to create the SelectDB data source.

Field name	Description
Basic Information
*Datasource Name	Must be unique within the DataOps Platform space. Can contain only letters, digits, and underscores, and can't start with a digit or an underscore
Remarks	Optional
Data source configuration
not distinguish environment / Independent setting	Choose one: not distinguish environment means the production and development environments share one configuration; Independent setting means the two environments are configured independently
*Server Address/IP	Address of the server where the SelectDB database runs. Separate multiple addresses with commas
*Port	Port used to access SelectDB
*Database	Name of a database already created in SelectDB
*Username	Username with permission to access the database
*Password	Password of the username
Advanced	Other advanced parameters required to connect to the database; you can customize them
Note: Cluster deployment mode by default

Parameters whose names start with * are required; parameters without * are optional.

Create an offline sync task

After you create the SelectDB data source and pass the connectivity test as described above, you can configure a SelectDB offline write task for your scenario.

SelectDB as the Data Target

Select SelectDB as the Data Target and configure the following parameters:

Field name	Description
*Data source (type)	Select SelectDB as the target type of the Data Target. This drop-down lists only the types of data sources that already exist in the current space, so SelectDB isn't listed until you create a SelectDB data source. Create one first through + Data Sources at the bottom of the drop-down or Data sources management
*Data source (data source)	A SelectDB data source registered on the data source management page; select it from the drop-down list. If you haven't created the data source yet, click the Data sources management button to create a SelectDB data source.
*Fully qualified name (catalog)	internal; external isn't supported yet
*Fully qualified name (database)	Name of the database to write to
*Target Table	The table to write to; select it from the drop-down list. There's a Create Table shortcut next to the drop-down
*Partition Field Value	You can define the partition field value through the Input Method. Suppose the partition field is days: Constant is a custom value: days = 2025-01-01 Parameter is a space parameter: days =${bd}, which takes the value of the base date at run time Field is a source table field: days = "dt1"
*Write Mode	Insert/upsert new data: Insert Into Overwrite: Insert Overwrite

Field mapping

After configuring the data source and the target, create field mappings. The system automatically syncs data from source fields to target fields based on the mappings. You can configure field mappings in three ways:

• Method 1: Custom selection. Select a source table field, then select the target field in the target table
• Method 2: Name Mapping. The system automatically maps fields with the same name in the source and target tables
• Method 3: Line Mapping. The system automatically maps fields in the same row

Note that each target field can correspond to only one source field

Write behavior of partitioned and non-partitioned tables

Target table	Partition status	insert	Insert Overwrite
Non-partitioned table	—	Write directly	Overwrite
Non-auto-partitioned table (Partition)	Partition exists	Write directly	Overwrite
Non-auto-partitioned table (Partition)	Partition doesn't exist	Write fails (Unknown partition)	Write fails (Unknown partition)
Auto-partitioned table (Auto Partition, enable_auto_create_when_overwrite = true by default)	Partition exists	Write directly	Overwrite
Auto-partitioned table (Auto Partition)	Partition doesn't exist	Create the partition and write	Create the partition and write

View original image

Basic information settings

Finally, set the basic information of the integration plan, including the plan name, owner, synchronization rate, and remarks.

When you're done, click Save to create the integration plan.

Note: The plan name can't be changed after it's saved

The details page looks like this:

Mount an offline sync plan on a Flow

Mount on Flow

1. Start mounting

• On the integration plan details page, click Mount on task flow in the upper-right corner

2. Select a Flow

• Select the target Flow from the drop-down menu

• To create a new Flow:

  • Click the New Flow shortcut button below the drop-down menu
  • Or go to the Dev module to create one

• Tip: If the target Flow isn't shown, click the ↻ refresh button on the right

3. Create a sync node

• In the Flow, create a node of the Offline sync plan type
• The node is linked to the current integration plan. When the node runs, it triggers a run of that integration plan

4. Complete mounting

• Click Create node and mount on it to complete the configuration
• After mounting succeeds, click Go to flow page to view the result right away

tip

Note: The task node created by mounting is in the unreleased state. We recommend going to the Flow and releasing the node.

Unmount from a Flow

To unmount an offline sync plan from a Flow:

• If the Flow hasn't been released yet, go to the Flow that the plan is mounted on and delete the Offline sync plan task node in Dev Mode.
• If the Flow has already been released, after deleting the Offline sync plan task node in Dev Mode, release the Flow again. This also unmounts the node from the Production environment.