Dark Mode

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

[feat](Variant) Support NestedGroup for variant columns with search DSL NESTED clause#60847

Open
eldenmoon wants to merge 11 commits intoapache:masterfrom
eldenmoon:feat-nested-commit
Open

[feat](Variant) Support NestedGroup for variant columns with search DSL NESTED clause#60847
eldenmoon wants to merge 11 commits intoapache:masterfrom
eldenmoon:feat-nested-commit

Conversation

Copy link
Member

eldenmoon commented Feb 26, 2026 *
edited
Loading

NESTED clause

Introduce NestedGroup support for tracking nested array structures in variant columns, including storage layer definitions, core data structures, read/write path integration, FE pruning rule enhancements, and NESTED clause in search DSL.

IDL definitions:

Add NestedGroupInfoPB, NestedOffsetsIndexPB in segment_v2.proto
Add ColumnPathInfo fields for nested group metadata
Add NESTED clause type and nested_path field in Exprs.thrift
Core data structures (BE):

nested_group_path.h: Path utilities for nested group identification
nested_group_provider.h/cpp: Provider interface with default no-op implementation
Offsets mapping index and offset tracking utilities for nested arrays
Variant reader/writer integration (BE):

Integrate NestedGroup-aware read path in variant_column_reader

Integrate NestedGroup-aware write path in variant_column_writer_impl
Add segment_iterator and segment_creator integration
Enhance path_in_data, variant_util, json_parser for nested structures
Add variant_nested_group_max_depth config
Update variant_statistics for nested group tracking
FE pruning and rewrite rules:

Add AccessPathExpressionCollector for access path collection

Refactor NestedColumnPruning and VariantSubPathPruning logic
Enhance ExpressionAnalyzer and expression context for pruning support
Add NestedGroup array support in Explode/ExplodeOuter
Search DSL NESTED clause:

Add NESTED token and grammar rules in SearchLexer/Parser.g4

Implement NESTED clause parsing in SearchDslParser
Implement NESTED function evaluation in function_search and vsearch
Tests:

nested_group_path_test, nested_group_path_filter_test, nested_group_provider_test
function_search_nested_test, vsearch_expr_test updates
PruneNestedColumnTest, VariantPruningLogicTest, SearchDslParserTest (FE)
4 new regression test suites for variant pruning

eldenmoon added 5 commits February 26, 2026 14:58
Add protobuf definitions for NestedGroup storage support:
- NestedGroupInfoPB for nested group statistics
- NestedOffsetsIndexPB for nested offsets index
- Fields in ColumnPathInfo for nested group metadata
- NESTED_OFFSETS_INDEX enum value

Add thrift definition for NESTED search clause:
- NESTED clause type in TSearchClause
- nested_path field for NESTED clause
Introduce the NestedGroup framework for tracking nested array structures
in variant columns:

- nested_group_path.h: Path utilities for nested group identification
- nested_group_provider.h/cpp: Provider interface with default no-op impl
- nested_offsets_mapping_index.h/cpp: Offsets mapping index for nested arrays
- offset_manager.h/offset_manager_impl.h: Offset tracking utilities

Tests:
- nested_group_path_test.cpp: Path utility tests
- nested_group_path_filter_test.cpp: Path filter tests
- nested_group_provider_test.cpp: Provider contract tests
Add NESTED clause to the search DSL for querying nested array structures
within variant columns.

FE changes:
- SearchLexer/Parser.g4: Add NESTED token and grammar rules
- SearchDslParser.java: Parse NESTED clause with path specification
- SearchPredicate.java: Support NESTED clause type

BE changes:
- vsearch.cpp: Evaluate NESTED clause in search expressions
- function_search.cpp/h: Implement NESTED function evaluation

Tests:
- SearchDslParserTest.java: FE parser tests
- function_search_nested_test.cpp: BE NESTED function tests
- vsearch_expr_test.cpp: Search expression tests
Improve FE rewrite rules for variant sub-path pruning and nested column
pruning to better support variant column access patterns.

FE changes:
- AccessPathExpressionCollector.java: New collector for access paths
- AccessPathPlanCollector.java: Enhanced plan collection
- NestedColumnPruning.java: Refactored nested column pruning logic
- VariantSubPathPruning.java: Enhanced sub-path pruning
- ExpressionAnalyzer.java: Expression analysis updates
- Explode/ExplodeOuter.java: NestedGroup array support

BE changes:
- vexpr.cpp/h: Expression node enhancements
- vexpr_context.h: Context changes for pruning support

Tests:
- PruneNestedColumnTest.java: Nested column pruning tests
- VariantPruningLogicTest.java: Variant pruning logic tests
Integrate NestedGroup support into the variant column read/write path
and related utility components.

Core reader/writer:
- variant_column_reader.cpp/h: NestedGroup-aware read path
- variant_column_writer_impl.cpp/h: NestedGroup-aware write path
- variant_statistics.h: Statistics tracking for nested groups
- column_reader.cpp/h: Column reader NestedGroup support
- segment_iterator.cpp: Iterator integration
- segment_creator.cpp: Creator integration

Utilities:
- path_in_data.cpp/h: Path utilities for nested structures
- variant_util.cpp/h: Variant utility enhancements
- json_parser.cpp: JSON parser improvements
- column_variant.cpp/h: Column variant changes
- config.cpp/h: variant_nested_group_max_depth config
- exec_env.h: Query cache setter

Tests & regression:
- Modified existing tests for compatibility
- 4 new regression test suites for variant pruning
Copilot AI review requested due to automatic review settings February 26, 2026 07:10
eldenmoon requested a review from zclllyybb as a code owner February 26, 2026 07:10
Copy link
Contributor

hello-stephen commented Feb 26, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Copilot started reviewing on behalf of eldenmoon February 26, 2026 07:11 View session
eldenmoon force-pushed the feat-nested-commit branch from 3271773 to f9c0e67 Compare February 26, 2026 07:12
Copilot AI reviewed Feb 26, 2026
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces NestedGroup-related infrastructure for VARIANT (multi-level array/object nesting) and adds a NESTED(path, query) clause to the search DSL, alongside expanded access-path/sub-path pruning support and coverage tests across FE/BE and regression suites.

Changes:

  • Extend FE search DSL parsing/AST + thrift explain to support a top-level NESTED() clause with a nested_path.
  • Add NestedGroup metadata/index plumbing (proto, iterators, mapping index) and related BE execution support for nested search evaluation.
  • Expand VARIANT access-path/sub-path pruning support (including numeric array indices) and add regression + unit tests.

Reviewed changes

Copilot reviewed 73 out of 74 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
regression-test/suites/variant_p0/test_variant_mixed_object_array_pruning_debugpoint.groovy New regression coverage for mixed object/array pruning with debugpoints
regression-test/suites/variant_p0/test_variant_compaction_root_empty_after_full.groovy New regression to ensure compaction doesn't empty VARIANT root
regression-test/suites/variant_p0/test_variant_access_path_pruning_debugpoint.groovy New regression coverage for access path pruning via debugpoints
regression-test/suites/variant_p0/test_sub_path_pruning.groovy Adds explode-related pruning assertions/explain checks
gensrc/thrift/Exprs.thrift Adds NESTED clause support via nested_path in thrift
gensrc/proto/segment_v2.proto Adds NestedGroup-related metadata and a nested offsets index proto
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/expressions/functions/scalar/SearchDslParserTest.java Adds FE unit tests for NESTED() parsing/validation rules
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/VariantPruningLogicTest.java New FE pruning logic tests for VARIANT access paths/subcolumns
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/PruneNestedColumnTest.java Extends pruning tests to cover VARIANT access path propagation
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/SearchDslParser.java Implements NESTED() parsing, AST fields, and top-level validation
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/generator/ExplodeOuter.java Allows EXPLODE_OUTER to accept VARIANT inputs
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/generator/Explode.java Allows EXPLODE to accept VARIANT inputs
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/VariantSubPathPruning.java Supports integer-like literal indices in VARIANT subpaths
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/SlotTypeReplacer.java Includes VARIANT in slot replacement/pruning decision
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/RewriteSearchToSlots.java Minor cleanup around variant-subcolumn comment/handling
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/NestedColumnPruning.java Extends pruning entry conditions & access-path handling for VARIANT
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/AccessPathPlanCollector.java Collects access paths for VARIANT and adjusts generate handling
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/AccessPathExpressionCollector.java Builds VARIANT access paths including numeric indices
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/ExpressionAnalyzer.java Marks VARIANT usage as "hasNestedColumns" for pruning pipeline
fe/fe-core/src/main/java/org/apache/doris/analysis/SearchPredicate.java Includes nested_path in explain and thrift conversion
fe/fe-core/src/main/antlr4/org/apache/doris/nereids/search/SearchParser.g4 Adds parser rule for nestedQuery
fe/fe-core/src/main/antlr4/org/apache/doris/nereids/search/SearchLexer.g4 Adds lexer mode/tokens for nested path scanning
be/test/vec/function/function_search_test.cpp Updates clause categorization/mapping for NESTED; moves tests out
be/test/vec/function/function_search_nested_test.cpp New BE unit tests focused on NESTED clause behavior
be/test/vec/exprs/vsearch_expr_test.cpp Updates inverted index context creation; adds NESTED fallback test
be/test/vec/exprs/vexpr_evalute_inverted_index_test.cpp Updates IndexExecContext construction signature
be/test/vec/exprs/try_cast_expr_test.cpp Adds include needed after context/header refactor
be/test/olap/rowset/segment_v2/segment_iterator_no_need_read_data_test.cpp Updates test expectation for extracted VARIANT read-data behavior
be/test/olap/rowset/segment_v2/nested_group_provider_test.cpp New tests for default NestedGroup provider contracts (CE behavior)
be/test/olap/rowset/segment_v2/nested_group_path_test.cpp New tests for nested group path utilities/constants
be/test/olap/rowset/segment_v2/nested_group_path_filter_test.cpp New tests for NestedGroup path filtering behavior
be/test/olap/rowset/segment_v2/external_col_meta_util_test.cpp Removes non-English comment note
be/src/vec/runtime/vparquet_transformer.cpp Fully-qualifies parquet types/exceptions to avoid ambiguity
be/src/vec/olap/olap_data_convertor.cpp Adds convertor for unsigned bigint (offset/length internal cols)
be/src/vec/json/path_in_data.h Adds array-depth helper + new path utility declarations
be/src/vec/json/path_in_data.cpp Implements prefix stripping + path append helpers
be/src/vec/json/json_parser.cpp Adjusts nested array handling & array alignment behavior
be/src/vec/functions/function_search.h Adds nested-search APIs and forward declarations
be/src/vec/functions/function_search.cpp Implements NESTED evaluation flow + execution context refactor
be/src/vec/exprs/vsearch.cpp Passes IndexExecContext/segment info; improves VARIANT subcolumn binding
be/src/vec/exprs/vexpr_context.h Extends IndexExecContext with segment pointer + iterator options
be/src/vec/exprs/vexpr.h Moves contains_blockable_function definition out-of-header
be/src/vec/exprs/vexpr.cpp Implements new contains_blockable_function; SEARCH_EXPR treated as slot-acting
be/src/vec/exec/scan/olap_scanner.cpp Uses parent_unique_id fallback for access-path maps
be/src/vec/common/variant_util.h Tracks presence of nested groups in variant extended info
be/src/vec/common/variant_util.cpp Nullable-aware array base-type detection; compaction skips extended schema when nested groups exist
be/src/vec/columns/column_variant.h Adds missing include for column vector usage
be/src/vec/columns/column_variant.cpp Minor include cleanup/formatting
be/src/runtime/exec_env.h Adds setter for inverted index query cache
be/src/olap/rowset/segment_v2/variant/variant_statistics.h Adds NestedGroup stats, pb (de)serialization updates, renames doc-value map
be/src/olap/rowset/segment_v2/variant/variant_external_meta_reader.cpp Removes non-English comment note
be/src/olap/rowset/segment_v2/variant/variant_column_writer_impl.h Introduces NestedGroup write provider integration and stats merging hooks
be/src/olap/rowset/segment_v2/variant/variant_column_writer_impl.cpp Wires NestedGroup provider into write pipeline; stats merge + writer refactors
be/src/olap/rowset/segment_v2/variant/variant_column_reader.h Adds NestedGroup reader structs and planning hooks
be/src/olap/rowset/segment_v2/variant/offset_manager_impl.h New shared implementation for offset padding/reading utilities
be/src/olap/rowset/segment_v2/variant/offset_manager.h New offset manager interface for NestedGroup offset handling
be/src/olap/rowset/segment_v2/variant/nested_offsets_mapping_index.h New mapping index interface + reader/writer declarations
be/src/olap/rowset/segment_v2/variant/nested_offsets_mapping_index.cpp Implements nested offsets mapping index build/read/map logic
be/src/olap/rowset/segment_v2/variant/nested_group_provider.h Defines NestedGroup provider interfaces, filters, and factories
be/src/olap/rowset/segment_v2/variant/nested_group_provider.cpp Default CE providers (no-op/disabled) + default path matching
be/src/olap/rowset/segment_v2/variant/nested_group_path.h New helpers/constants for nested-group path encoding
be/src/olap/rowset/segment_v2/variant/hierarchical_data_iterator.cpp Skips NestedGroup subcolumns in hierarchical merge; adds debug checks
be/src/olap/rowset/segment_v2/segment_iterator.cpp Uses full path for extracted column field_name; updates inverted-index type inference; passes segment/iter opts into IndexExecContext
be/src/olap/rowset/segment_v2/index_writer.cpp Uses column path when constructing extracted-column field_name
be/src/olap/rowset/segment_v2/index_file_writer.cpp Include tweak for atomic usage
be/src/olap/rowset/segment_v2/column_writer.h Adds NestedGroup provider/stats members for variant writers
be/src/olap/rowset/segment_v2/column_reader_cache.cpp Removes stray preprocessor character
be/src/olap/rowset/segment_v2/column_reader.h Exposes lazy-loading accessors for zone-map and ordinal indexes
be/src/olap/rowset/segment_v2/column_reader.cpp Implements lazy-loading accessors; recognizes new index type in init
be/src/olap/rowset/segment_creator.cpp Minor header/include cleanup and param renames
be/src/common/config.h Adds config knob for max nested group depth
be/src/common/config.cpp Defines default for nested group max depth
Comments suppressed due to low confidence (1)

be/src/vec/json/json_parser.cpp:35

  • json_parser.cpp references config::variant_max_json_key_length but no longer includes common/config.h, so this should fail to compile with config undefined. Add back the #include "common/config.h" (keep the IWYU pragma if needed).

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

eldenmoon force-pushed the feat-nested-commit branch from f9c0e67 to c61965d Compare February 26, 2026 07:26
eldenmoon commented Feb 26, 2026
eldenmoon commented Feb 26, 2026
eldenmoon commented Feb 26, 2026
eldenmoon force-pushed the feat-nested-commit branch from caa1183 to bfef111 Compare February 26, 2026 09:05
eldenmoon changed the title Feat nested commit [feat](Variant) Support NestedGroup for variant columns with search DSL NESTED clause Feb 26, 2026
eldenmoon requested a review from Copilot February 26, 2026 09:22
Copilot started reviewing on behalf of eldenmoon February 26, 2026 09:23 View session
Copilot AI reviewed Feb 26, 2026
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 65 out of 65 changed files in this pull request and generated 5 comments.

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +1039 to +1040
uint32_t idx =
static_cast(atoi(rel_str.substr(bucket_prefix.size()).c_str()));
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cast to uint32_t from int could be unsafe if atoi returns a negative value or the parsed integer is larger than UINT32_MAX. Consider using std::stoul with error handling instead of atoi, or add validation to ensure the parsed value is within valid range.

Copilot uses AI. Check for mistakes.
if (rel_str.find(DOC_VALUE_COLUMN_PATH) != std::string::npos) {
size_t bucket = rel_str.rfind('b');
int bucket_value = std::stoi(rel_str.substr(bucket + 1));
uint32_t bucket_value = static_cast(std::stoi(rel_str.substr(bucket + 1)));
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to line 1040, this cast from stoi result to uint32_t is potentially unsafe. The stoi function can throw exceptions for invalid input and may return negative values. Consider using std::stoul with proper error handling or add validation checks.

Copilot uses AI. Check for mistakes.
Comment on lines +323 to +324
CHECK(column->size() == nrows) << "column->size()=" << column->size() << ", nrows=" << nrows
<< ", path=" << node.path.get_path();
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CHECK macro will cause the program to abort on failure. For production code, consider using a more graceful error handling approach that returns a Status error instead of aborting the process, unless data corruption is guaranteed and aborting is the safest option.

Suggested change
CHECK(column->size() == nrows) << "column->size()=" << column->size() << ", nrows=" << nrows
<< ", path=" << node.path.get_path();
if (column->size() != nrows) {
return Status::InternalError(
"Column size {} not equal to nrows {} at path {}",
column->size(), nrows, node.path.get_path());
}

Copilot uses AI. Check for mistakes.
Comment on lines +629 to +631
const std::unordered_map& field_name_to_column_id,
std::shared_ptr& result_bitmap) const {
(void)field_name_to_column_id;
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The field_name_to_column_id parameter is marked with (void) to suppress unused warnings, but it's passed through the call chain. If this parameter is intended for future use, add a TODO comment explaining its purpose. If it's truly unused, consider removing it from the function signature to reduce complexity.

Suggested change
const std::unordered_mapint>& field_name_to_column_id,
std::shared_ptr& result_bitmap) const {
(void)field_name_to_column_id;
[[maybe_unused]] const std::unordered_mapint>& field_name_to_column_id,
std::shared_ptr& result_bitmap) const {
// TODO: Use field_name_to_column_id to map nested field names to column IDs
// when evaluating nested queries, so that nested_path and its children
// can be resolved consistently with the outer query.

Copilot uses AI. Check for mistakes.
eldenmoon force-pushed the feat-nested-commit branch from bfef111 to d71cce4 Compare February 26, 2026 09:47
Copy link
Member Author

eldenmoon commented Feb 26, 2026

run buildall

Copy link
Contributor

hello-stephen commented Feb 26, 2026

Cloud UT Coverage Report

Increment line coverage

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.33% (1796/2264)
Line Coverage 64.83% (32020/49393)
Region Coverage 65.55% (15984/24383)
Branch Coverage 56.03% (8498/15168)

Copy link

doris-robot commented Feb 26, 2026

TPC-H: Total hot run time: 28886 ms machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d71cce408225995e8677ec4d6b02996b11735eab, data reload: false

------ Round 1 ----------------------------------
============================================
q1 17708 4499 4350 4350
q2 q3 10735 811 537 537
q4 4742 359 260 260
q5 8202 1203 1020 1020
q6 237 180 148 148
q7 815 862 669 669
q8 10765 1501 1325 1325
q9 6766 4742 4748 4742
q10 6888 1895 1621 1621
q11 466 259 239 239
q12 758 568 474 474
q13 17802 4216 3435 3435
q14 226 232 229 229
q15 976 806 789 789
q16 730 715 684 684
q17 735 812 460 460
q18 6019 5771 5275 5275
q19 1193 987 621 621
q20 509 507 384 384
q21 4538 1848 1380 1380
q22 336 283 244 244
Total cold run time: 101146 ms
Total hot run time: 28886 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1 4532 4330 4356 4330
q2 q3 1797 2184 1732 1732
q4 863 1176 773 773
q5 4013 4384 4319 4319
q6 178 177 140 140
q7 1726 1626 1490 1490
q8 2420 2679 2548 2548
q9 8004 7558 7345 7345
q10 2669 2835 2423 2423
q11 509 436 413 413
q12 514 589 465 465
q13 3991 4615 3573 3573
q14 279 324 289 289
q15 874 814 822 814
q16 723 748 707 707
q17 1175 1706 1347 1347
q18 7007 6832 6703 6703
q19 946 839 937 839
q20 2096 2261 2045 2045
q21 4023 3461 3329 3329
q22 454 437 381 381
Total cold run time: 48793 ms
Total hot run time: 46005 ms

Copy link

doris-robot commented Feb 26, 2026

TPC-DS: Total hot run time: 183590 ms machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d71cce408225995e8677ec4d6b02996b11735eab, data reload: false

query5 4924 653 516 516
query6 328 222 226 222
query7 4213 484 272 272
query8 367 247 235 235
query9 8732 2788 2743 2743
query10 543 378 326 326
query11 17012 17804 17083 17083
query12 205 144 137 137
query13 1321 511 383 383
query14 7736 3406 3057 3057
query14_1 2925 2954 2879 2879
query15 216 202 182 182
query16 1090 520 467 467
query17 1254 734 642 642
query18 2834 477 374 374
query19 231 266 188 188
query20 137 142 132 132
query21 227 166 150 150
query22 5373 5209 4949 4949
query23 17283 16818 16658 16658
query23_1 16704 16719 16631 16631
query24 6889 1598 1218 1218
query24_1 1202 1248 1233 1233
query25 569 486 436 436
query26 1240 307 150 150
query27 2707 461 291 291
query28 4479 1877 1900 1877
query29 781 542 486 486
query30 304 243 204 204
query31 895 724 644 644
query32 78 71 65 65
query33 510 326 273 273
query34 901 907 555 555
query35 633 668 589 589
query36 1065 1110 1056 1056
query37 134 90 79 79
query38 2958 3031 2841 2841
query39 881 868 845 845
query39_1 872 807 837 807
query40 230 149 136 136
query41 63 59 57 57
query42 102 98 99 98
query43 372 390 346 346
query44
query45 198 192 182 182
query46 887 975 614 614
query47 2142 2115 2045 2045
query48 307 307 231 231
query49 618 450 369 369
query50 689 271 212 212
query51 4118 4113 4011 4011
query52 104 106 97 97
query53 286 329 281 281
query54 288 298 258 258
query55 87 84 80 80
query56 313 312 299 299
query57 1382 1322 1264 1264
query58 279 271 270 270
query59 2602 2692 2562 2562
query60 328 328 312 312
query61 156 153 152 152
query62 636 603 544 544
query63 324 283 275 275
query64 4825 1287 1026 1026
query65
query66 1374 449 346 346
query67 16407 16412 16206 16206
query68
query69 393 312 284 284
query70 1009 974 912 912
query71 335 307 285 285
query72 2750 2626 2421 2421
query73 542 552 323 323
query74 10002 9888 9713 9713
query75 2886 2765 2499 2499
query76 2305 1061 665 665
query77 355 426 299 299
query78 11240 11570 10714 10714
query79 1144 807 599 599
query80 1343 621 523 523
query81 558 291 249 249
query82 992 153 120 120
query83 361 255 258 255
query84 251 116 97 97
query85 885 481 428 428
query86 409 312 291 291
query87 3124 3095 2997 2997
query88 3578 2684 2680 2680
query89 444 371 343 343
query90 2022 176 170 170
query91 165 150 131 131
query92 76 75 72 72
query93 953 833 506 506
query94 647 309 296 296
query95 587 389 322 322
query96 630 513 232 232
query97 2444 2477 2419 2419
query98 231 218 218 218
query99 1004 998 909 909
Total cold run time: 255064 ms
Total hot run time: 183590 ms

Copy link
Member Author

eldenmoon commented Feb 26, 2026

run buildall

Copy link
Contributor

hello-stephen commented Feb 26, 2026

Cloud UT Coverage Report

Increment line coverage

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.33% (1796/2264)
Line Coverage 64.82% (32019/49393)
Region Coverage 65.52% (15976/24383)
Branch Coverage 56.03% (8499/15168)

Copy link

doris-robot commented Feb 26, 2026

TPC-H: Total hot run time: 28840 ms machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5e66e1acfcd0f059d991858bb4d7420f1667bc39, data reload: false

------ Round 1 ----------------------------------
============================================
q1 16556 4473 4319 4319
q2 q3 10266 788 526 526
q4 4598 371 254 254
q5 7575 1224 1028 1028
q6 173 183 147 147
q7 782 849 678 678
q8 9468 1474 1311 1311
q9 4814 4725 4709 4709
q10 6744 1882 1634 1634
q11 460 264 248 248
q12 720 568 482 482
q13 17761 4218 3441 3441
q14 236 233 208 208
q15 922 798 793 793
q16 724 731 678 678
q17 739 868 431 431
q18 6307 5422 5238 5238
q19 1130 987 592 592
q20 531 500 390 390
q21 4582 2024 1460 1460
q22 432 310 273 273
Total cold run time: 95520 ms
Total hot run time: 28840 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1 4727 4553 4613 4553
q2 q3 1793 2231 1739 1739
q4 852 1192 769 769
q5 4037 4376 4289 4289
q6 192 174 146 146
q7 1784 1675 1506 1506
q8 2467 2870 2518 2518
q9 7465 7437 7372 7372
q10 2673 2853 2406 2406
q11 537 452 419 419
q12 505 602 460 460
q13 3929 4386 3592 3592
q14 270 297 266 266
q15 841 810 849 810
q16 733 774 741 741
q17 1191 1511 1335 1335
q18 7216 6697 6648 6648
q19 902 894 876 876
q20 2107 2208 2028 2028
q21 3958 3585 3360 3360
q22 478 433 395 395
Total cold run time: 48657 ms
Total hot run time: 46228 ms

Copy link

doris-robot commented Feb 26, 2026

TPC-DS: Total hot run time: 183630 ms machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5e66e1acfcd0f059d991858bb4d7420f1667bc39, data reload: false

query5 4912 620 531 531
query6 338 234 207 207
query7 4216 471 281 281
query8 331 250 246 246
query9 8731 2815 2754 2754
query10 533 369 323 323
query11 16962 17351 17045 17045
query12 182 136 122 122
query13 1301 482 372 372
query14 6528 3377 3092 3092
query14_1 3020 2953 2820 2820
query15 220 200 190 190
query16 996 506 452 452
query17 1567 756 632 632
query18 3022 458 345 345
query19 204 207 176 176
query20 147 130 127 127
query21 216 136 114 114
query22 4908 4874 4971 4874
query23 17161 16761 16558 16558
query23_1 16724 16677 16656 16656
query24 7040 1607 1223 1223
query24_1 1222 1231 1217 1217
query25 538 449 410 410
query26 1238 256 148 148
query27 2812 461 295 295
query28 4382 1899 1903 1899
query29 829 569 467 467
query30 307 241 208 208
query31 873 719 639 639
query32 80 74 68 68
query33 525 318 283 283
query34 923 891 562 562
query35 628 697 604 604
query36 1019 1097 926 926
query37 133 94 79 79
query38 2955 2886 2864 2864
query39 916 905 835 835
query39_1 816 829 811 811
query40 228 153 135 135
query41 66 58 59 58
query42 109 108 105 105
query43 381 381 353 353
query44
query45 195 185 180 180
query46 886 976 601 601
query47 2112 2149 2035 2035
query48 308 315 230 230
query49 625 461 368 368
query50 691 279 218 218
query51 4053 4049 4001 4001
query52 106 108 98 98
query53 289 339 278 278
query54 292 262 253 253
query55 90 83 81 81
query56 321 319 307 307
query57 1355 1335 1278 1278
query58 302 274 267 267
query59 2645 2675 2545 2545
query60 343 334 327 327
query61 154 155 154 154
query62 624 586 535 535
query63 321 279 272 272
query64 4814 1405 1125 1125
query65
query66 1417 467 373 373
query67 16477 16437 16425 16425
query68
query69 402 317 302 302
query70 989 974 994 974
query71 349 318 309 309
query72 3014 2813 2422 2422
query73 540 542 333 333
query74 9990 9899 9719 9719
query75 2853 2753 2429 2429
query76 2113 1026 676 676
query77 355 393 324 324
query78 11362 11549 10691 10691
query79 1148 809 607 607
query80 1574 608 531 531
query81 574 280 256 256
query82 964 148 114 114
query83 349 254 247 247
query84 249 122 97 97
query85 1167 496 445 445
query86 418 307 294 294
query87 3136 3070 2959 2959
query88 3566 2723 2664 2664
query89 430 364 354 354
query90 1902 182 171 171
query91 185 160 137 137
query92 85 70 68 68
query93 1046 869 509 509
query94 638 314 323 314
query95 589 339 391 339
query96 631 507 233 233
query97 2459 2495 2382 2382
query98 248 220 211 211
query99 1006 1010 889 889
Total cold run time: 255047 ms
Total hot run time: 183630 ms

Copy link
Contributor

hello-stephen commented Feb 26, 2026

BE UT Coverage Report

Increment line coverage 52.04% (549/1055)

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.58% (19625/37321)
Line Coverage 36.22% (183366/506302)
Region Coverage 32.53% (142272/437369)
Branch Coverage 33.46% (61675/184328)

Copy link
Contributor

hello-stephen commented Feb 26, 2026

BE Regression && UT Coverage Report

Increment line coverage 60.92% (608/998)

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.51% (26134/36545)
Line Coverage 54.33% (274218/504752)
Region Coverage 51.85% (228935/441510)
Branch Coverage 53.09% (98153/184892)

eldenmoon force-pushed the feat-nested-commit branch from 5e66e1a to 922ae43 Compare February 27, 2026 01:49
Copy link
Member Author

eldenmoon commented Feb 27, 2026

run buildall

Copy link

doris-robot commented Feb 27, 2026

Cloud UT Coverage Report

Increment line coverage

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.33% (1796/2264)
Line Coverage 64.79% (32003/49393)
Region Coverage 65.51% (15974/24383)
Branch Coverage 55.99% (8493/15168)

Copy link

doris-robot commented Feb 27, 2026

TPC-H: Total hot run time: 28583 ms machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 922ae430ce32c6db3880050c0eb3959170a6819b, data reload: false

------ Round 1 ----------------------------------
============================================
q1 16531 4512 4278 4278
q2 q3 10205 774 521 521
q4 4541 359 252 252
q5 7479 1205 1012 1012
q6 176 176 147 147
q7 801 831 661 661
q8 9387 1485 1349 1349
q9 4768 4734 4651 4651
q10 6760 1874 1617 1617
q11 451 265 247 247
q12 689 567 471 471
q13 17577 4269 3430 3430
q14 229 229 222 222
q15 946 797 798 797
q16 721 740 683 683
q17 703 871 428 428
q18 5980 5354 5158 5158
q19 1280 970 633 633
q20 508 486 387 387
q21 4408 1854 1399 1399
q22 338 291 240 240
Total cold run time: 94478 ms
Total hot run time: 28583 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1 4482 4373 4347 4347
q2 q3 1754 2196 1737 1737
q4 840 1155 780 780
q5 3991 4315 4294 4294
q6 184 174 141 141
q7 1727 1623 1478 1478
q8 2411 2655 2523 2523
q9 7163 7906 7370 7370
q10 2631 2916 2387 2387
q11 504 429 411 411
q12 495 601 460 460
q13 3959 4427 3586 3586
q14 281 290 269 269
q15 927 906 809 809
q16 750 794 719 719
q17 1166 1576 1324 1324
q18 7187 6793 6707 6707
q19 890 902 882 882
q20 2053 2130 1994 1994
q21 3957 3463 3264 3264
q22 497 497 386 386
Total cold run time: 47849 ms
Total hot run time: 45868 ms

Copy link

doris-robot commented Feb 27, 2026

TPC-DS: Total hot run time: 184918 ms machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 922ae430ce32c6db3880050c0eb3959170a6819b, data reload: false

query5 5224 629 500 500
query6 336 218 200 200
query7 4217 458 274 274
query8 330 239 228 228
query9 8747 2774 2718 2718
query10 552 365 352 352
query11 16951 17494 17124 17124
query12 260 131 123 123
query13 1557 458 350 350
query14 6860 3286 3100 3100
query14_1 2915 2901 2971 2901
query15 204 196 177 177
query16 977 503 492 492
query17 1293 750 648 648
query18 3029 463 364 364
query19 229 220 210 210
query20 153 152 163 152
query21 244 149 120 120
query22 5804 5672 5591 5591
query23 17656 17273 17059 17059
query23_1 16830 16767 16685 16685
query24 6931 1688 1239 1239
query24_1 1232 1223 1212 1212
query25 558 487 446 446
query26 1237 267 151 151
query27 2749 482 283 283
query28 4369 1901 1891 1891
query29 802 574 495 495
query30 312 243 211 211
query31 884 716 647 647
query32 86 71 76 71
query33 550 319 284 284
query34 916 913 548 548
query35 649 678 594 594
query36 1115 1130 1030 1030
query37 122 89 83 83
query38 2911 2942 2897 2897
query39 880 849 859 849
query39_1 818 830 823 823
query40 223 149 135 135
query41 65 58 57 57
query42 107 102 99 99
query43 385 390 362 362
query44
query45 196 195 180 180
query46 875 989 606 606
query47 2140 2189 2087 2087
query48 315 321 244 244
query49 618 462 399 399
query50 683 272 227 227
query51 4131 4081 4061 4061
query52 104 105 94 94
query53 284 333 283 283
query54 288 262 257 257
query55 85 85 80 80
query56 308 300 304 300
query57 1351 1366 1294 1294
query58 285 271 268 268
query59 2524 2612 2516 2516
query60 344 336 319 319
query61 145 144 143 143
query62 605 581 534 534
query63 300 275 274 274
query64 4719 1271 1006 1006
query65
query66 1373 461 357 357
query67 16306 16273 16288 16273
query68
query69 388 298 291 291
query70 1026 978 884 884
query71 346 305 298 298
query72 2795 2694 2465 2465
query73 545 541 326 326
query74 9958 9904 9740 9740
query75 2835 2752 2437 2437
query76 2126 1036 674 674
query77 380 393 303 303
query78 11284 11621 10723 10723
query79 3151 791 594 594
query80 1800 613 543 543
query81 583 281 248 248
query82 936 152 113 113
query83 353 258 233 233
query84 260 123 95 95
query85 939 481 431 431
query86 447 302 295 295
query87 3122 3052 2997 2997
query88 3576 2689 2672 2672
query89 427 368 343 343
query90 2010 194 168 168
query91 169 156 134 134
query92 81 75 74 74
query93 1884 875 506 506
query94 648 324 293 293
query95 560 342 318 318
query96 648 510 231 231
query97 2474 2497 2413 2413
query98 236 222 216 216
query99 998 988 912 912
Total cold run time: 258788 ms
Total hot run time: 184918 ms

Copy link

doris-robot commented Feb 27, 2026

BE UT Coverage Report

Increment line coverage 52.04% (549/1055)

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.54% (19607/37321)
Line Coverage 36.16% (183061/506302)
Region Coverage 32.49% (142088/437369)
Branch Coverage 33.43% (61615/184328)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

zclllyybb Awaiting requested review from zclllyybb zclllyybb is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants