Skip to content

Commit

Permalink
[opt](cache) enhance cache key computation by removing comments and t…
Browse files Browse the repository at this point in the history
…rimming SQL input (#46099)

- Currently, the SQL cache system in Doris may miss cache hits due to
semantically identical queries being treated as different because of:
  - Extra whitespace characters in the SQL query
  - SQL comments that don't affect the query execution
- For example, these queries are semantically identical but would
generate different cache keys:
  ```sql
  SELECT * FROM table;
  -- Same query with comments and extra spaces
  /* Comment */  SELECT   *   FROM   table  ;
  ```
- This PR improves the SQL cache hit rate by:
  - Trimming whitespace from SQL queries
  - Removing SQL comments before calculating the cache key MD5
- This ensures that queries that are semantically identical but differ
only in whitespace or comments will now hit the same cache entry,
improving cache efficiency and reducing unnecessary query executions
  • Loading branch information
Baymine authored and yiguolei committed Jan 6, 2025
1 parent ccb6c26 commit 009683e
Showing 1 changed file with 2 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
import org.apache.doris.mysql.FieldInfo;
import org.apache.doris.mysql.privilege.DataMaskPolicy;
import org.apache.doris.mysql.privilege.RowFilterPolicy;
import org.apache.doris.nereids.parser.NereidsParser;
import org.apache.doris.nereids.trees.expressions.Expression;
import org.apache.doris.nereids.trees.expressions.Variable;
import org.apache.doris.nereids.util.Utils;
Expand Down Expand Up @@ -349,7 +350,7 @@ public PUniqueId getOrComputeCacheKeyMd5() {

/** doComputeCacheKeyMd5 */
public synchronized PUniqueId doComputeCacheKeyMd5(Set<Variable> usedVariables) {
StringBuilder cacheKey = new StringBuilder(originSql);
StringBuilder cacheKey = new StringBuilder(NereidsParser.removeCommentAndTrimBlank(originSql.trim()));
for (Entry<FullTableName, String> entry : usedViews.entrySet()) {
cacheKey.append("|")
.append(entry.getKey())
Expand Down

0 comments on commit 009683e

Please sign in to comment.