Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

prototype skip encoding #8615

Open
wants to merge 7 commits into
base: max/sql-row-interface
Choose a base branch
from
Open

Conversation

max-hoffman
Copy link
Contributor

@max-hoffman max-hoffman commented Nov 28, 2024

prototype before/after:

BenchmarkOltpIndexScan-12    	     206	   5190640 ns/op	 5243491 B/op	  140229 allocs/op
BenchmarkOltpIndexScan-12    	     650	   1785686 ns/op	 3883499 B/op	   20767 allocs/op

The premise of this is a custom row type:

type ProllyRow struct {
	key, value val.Tuple
	kd, vd     val.TupleDesc
	ords       []int // placement access
}

func (r ProllyRow) GetBytes(i int, typ sql.Type) ([]byte, error) {
	pos := r.ords[i]

	if pos < r.kd.Count() {
		ret := r.kd.GetField(pos, r.key)
		return ret, nil
	}

	pos -= r.kd.Count()
	ret := r.vd.GetField(pos, r.value)
	return ret, nil
}

The idea is to go from val.Tuple straight to wire values:

	outVals := make([]sqltypes.Value, len(sch))
	var err error
	if br, ok := row.(sql.BytesRow); ok {
		for i, col := range sch {
			buf, err := br.GetBytes(i, col.Type)
			outVals[i] = sqltypes.MakeTrusted(col.Type.Type(), buf)
			if err != nil {
				return nil, err
			}
		}
		return outVals, nil
	}
  • correctness considerations for when storage encoding is not consistent with MySQL byte encoding
  • certain values will have to be materialized to satisfy engine logic that operates on Go types
  • the ordinal mapping above is 1-step, while main has 2-steps
  • this doesn't work with virtual columns

@max-hoffman
Copy link
Contributor Author

#benchmark

Copy link

github-actions bot commented Dec 2, 2024

@max-hoffman max-hoffman changed the title test skip encoding prototype skip encoding Dec 2, 2024
@max-hoffman
Copy link
Contributor Author

#benchmark

Copy link

github-actions bot commented Dec 2, 2024

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

read_tests from_latency to_latency percent_change
covering_index_scan 0.62 0.62 0.0
groupby_scan 16.71 17.95 7.42
index_join 2.26 2.39 5.75
index_join_scan 1.44 1.52 5.56
index_scan 55.82 14.46 -74.1
oltp_point_select 0.27 0.27 0.0
oltp_read_only 5.47 5.47 0.0
select_random_points 0.67 0.64 -4.48
select_random_ranges 0.65 0.67 3.08
table_scan 55.82 14.46 -74.1
types_table_scan 147.61 39.65 -73.14
write_tests from_latency to_latency percent_change
oltp_delete_insert 6.21 6.21 0.0
oltp_insert 3.07 3.07 0.0
oltp_read_write 11.65 10.84 -6.95
oltp_update_index 3.13 0.45 -85.62
oltp_update_non_index 3.07 0.45 -85.34
oltp_write_only 6.21 5.28 -14.98
types_delete_insert 6.55 6.67 1.83

@max-hoffman
Copy link
Contributor Author

#benchmark

Copy link

github-actions bot commented Dec 3, 2024

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

read_tests from_latency to_latency percent_change
covering_index_scan 0.61 0.62 1.64
groupby_scan 16.41 17.63 7.43
index_join 2.26 2.39 5.75
index_join_scan 1.44 1.52 5.56
index_scan 54.83 15.0 -72.64
oltp_point_select 0.26 0.26 0.0
oltp_read_only 5.28 5.28 0.0
select_random_points 0.63 0.62 -1.59
select_random_ranges 0.63 0.64 1.59
table_scan 55.82 14.46 -74.1
types_table_scan 142.39 39.65 -72.15
write_tests from_latency to_latency percent_change
oltp_delete_insert 6.21 6.21 0.0
oltp_insert 3.07 3.07 0.0
oltp_read_write 11.45 11.45 0.0
oltp_update_index 3.13 3.13 0.0
oltp_update_non_index 3.02 3.02 0.0
oltp_write_only 6.21 6.21 0.0
types_delete_insert 6.55 6.55 0.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants