Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(clustering/rpc): change sync version to string #14078

Open
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

chronolaw
Copy link
Contributor

Summary

KAG-5894

Checklist

  • The Pull Request has tests
  • A changelog file has been created under changelog/unreleased/kong or skip-changelog label added on PR if changelog is unnecessary. README.md
  • There is a user-facing docs PR against https://github.com/Kong/docs.konghq.com - PUT DOCS PR HERE

Issue reference

Fix #[issue number]

@github-actions github-actions bot added core/clustering core/db cherry-pick kong-ee schedule this PR for cherry-picking to kong/kong-ee labels Jan 3, 2025
@chronolaw chronolaw marked this pull request as ready for review January 3, 2025 04:36
@chronolaw chronolaw marked this pull request as draft January 3, 2025 10:21
@chronolaw chronolaw marked this pull request as ready for review January 6, 2025 02:50
Copy link
Contributor

@StarlightIbuki StarlightIbuki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MInor issues. Not blockers

kong/clustering/services/sync/rpc.lua Outdated Show resolved Hide resolved
kong/clustering/services/sync/rpc.lua Show resolved Hide resolved
function _M:init_cp(manager)
local purge_delay = manager.conf.cluster_data_plane_purge_delay

-- CP
-- Method: kong.sync.v2.get_delta
-- Params: versions: list of current versions of the database
-- example: { default = { version = 1000, }, }
-- example: { default = { version = "1000", }, }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-- -- example: { default = { version = "1000", }, }
++ -- example: { default = { version = "v1000", }, }

miss a V?

Copy link
Contributor Author

@chronolaw chronolaw Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are many places of such examples, but I think that they are just an explanation, need not be exact code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do think that we should update them, we could do it in another PR.

local byte = string.byte
local CHAR_V = byte("V")

-- version string must start with char 'V'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel this format is not good for future extensibility. Maybe:

02xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Where xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx is the hex encoded version number. Essentially: "02%030x". This allows us to somewhat distinguish in the future when a new format is used. We can simply add 03, 04, etc.

Copy link
Contributor Author

@chronolaw chronolaw Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the string is like 02xxx then we can not distinguish between a hex string (ngx.md5) and a version string.

e.g.: a v1 config hash starts with 02 (02ABCDEF...) by chance.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we could set the version string a V0xxx V1xxx

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chronolaw The version string is provided by the DP, and DP should know if it is a valid version or not. If the string is not valid, then DP should simply send 02000000000000000000000000000000 instead. Does CP need to know about this detail?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I said above, if the version string stored in lmdb is 02ABCDEF..., and it is a config hash generated by a YAML by change, how DP could judge it valid?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to V02_


local is_valid_version
do
local VER_PREFIX = "V02_"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
local VER_PREFIX = "V02_"
local VER_PREFIX = "v02_"

I still prefer lowercased v, this looks like the format others use (e.g. GitHub tokens).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cherry-pick kong-ee schedule this PR for cherry-picking to kong/kong-ee core/clustering core/db size/M skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants