Start testing with Ruby 3.4 on every PR

[deivid-rodriguez]

What was the end-user or developer problem that led to this PR?

In preparation for final Bundler 2.6.0 and RubyGems 3.6.0 releases, I'd like to make sure everything works with Ruby 3.4.

What is your fix for the problem, implemented in this PR?

Add 3.4.0-preview2 to our CI.

Make sure the following tasks are checked

• Describe the problem / feature
• Write tests for features and bug fixes
• Write code to solve the problem
• Make sure you follow the current code style and write meaningful commit messages without tags

[MSP-Greg]

@deivid-rodriguez

Please consider using head. All the head builds available in GHA are tested (see https://github.com/ruby/ruby-dev-builder CI), and only replace the previous build if all the tests pass.

This has been in place for quite a while, and was done because other CI providers were running head builds and not testing them. They just ran make install and replaced the previous build. No one liked that...

[deivid-rodriguez]

We already have a daily job that tests with head, that works fine.

[deivid-rodriguez]

To clarify, I understand that only replacing the previous dev build if all ruby tests pass is a good thing, but there's still some inherent unstability to testing against ruby-head that I don't want all PRs to inherit. We've been testing with ruby head daily for a while now and I address failures with it when I find time without allowing them to reach all PRs and result if false CI failures. This works very well so I see no reason to change it.

[MSP-Greg]

I understand your position.

I think there is a converse. If RubyGems/Bundler commits automatically move to ruby/ruby, doesn't the potential exist for breakage in ruby/ruby, since you're not testing against Ruby head?

I'm not sure how that movement is initiated, so...

[deivid-rodriguez]

Yes, that's a good point. I think that's not ideal either and it indeed causes issues sometimes. I proposed https://bugs.ruby-lang.org/issues/16778 to try fix it permanently but it's not going to happen.

Most issues caused by this automatic synchronization are usually due to how ruby-core runs bundler tests different, so to alleviate that, we do test with ruby-head on every PR in a special "ruby-core" job that runs tests in the same way ruby-core runs them. It's not perfect (due to the instability of head), but it does catch issues sometimes, so we live with it. When failures are still not caught, usually ruby-core maintainers address them quickly.