I get the feeling too many people are sleeping on CentOS Stream because of how the CentOS #Linux EOL thing went down.
I mean sure, the messaging wasn't great but it happened and it is what it is. Don't let that detract from the greatness of the #CentOS Stream project in its own right and as a collaboration point for all the #RHEL based distros.
The gravity seems lost on most that for the first time ever the RHEL development process happens in the open. That's amazing. I love it. #community
@maxamillion While I agree, it's definitely not helping things that CentOS Stream still goes EOL well before RHEL, AlmaLinux, etc.
@maxamillion That doesn't mean it isn't good or useful, but that it takes more intentional consideration in deploying it than CentOS Linux did.
@maxamillion This is unfortunately not trivial. I've had some trouble with filing Github issues against CentOS Stream because the upstream only now "supports Rocky", etc. Having to cut through the FUD around it gets tiring, honestly, but CS is still really nice for having fast track container patches and as a CI platform. I mean, I'm still using it very much, but I understand why there's preference for the rebuilds.
@vwbusguy@mastodon.online@maxamillion I would also be quick to point out that CS is part of the success story for AlmaLinux as they build against CS repos ahead of their own releases, which is part of why they're so quick to have stuff out when RHEL releases (especially compared to Rocky and Oracle who tend to have significant more lag).
@maxamillion I don't have the bandwidth for this right now, but sometime I'd be curious to test the differences between CS and AlmaLinux with the testing repo enabled.
@vwbusguy biggest difference is you can actually contribute to CS. Alma is a rebuild and if you need to fix something or get a feature into Alma then you submit a merge request to CS and wait until it lands in Alma downstream.
@maxamillion Oh sure, I understand that as someone who is into contributing to open source and building stuff. It's a harder sell in a place that's just looking for a platform to run their production stuff. No one wants to risk having to explain that prod is down because of a patch in CS that broke something and the vendor no longer "supports CentOS" (even if it's eventually going to be a problem for the vendor in the next RHEL release).
@vwbusguy @maxamillion Translates to “we don’t want to pay, but we want assurances and garantuees for free and will blame others happily” #SarcasmButOnlyHalf
@jwildeboer @vwbusguy seems that way
@maxamillion @jwildeboer That's one way to look at it. I say this as some someone who maintains CentOS Stream, RHEL, and the rebuilds in my dayjob. Frankly, the rebuilds mean you're likely to get vendor support and don't have to deal with rhsm (or acquiring contacts if they don't already exist). Often those decisions get made more out of convenience than social responsibility or even brand loyalty.
@maxamillion @jwildeboer Personally, the biggest reasons *I* might choose AlmaLinux over CentOS Stream or RHEL for something:
* Longer lifespan
* Desire less frequent updates
* Don't feel like messing with subscription-manager
At least with AlmaLinux, we can also contribute with testing and such for ELevate, which also helps upstream (and other downstreams).
@maxamillion @jwildeboer The CentOS SIG projects like Hyperscale are a major reason I might pick CentOS Stream. None of the others have btrfs options at install, for example.
I've also found CentOS Stream to be really nice for standalone container hosts since the container runtime (podman, crun, etc) patches land sooner.
@vwbusguy @maxamillion @jwildeboer Yep, subscription manager is really not great. Number 1 reason to avoid RHEL.
While these days with OCP4 things are all fine, in OCP3 times with subscribed RHEL hosts, subscription manager was the main reason for outages. Don't get me wrong, not frequent, but if hosts didn't come up, then 8/10 times it was a problem with rhsm.
@sheogorath @maxamillion @jwildeboer So much this. Having a mixed environment between traditional RHEL and ose3 hosts somehow caused a terrible mess of things as entitlements kept getting misplaced. I had an Ansible playbook run constantly to fsck rhsm. And if a box was offline long enough to not trust rhsm keys anymore, it was difficult to recover.
@vwbusguy @sheogorath @jwildeboer OSE3 was a bit of a mess. OCP4 is much better
@maxamillion @sheogorath @jwildeboer OCP4 has been its own awkward mess. It traded crazy complex Ansible stuff for complex terraform and operator abstracted stuff, which is great until it needs lower level debugging and then it's generally more complicated, especially with the low level stuff needing to go through ignition/ostree/manifests, etc. Still, I very much like that it plays better with the rest of the k8s world outside of RH than ose3 did.
@vwbusguy @sheogorath @jwildeboer oh yeah, debugging operators is insanity
@maxamillion @jwildeboer (I'm aware that ELevate is leapp based, but I've actually had success with ELevate where I've had straight leapp royally fail on RHEL.)
@maxamillion @jwildeboer Oh yeah, and the rebuilds post security errata, which you don't get with CentOS Stream.
@vwbusguy @jwildeboer I'm curious where they get that errata from if they're rebuilding from CS. 
@maxamillion @vwbusguy @jwildeboer They aren't rebuilding from CS. They're rebuilding from exported RHEL sources. Their errata is copied from RHEL with text search and replace, no validation.
@carlwgeorge @vwbusguy @jwildeboer isn't redistribution of errata a violation of the data usage terms? Or did I imagine that?
@maxamillion @vwbusguy @jwildeboer IIRC you can view the errata without a subscription, thus no usage terms. But I don't know for sure.
@maxamillion @carlwgeorge @jwildeboer Yeah, I believe it's public. If it's security errata, it's going to be tied to a CVE anyway, so it should be public in some manner anyway unless it's embargoed.
@carlwgeorge @maxamillion @jwildeboer Correct. To clarify, I didn't mean to suggest that official AlmaLinux releases are from CS and not RHEL sources, but rather that they pull from CS sources for their own internal development and testing ahead of RHEL based releases. AFAIK, the errata is copied from RHEL, but it's still less clear about what CVEs are patched when in CS repos unless it's in the rpm changelog and there's something that can scrape and aggregate that info.
@vwbusguy @maxamillion @jwildeboer The "desire less frequent updates" point doesn't make sense to me. You get updates when you want to apply them. If you want to update weekly, monthly, quarterly, etc., then that's how often you get updates. Nothing is forced on you.
The difference is that the updates are made available regularly as they pass QA, rather than the staircase model (many updates at minor release time, few updates in between).
@vwbusguy @maxamillion It sounds like you're describing a shop that applies updates in prod without first testing them in non-prod. How do they explain when a regression or change in RHEL brings down prod? It's not really different. Testing updates before deploying them in prod is a best practice no matter what OS you're using.
@maxamillion @carlwgeorge Everyone has a test environment. Some are fortunate enough to have one that isn't their production.
@vwbusguy @maxamillion @carlwgeorge for everyone else, there's rawhide.
@vathpela @maxamillion @carlwgeorge You mean I shouldn't be running production on ELN?! 
@vwbusguy @maxamillion It's going to be very close, because CentOS Stream is very close to RHEL itself. When I've measured it, it's typically 90-95% the same software versions depending on where in the cycle you are.
@carlwgeorge @maxamillion Sometimes things that are in CS that aren't in RHEL end up in the dev/testing repo for AlmaLinux. For example, when OpenLDAP got abruptly pulled and reinstated in RHEL, it became available in CS first and was in the Alma testing repos ahead of the next RHEL release that added it back in.
@vwbusguy @maxamillion I don't think that's accurate. Have they said that anywhere?
@carlwgeorge @maxamillion It's something I've observed directly in their git repos. For example, leading up to the RHEL9 release, many of their repositories had CS9 branches. Same is true for their cloud images that had CentOS Stream 9 as the base image before AlmaLinux 9 was released and the repo rebased to it.
@vwbusguy @maxamillion Having c9s git branches doesn't mean they actually built against CS. They likely just sync those branches from GitLab as a reference, or when they need to build a buildroot-only package. I could be missing something, but lacking actual confirmation from their team I wouldn't assume they build against CS at all.
Temporarily basing their images against CS to validate build pipelines makes sense, but clearly isn't an ongoing thing helping their rebuild turnaround time.