libwebp in Atomic overriding EPEL

General Discussion of atomic repo and development projects.

Ask for help here with anything else not covered by other forums.
prupert
Forum Regular
Forum Regular
Posts: 573
Joined: Tue Aug 01, 2006 2:45 pm
Location: Netherlands

libwebp in Atomic overriding EPEL

Unread post by prupert »

Hi Scott,

There are libwebp packages in the Atomic repo with the same version as in EPEL. Is there a good reason for overriding the packages in EPEL?
Lemonbit Internet Dedicated Server Management
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8330
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: libwebp in Atomic overriding EPEL

Unread post by scott »

Its duplicated from epel, it is a dependency for SCL PHP 7.0
prupert
Forum Regular
Forum Regular
Posts: 573
Joined: Tue Aug 01, 2006 2:45 pm
Location: Netherlands

Re: libwebp in Atomic overriding EPEL

Unread post by prupert »

Why not put a requirement on the EPEL repo? That may be much better than duplicating packages, and consequently overriding them because of the priorities setup in the Atomic repo config, and having the risk of Atomic forgetting to update when EPEL updates that package.
Lemonbit Internet Dedicated Server Management
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8330
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: libwebp in Atomic overriding EPEL

Unread post by scott »

Yeah this is a great question, when it makes sense to duplicate and when not to duplicate. This gets into the kinds of meta problems that you have when running a really large repo (1 million+ systems in any given week, given the systems we see hitting us**)

TL; DR: It made sense to duplicate this for this specific package (SCL PHP 7.0) because of the popularity of this specific project, and not everyone uses EPEL***.

The nitty gritty: There are so many things to talk about here, I dont want to make this a giant response.... Needless to say, nobody ever asks me what its like to do this :P I'm going to use some specific examples to show some ways this can pan out so I dont make this huge. Suffice to say there is no one-size-fits-all approach to running a repo.

* When packaging you always want to ask yourself who the audience is that is going to use this. I use 3 buckets: They are the non-technical, technical, and technical-but-very-busy.

* What is the value proposition for the package. Ie, what business problem is it solving. My buckets are: Compliance, new business functionality, reduce support overhead


Example 1: Asterisk
Bucket: Technical audience, you can expect them to make many steps
Bucket: new business functionality
You can expect this audience to either have a 3rd party repo, or if they dont, they'll *PROBABLY* be OK with adding one. The caveat to that is the kind of application a business would be using, and they may have Approved Product Lists/Approved Vendor Lists. Meaning they might be technically aware of a dependency available from a source like EPEL, but not be allowed to use it for legal/policy reasons. We have many many "enterprise" customers with APL/AVL issues that use the repo for that reason. (ie. They pay for an atomic feed & we handle APL paperwork). We'd duplicate epel packages like speex here.


Example 2: Openvas
Bucket: Technical-but-very-busy
Bucket: Compliance, reduce support overhead
Like the asterisk example above, they have the skills but run thousands of systems & no time. Frequently these are pure RHEL users, frequently with no internet access. They use things like satellite/spacewalk, are restricted by APL's, or dont have the registrations set up because of the size of the environment. We've duplicated libksba here (this is from base on centos, and from util on rhel) since its come up over and over in the forums here and on the openvas lists & irc.

Example 3: PHP Panda
Bucket: Non-technical
Bucket: reduce support overhead
Adoption of any project with the a non-technical audience drops with each additional step required to install it. So you take that into consideration, and the kind of feedback you'd get back from that kind of an audience if their is an issue (they will recognize a problem, but may not have the experience to articulate it in a meaningful way. Then its on you to figure it out from the information provided). Multiply that by the size of the project, in this case I knew it was around 200,000 installs, and extrapolate the impact on an un-resolvable dependency. If this was in a technical bucket, it might have had a chance at being purely external, but if you look at the kinds of business problems epel solves, not a lot of them are are high impact things like php or asterisk (epel dropped asterisk as of EL7. We'll be picking that up, see example 1 above). So in this case, it was duplicated to lower overall support costs, and to increase the rate of adoption.

So there you have it. Since I started Atomic in 2001, Ive seen a lot of things & people come and go that looked promising and didnt pan out (oh redcarpet, how I miss you). In 2001, there was no yum, no git, no fedora, no centos, no epel. There were only 3 or 4 other packagers out there back then, to now with great tools like scl, mock, and copr that make it easier for everyone to do it. There are entire portals dedicated to building communities (like bohdi) around packaging.

Closing note here, Atomic started from single plesk project dimitri asked me to do to prototype a competitor to Microsoft Exchange (this never panned out :P) to what it is now. And for what its worth... the first 2 packages in atomic were clamav and.... <drum roll> PHP.

** There are both public and private repos of our repos that we do not have visibly into to, so we have no idea how big their user bases are. If you'd like to access to the rsync feed for mirroring, let us know.

*** Why dont they use EPEL? Id say its either because they don't know it exists, or they do know it exists, but are provisioning from a local repo for mass deployment reasons.
Post Reply