build from src -vs- install a RH binary (was RE: [wplug] Which R ed Hat version is best?)

Vanco, Donald VANCOD at PIOS.com
Wed Apr 23 15:06:54 EDT 2003


Mike Griffin wrote:
> rebuilding an rpm is like building from a tar.gz, it recompiles the
> code against your particular system and puts it back into the rpm
> package for ease of installation.
	So - if I compile against the source RPM, and the SPEC file has
available hooks in it to compile for multiple architectures - what gets
built? (--target <arch> operation)

> I won't get into exactly how this process works, but if you are
> curious...  check out the spec files (this is what the tar.gz/tar.bz2
> is compiled against.. it's a configured file telling the package what
> options to compile with, and what to put where.)
	I have  - and here's my notes on my experiences if anyone is
interested.......


	And I still stand by my statement that this requires far too many
development packages be installed to be a viable alternative to the average
joe/josephine.


>From a previous project:
(I'm not editing this - I've leaned much since then and what's in there
might not be "best practice")

	To build binary (i.e. installable) RPMs you need to use rpmbuild.
It's quite the powerful tool can do quite a few things with sources - either
source RPMs or tarballs.  However - there is a requirement of a ".spec" file
for a correct build.  I'll not get into the .spec file other than to say it
specifies things like architectures to build, directories and whatnot,
specific package dependencies, and deprecations.  Install a source package
and read the .spec file and you'll have a very clear idea of what it does -
it's the blueprint for the "building" of the package!  "man" is your friend,
or the excellent Red Hat docs available as part of your distro or online at
RedHat.com.  I cannot stress the value of reading this material if you want
to maintain your sanity and accomplish your goal in as short a time as
possible.

	READ!

	One thing not covered in the manual - if you build packages without
"cleaning up" during the process you will need MASSIVE amounts of free
space!!


Here's what I had to do for Advanced Server:
	To build sources you'll need to have the "-devel" packages
installed.  This can represent a unique challenge.  You'll either have to
take steps to insure that you only download those sources for packages you
have installed OR you'll have to install ALL of the development packages.
	The former can likely be accomplished by using the contents of
/var/log/rpmpkgs and comparing it to the updates available via Red Hat.
	The latter will require that you, at a minimum, install 150 "-devel"
packages, along with a myriad of other applications (details in a second)

	I chose the latter - so that I could simply compile ALL of the
updates and share them with you kind folks.

	So - I knew that I had to install "-devel" packages to avoid failed
dependencies.  Did I have to install them all?  NO - but it was much easier
than trying to pick-n-choose those I would require as dictated by the 218
packages I was to generate from the 84 source RPMs I pulled.

	Also - there are application dependencies.  There's 2 ways to
resolve these - try building the RPM until the process pukes and tells you
what to install, or grep your way through the .spec files, hunting for
"Build" (you'll see why in a sec) .  This gave me aboot a 99% success ratio.

	Lastly, there are those packages that can be compiled for differing
(x86) architectures - namely the kernel and glibc - that will yield a
"faster" system from the finished product.  Here again grep was my friend,
and looking for instances of "arches" proved to be all I needed to find
those packages that were "686" capable.  A more refined grep might involve
"auxarches" to look for "auxilliary architectures".

Let me do a case study for you.  I will select to build updates for kdegames
on RH 8.0. 
First - I want to install the source for the updates (in this case, it's the
source straight from the distro - but take a trip to the Magic Kingdom with
me and pretend it's an update - the principle is what counts)

So - I need to install kdegames-3.0.3-3.src.rpm
rpm -ivh kdegames-3.0.3-3.src.rpm
   1:kdegames               ###########################################
[100%]

...at first glance this looks like it went just fine - but that's not always
the case.  

Let's try to build (I'll detail command syntax in a bit):
rpmbuild -bb kdegames.spec
error: Failed build dependencies:
        qt-devel is needed by kdegames-3.0.3-3
        kdelibs-devel is needed by kdegames-3.0.3-3
        libjpeg-devel is needed by kdegames-3.0.3-3
        libpng-devel is needed by kdegames-3.0.3-3

	...as you can see - I have requirements yet to meet before I can
build.  Here's where grep comes in handy:
grep Build kdegames.spec
BuildPrereq: qt-devel kdelibs-devel zlib-devel libjpeg-devel libpng-devel
BuildRoot: %{_tmppath}/%{pkg}-buildroot
BuildRequires:desktop-file-utils >= %{desktop_file_utils_version}
BuildRequires: kdelibs
BuildRequires: kdelibs-devel
BuildRequires: autoconf
BuildRequires: automake
BuildRequires: libtool
BuildRequires: glibc-devel
BuildRequires: gcc
BuildRequires: gcc-c++
BuildRequires: libstdc++-devel
BuildRequires: qt-devel

	You can see the list above is long - and many of the things listed
by "BuildRequires" lines are installed on any system used to compile code -
so narrowing the grep to "BuildPrereq" will yield more concise results:
grep BuildPre kdegames.spec
BuildPrereq: qt-devel kdelibs-devel zlib-devel libjpeg-devel libpng-devel

	So  - I must satisfy those deps in order to build this package.

	Hopefully 2 things are becoming clear - building from source can
really call for a "bloated" system to accomplish things with any speed, and,
that being true, you just might be better off using a system with an
"Everything" installation as a foundation for "rolling your own" RPM
binaries.  In the long run it's much quicker!


	This is getting a bit long in the tooth already, so I'll get on to
building the packages.


	If you have a full system you can attempt to build all of the
binaries at once.  Fear not - if something is messed up you can restart the
process after fixing whatever is awry.

	In it's simplest form, you want to build the binary RPM for a given
package name.  Without going into ghastly detail, building RH RPMs from RH
source RPMs will utilize the tree in /usr/src/redhat.  A quick look at the
structure here (and the rpmbuild man page) should make this structure clear.
All of the directories have purpose.

	All of the .spec files I mentioned will be in ~SPECS.  Move into
this directory.
	
	Building an rpm at this point can be as easy as:
rpmbuild -bb <filename>.spec
	This will build the installable RPM and place it into the ../RPMS
tree (based on it's architecture - e.g. i386).  HOWEVER - there's a couple
potential issues here.
	- it does not clean up after itself
	- (in light of above) if a given package in a wildcard list does not
build correctly (say the 5th package in a list of 80) there are .spec files
that are still in place for packages that have already been built - meaning
that issuing the wildcard build again will re-build based on the persistent
presence of the .spec file.

	Here again - read the man pages to get a clear understanding of
this, but this is what I do:
(from within /usr/src/redhat/SPECS/)

rpmbuild --quiet --bb --clean --rmsource --rmspec *.spec
	- options to consider: (read the man page!)
	--ba
	-- target <arch>  (note that using the "clean up commands" in my
sample above will make this IMPOSSIBLE as the tree will be wiped clean)

	If you plan to build for multiple architectures -ba is likely for
you, or build them separate from the rest.

	What does that do?  It builds the binary (as opposed to the "ba"
build all option), cleans the build directory, removes the .spec file and
strips the source (from SOURCES)

	NOTE - the RPM database has no concept of installed .src RPMs!
While it can install them, it cannot remove them!  They must be cleaned as
part of the build or stripped out after the process.
	NOTE - not cleaning up (as in my example above) will bloat the
system as the builds progress.  Badly.

	Final Note - this process, on a 4-way x440 (yes, the compiler was
threading across multiple CPUs - much to my surprise) took a good 12 hours
to complete.  I had aboot 84 source update packages, which generated in
excess of 220 binary packages (and would have generated quite a few more
given the right build parameters).


	If it's not clear (after reading the man pages) all of the build
binaries are in the /usr/src/redhat/RPMS tree - most in i386, i686, and
noarch.



Possibility I did not investigate:
	Supposedly, rpmbuild can do ALL of the above (on a "Everything"
install I'm sure) with a singe command (i.e. install the source RPM, AND
build the binary packages) with
rpmbuild --rebuild <packagename-x.x-xx.src.rpm>
	I did not try that as I knew my system would have many unmet
requirements.  If it's not clear, it would be advisable to use the --clean
and the --rm(option) options here to minimize system bloat.


	Hopefully, you can see from this process that there is great value
in paying for an update channel into Red Hat Advanced Server.  Given the
splintering of this product line (Red Hat Enterprise Linux Advanced Server,
Red Hat Enterprise Linux Advanced Server ES, Red Hat Enterprise Linux
Advanced Workstation) the update channels are only going to grow.





> 
> On Wednesday, April 23, 2003, at 02:20 PM, Vanco, Donald wrote:
> 
>> Mike Griffin wrote:
>>> You can upgrade the packages manually.  Download newer packages, i
>>> recommend source packages which can be identified by .src.rpm. To
>>>    install an rpm: rpm -Uvh packagename.rpm
>>> 
>>> For a .src.rpm:
>>>    rpmbuild --rebuild packagename.rpm
>>>    cd /usr/src/redhat/RPMS/i386
>>>    rpm -Uvh packagename.rpm
>> 
>> 	I'm curious - why would you recommend that someone build the
>> binaries from source rather than simply installing the source
>> binaries available from RH?  Seems like a whole lot of work for
>> nothing additional. Further - other than the binaries doesn't this
>> also rebuild the source RPMS, etc?? I haven't done too much with
>> building from source RPMS (other than all of the AS updates) and my
>> experiences were that TONS (i.e. ALL) development packages had to be
>> installed and there was a slew of garbage left on my system (albeit
>> in the usr/src/redhat tree) that I had to clean sweep later. 
>> 
>> 	So what's the point of working with sources when the (signed)
>> binaries already exist?? 
>> 
>> Don
>> 
>> 
>>> On Wednesday, April 23, 2003, at 01:57 PM, Sal Mangiapane wrote:
>>> 
>>>> I never upgraded before.  Do I just download the rpm and then what?
>>>> 
>>>> Thanks and God bless,
>>>> 
>>>> sal
>>>>> -----Original Message-----
>>>>> From: wplug-admin at wplug.org
>>> [mailto:wplug-admin at wplug.org]On Behalf Of
>>>>> Mike Griffin
>>>>> Sent: Wednesday, April 23, 2003 1:45 PM
>>>>> To: wplug at wplug.org
>>>>> Subject: Re: [wplug] Which Red Hat version is best?
>>>>> 
>>>>> 
>>>>> RH 7.3 is pretty stable. Last I heard, 8 had some bugs and 9 is
>>>>> experimental. 
>>>>> 
>>>>> But, why not just update the packages in question? beats
>>>>> reinstalling the OS. 
>>>>> 
>>>>> Mike
>>>>> 
>>>>> On Wednesday, April 23, 2003, at 01:35 PM, Sal Mangiapane wrote:
>>>>> 
>>>>>> Hi all,
>>>>>> 
>>>>>> I am currently running Red Hat 7.2 and I want to upgrade.
>>>  Should I
>>>>>> upgrade to 7.3, 8, or 9?
>>>>>> 
>>>>>> All I really need are an updated gcc and vi.  I exclusively use
>>>>>> SSH and command line interfaces.  But, I want to stay current
>>>>>> without getting bleeding edge.



More information about the wplug mailing list