Running ./configure && make install builds and installs the libraries or executables directly from the source code.
The make install step basically just copies the final files into your system. Many sources come with a special make uninstall rule to remove them again, but this is not guaranteed and of course only works as long as you have the configured sources around. Also, this does not take care of required dependencies.
Often there is only the source code available for a certain package, so this is the only way to go. Also, ./configure usually accepts lots of options allowing you to tailor your package.
Not being able to find out what software installed which file, and the lack of a reliable way to remove them from the system are major shortcomings of this approach.
Note: use the make method only when there is no .rpm package available or you need to change some compile-time options.
RPM (Redhat Package Manager)
rpm installs already configured and compiled software in your system and it also comes with a uninstall to get rid of it again. The packages have to be created by somebody. This person already decided on what features to include and how to best integrate the package into your system layout. It also comes with a list of dependencies.
Since rpms are used for many distributions there, you will often want to make sure that this rpm was written for your distribution so that install paths, dependencies and other housekeeping things integrate well.
On Debian systems, the equivalent package format is .deb and the installation and database is handled by the dpkg tool.
yum is an additional wrapper around rpm. It keeps its own database of rpm files available for your distribution, generally in online repositories. For the stable versions of most distributions all packages inside that database will play well with each other. This database can be searched (e.g. with yum search some_name).
It will also automatically resolve dependencies for you. Packages (and with some extra help their dependencies) can be easily uninstalled as well.
On Debian systems, the equivalent repository and dependency-resolution tools are provided by Apt (apt-get and aptitude).