When you have more than one gentoo machine, a good thing to do to manimize the bandwidth usage, is to run some local mirrors, so that you would get the file only once, for all the machines.
For the portage tree, you can easily see an implementation for it on http://www.gentoo.org/doc/en/rsync.xml. But the problem comes with the sources mirror, as they used most of the bandwidth. Also mirroring the whole gentoo sources, needs a lot of resources, and in the end, you keep a lot of files, that there is a good chance that nobody would ever get from your local network.
So what we need is something like a proxy server, to get the file from it, if someone got it before, or download it from the online mirror, send the file, and keep a local copy on it, in case someone else would request it afterwards.
I know you can find a lot of solutions for that, but I made this one, in case anybody would benifit from it.
The Way
First, setup a the folder where your cache would be, in my example it was /var/gentoo-mirror/
For the apache configuration:
Alias /gentoo-mirror /var/gentoo-mirror
</IfModule>
<Directory “/var/gentoo-mirror”>
ErrorDocument 404 /gentoo-mirror/404.php
Order allow,deny
Allow from all
</Directory>
This Configuration setup the the alias to our mirror “http://server/gentoo-mirror/” to go to the wanted folder. Also when a certain file is not found (404) it would call a script in that folder called 404.php.
This script now is reponsible for getting the new file, and saving it in the proper location.
So the 404.php would be something like mine.
define(‘PACKET_SIZE’,8194);
// CONFIG here
$serverPath = “/gentoo-mirror”;
$serverStorePath = “/var/gentoo-mirror”;
$mirrorServer = ‘de-mirror.org’;
$mirrorPath = ‘/distro/gentoo’;
function createDir($mainPath,$directory) {
$Dirs = explode(‘/’,$directory);
$path = $mainPath;
for ($i = 0; $i < count($Dirs); $i++) {
$path = $path.‘/’.$Dirs[$i];
if(!is_dir($path)) {
mkdir($path);
}
}
}
function echoHeadHeaders($server,$path) {
$fp = fsockopen($server,80);
if (!$fp) {
return false;
}
fwrite($fp, “HEAD $path HTTP/1.1\r\nHOST: $server\r\n\r\n”);
while (!feof($fp)) {
$line = fgets($fp);
if (stristr($line,“HTTP/1.1 404”) !== false) {
return false;
}
if (stristr($line,“Content-Length:”) !== false) {
header($line);
}
if (stristr($line,“Content-Type:”) !== false) {
header($line);
}
}
return true;
}
function getFileAndPrint($mirrorServer,$FromPath, $toPath) {
if (!echoHeadHeaders($mirrorServer,$FromPath)) {
return false;
}
$FromPtr = fopen(‘http://’.$mirrorServer.$FromPath,‘r’);
if (!$FromPtr) {
return false;
}
$ToPtr = fopen($toPath,‘w+’);
if (!$toPath) {
return false;
}
$buffer = ”;
while (!feof($FromPtr)) {
$buffer = fread($FromPtr,PACKET_SIZE);
echo $buffer;
fwrite($ToPtr, $buffer);
}
fclose($FromPtr);
fclose($ToPtr);
return true;
}
$requestedFile = str_replace($serverPath,”,$_SERVER[“REQUEST_URI”]);
header(“HTTP/1.1 200 OK”);
createDir($serverStorePath,dirname($requestedFile));
if (!getFileAndPrint($mirrorServer,$mirrorPath.$requestedFile,
$serverStorePath.‘/’.$requestedFile)) {
header(“HTTP/1.1 404 Not Found”);
echo “File Not Found”;
}
Please change the variables $mirrorServer and $mirrorPath to the online mirror you want to use.
Just to point it out: the mirror http://de-mirror.org/distro/gentoo/, the $mirrorServer takes the server name ‘de-mirror.org’ and the $mirrorPath is the rest after that ‘/distro/gentoo’
And now, we need to make sure that the folder we have is writable by apache. Also in php.ini that allow_url_fopen = On.
Hopefully, someone will benifit from this.
UPDATE:
The code from this post has been added to git-hub @ http://github.com/alaaibrahim/ala-scripts/ under the folder GentooCacheMirror.