In May of 2009, two of Google’s top engineers wrote a 100 page paper entitled “The Datacenter as a Computer”. It covers many details of designing and maintaining large-scale services, as well as insight and best practices drawn from Google’s infrastructure and operations.
Abstract:
As computation continues to move into the cloud, the computing platform of interest no longer resembles a pizza box or a refrigerator, but a warehouse full of computers. These new large datacenters are quite different from traditional hosting facilities of earlier times and cannot be viewed simply as a collection of co-located servers. Large portions of the hardware and software resources in these facilities must work in concert to efficiently deliver good levels of Internet service performance, something that can only be achieved by a holistic approach to their design and deployment. In other words, we must treat the datacenter itself as one massive warehouse-scale computer (WSC). We describe the architecture of WSCs, the main factors influencing their design, operation, and cost structure, and the characteristics of their software base. We hope it will be useful to architects and programmers of today’s WSCs, as well as those of future many-core platforms which may one day implement the equivalent of today’s WSCs on a single board.
My Macbook Pro has a nasty habit of slowing to a crawl after I’ve used it for a few days. Although it has 4GB of memory, it eventually decides to starting paging out to disk even though my applications are only using 2-3 GB of memory. Once this happens, performance remains poor until a reboot. I’ve tried to close and relaunch all my applications, but it doesn’t really help. It seems the virtual memory manager prefers to page out memory that has not been used recently, whether it needs to or not. Basically, if you keep an application open long enough, it’s going to have some of it’s memory swapped to disk. Have you ever noticed how long it takes to open that Safari tab you opened yesterday, but never got around to reading until today? This issue has frustrated me for some time, so I decided to look for a solution. Fortunately I think I found it, but this solution is not recommended for everyone. Since I never use 4GB of memory at once, I decided to completely disable swap on my Macbook Pro. I ran the following commands to disable the dynamic_pager process and remove the swapfiles.
sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.dynamic_pager.plist
sudo rm /private/var/vm/swapfile*
Once that’s done you’ll want to restart and verify that dynamic_pager process is no longer running. In addition, make sure no swapfiles were created in /private/var/vm/. Now that swap is disabled, your system will no longer page out ram to disk. As long as your memory use remains below the amount of RAM in your machine, your system should function normally no matter how long you leave your applications open. However if you do use all your memory and your system is unable to swap, it is likely to crash and require a restart. If this happens, you should probably re-enable the dynamic_pager process by running the following command.
sudo launchctl load -wF /System/Library/LaunchDaemons/com.apple.dynamic_pager.plist
UPDATE: You can also disable Spotlight to free up more ram and reduce disk activity. The following commands are used to disable and enable Spotlight.
sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist
sudo launchctl load -wF /System/Library/LaunchDaemons/com.apple.metadata.mds.plist
One day this will be buried deep in the archives. 