Willow Troubleshooting
The following are some tips for diagnosing and fixing problems with the proxy server.
Performance
The server is CPU bound. If it is running slow then it is most likely because is it consuming all of the CPU. To check this, first make sure that willow is the only thing running on the server. Then run top and see if there is any idle time. If there is little to no idle time and there are a number of willow processes taking up a lot of CPU then you have found the problem (if not then it is an entirely different problem so the rest of this tip isn't going to help).
Once you have determined the problem is CPU then you need to figure out what specifically the problem is. It could be that the server just can't handle the load. Check to see how many requests are being served by looking in the logs. Run top and see if a number of willow processes are switching in and out for the top CPU users, and that there isn't just a few consuming a lot of CPU for a long time. If this is the case it may simply be that the server doesn't have enough raw processing power to meet the demands of the network.
If there are just a few processes in top taking up a lot of CPU there could
be a client-side issue. One way to mitigate this is to define:
cpulimit = 30
in the willow.conf file. This will limit how much CPU a single process
will use (where the number is how many seconds at 100% CPU). Try different
numbers here and see if it changes the performance characteristics of the
server.
It is also helpful to look through the logs and see what sites are being
visited and what is being downloaded the most. Turning caching on will
speed up the server significantly and lighten the processing load so
make sure that something similar to:
cachedir = '/path/to/caching/dir'
cachetypes = ['IMAGE','VIDEO','APPLICATION/X-SHOCKWAVE-FLASH']
is defined in the willow.conf file.
The different filters in use also greatly affect the CPU consumption. The content filter in particular is what consumes the most CPU, so removing it from the list of filters temporarily can help determine how much of an impact it is having in overall performance. The domainfilter will short-circuit the rest of the filters running, so putting more known-to be good sites in it (especially ones that are frequently visited) can speed things up a lot.