Here’s a visualization concept I came up with a while back to look at the way search engines and word-of-mouth affects hit frequency on the iBiblio web-traffic log. iBiblio consists of around 420 sites. Each one of the circles you see represents one of the websites. The size of each pie slice inside grows with respect to the number of hits by individual search engines (see the legend for which ones). The size of the circle grows with respect to the overall number of hits by people other than search engines. Hits are counted by number of unique incoming IP addresses per day. Links get drawn between cliques of websites where more than 1/4th of the unique IP addresses are the same on that day, meaning, more or less, that those sites often share traffic.
This viz was developed entirely using Haskell and Cairo to crunch the weblogs and draw the data. The total amount of data was around 10TB (yes, terabytes), and the visualization took about a day to process into a static animation. Note that these are both size-compressed. The original is meant to run on a wall-sized (16′x9′) or on our specialized visualization dome. If you click on the image, you can see the original size.