Appendix A: Varnish Programs¶
Varnish provides several tools to help monitor and control Varnish. varnishadm, used to access the management interface, is the only one that can affect a running instance of Varnish.
All the other tools operate exclusively on the shared memory log, often called shmlog in the context of Varnish. They take similar (but not identical) command line arguments, and use the same underlying API.
Among the log-parsing tools, varnishstat is so far unique in that it only looks at counters. The counters are easily found in the shmlog, and are typically polled at reasonable interval to give the impression of real-time updates. Counters, unlike the rest of the log, are not directly mapped to a single request, but represent how many times some specific action has occurred since Varnish started.
The rest of the tools work on the round robin part of the shmlog, which deals with specific requests. Since the shmlog provides large amounts of information, it is usually necessary to filter it. But that does not just mean “show me everything that matches X”. The most basic log tool, varnishlog, will do precisely that. The rest of the tools, however, can process the information further and display running statistical information.
If varnishlog is used to dump data to disk, varnishreplay can simulate a similar load. varnishtest is used for regression tests, mainly during development. Both are outside the scope of this course.
There is a delay in the log process, though usually it is not noticeable. The shmlog is 80MB large by default, which gives some potential history, but that is not guaranteed and it depends heavily on when the last roll-around of the shmlog occurred.
varnishtop -i TxStatus list length 6 hostname 3864.45 TxStatus 200 1001.33 TxStatus 304 33.93 TxStatus 301 3.99 TxStatus 302 3.00 TxStatus 404 1.00 TxStatus 403
- Group tags and tag-content by frequency
varnishtop groups tags and the content of the tag together to generate a sorted list of the most frequently appearing tag/tag-content pair.
Because the usefulness is only visible once you start filtering, it is often overlooked. The above example lists status codes that Varnish returns.
Two of the perhaps most useful variants of varnishtop is:
varnishtop -i TxUrlcreates a list of URLs requested from a web server. Use this this find out what is causing back-end traffic and start hitting items on the top of the list.
varnishtop -i TxStatuslists what status codes Varnish returns to clients. (As shown above)
Some other possibly useful examples are:
varnishtop -i RxUrldisplays what URLs are most frequently requested from a client.
varnishtop -i RxHeader -I 'User-Agent:.*Linux.*'lists User-Agent headers with “Linux” in it (e.g: most used Linux web browsers, that report them self as Linux).
varnishtop -i RxStatuswill list status codes received from a web server.
varnishtop -i VCL_callshows what VCL functions are used.
varnishtop -i RxHeader -I Referrershows the most common referrer addresses.
10.10.0.1 - - [24/Aug/2008:03:46:48 +0100] "GET \ http://www.example.com/images/foo.png HTTP/1.1" 200 5330 \ "http://www.example.com/" "Mozilla/5.0"
If you already have tools in place to analyze Apache-like logs (NCSA logs), varnishncsa can be used to print the shmlog as ncsa-styled log.
Filtering works similar to varnishlog.
# # # # ## ### ### ### ### ### | ### | ### | | ### |||| ### # |||| #### # |##|##### # # # # # +-------+-------+-------+-------+-------+-------+-------+-------+------- |1e-6 |1e-5 |1e-4 |1e-3 |1e-2 |1e-1 |1e0 |1e1 |1e2
Exercise: Try the tools¶
- Send a few requests to Varnish using
GET -e http://localhost:8000
- verify you have some cached objects using
- look at the communication with the clients, using
varnishlog. Try sending various headers and see them appear in varnishlog.
- Run siege against localhost while looking at varnishhist