I know a thing or two about robots.txt , there is a copy of robots.txt I have been using for most of my sites, can any one shed more light on robots.txt files?

----------example robots.txt file-----------------

# robots.txt for (site name should go here)
User-agent: *
Disallow:

-----------example ends---------------------