User-agent: NinjaBot
User-Agent: W3C-checklink
Allow: /
User-Agent:*
Disallow: /mobile/
Disallow: /mobile_jbri/