Search engine crawlers
Robots.txt file in docroot.
Causes issues with not finding all content linked on site. Pagination should be followable.
Events list, news list, all content list like glossary, degree explorer, etc.
Need to be 100% transparent to search engines for better seo
Removed one line: Disallow: *?* from robots.txt.