The Wayback Machine, from Internet Archive, allows you to search web pages which have been archived. Not every website will have entries, but if you needed to see, for example, what the Yahoo! homepage from May 17, 2002 looked like, this would be a great source.
CyberCemetery, hosted by the University of North Texas, is an archive of government operations which are either no longer in operation or have issued a final report.
Governmentattic.org has electronic copies of scores of federal documents released under the Freedom of Information Act.
