Web crawling ethics revisited: Cost, privacy, and denial of service
M. Thelwall, and D. Stuart. Journal of the American Society for Information Science and Technology, 57 (13):
1771-1779(2006)
Abstract
Ethical aspects of the employment of Web crawlers for
information science research and other contexts are reviewed.
The difference between legal and ethical uses of
communications technologies is emphasized as well as
the changing boundary between ethical and unethical
conduct. A review of the potential impacts on Web site
owners is used to underpin a new framework for ethical
crawling, and it is argued that delicate human judgment
is required for each individual case, with verdicts likely
to change over time. Decisions can be based upon an
approximate cost-benefit analysis, but it is crucial that
crawler owners find out about the technological issues
affecting the owners of the sites being crawled in order
to produce an informed assessment.
%0 Journal Article
%1 Thelwall-Web-2006
%A Thelwall, M.
%A Stuart, D.
%D 2006
%J Journal of the American Society for Information Science and Technology
%K Information_Ethics crawling ethics privacy web_ wismasys0809
%N 13
%P 1771-1779
%T Web crawling ethics revisited: Cost, privacy, and denial of service
%U http://www3.interscience.wiley.com/cgi-bin/fulltext/112776551/PDFSTART
%V 57
%X Ethical aspects of the employment of Web crawlers for
information science research and other contexts are reviewed.
The difference between legal and ethical uses of
communications technologies is emphasized as well as
the changing boundary between ethical and unethical
conduct. A review of the potential impacts on Web site
owners is used to underpin a new framework for ethical
crawling, and it is argued that delicate human judgment
is required for each individual case, with verdicts likely
to change over time. Decisions can be based upon an
approximate cost-benefit analysis, but it is crucial that
crawler owners find out about the technological issues
affecting the owners of the sites being crawled in order
to produce an informed assessment.
@article{Thelwall-Web-2006,
abstract = {Ethical aspects of the employment of Web crawlers for
information science research and other contexts are reviewed.
The difference between legal and ethical uses of
communications technologies is emphasized as well as
the changing boundary between ethical and unethical
conduct. A review of the potential impacts on Web site
owners is used to underpin a new framework for ethical
crawling, and it is argued that delicate human judgment
is required for each individual case, with verdicts likely
to change over time. Decisions can be based upon an
approximate cost-benefit analysis, but it is crucial that
crawler owners find out about the technological issues
affecting the owners of the sites being crawled in order
to produce an informed assessment.},
added-at = {2008-10-29T15:41:42.000+0100},
author = {Thelwall, M. and Stuart, D.},
biburl = {https://www.bibsonomy.org/bibtex/25d3c9538b573aac3672600bf68668ad8/zoka},
interhash = {64503c0b4a8ac7f94d86edbc6bf14566},
intrahash = {5d3c9538b573aac3672600bf68668ad8},
journal = {Journal of the American Society for Information Science and Technology},
keywords = {Information_Ethics crawling ethics privacy web_ wismasys0809},
number = 13,
pages = {1771-1779},
timestamp = {2008-10-29T15:41:42.000+0100},
title = {Web crawling ethics revisited: Cost, privacy, and denial of service},
url = {http://www3.interscience.wiley.com/cgi-bin/fulltext/112776551/PDFSTART},
volume = 57,
year = 2006
}