Scrapy is built up on Twisted so while running in a single thread, a Scrapy application does not block while waiting data arrival from the network. so when data arrives, a callback function is called with this data as parameter. this event-driven or callback-based. Helpers includes * Parsers/interpreters for most common languages found in the web (xml/html/csv/javascript). * Network request and response pre and post process pluggable middleware to comply with specific network tasks, such as HTTP authentication, redirection, compression, network caching, network debug info logging, download retrying, cookie management, etc. * Scraping results post process pluggable middleware. * Pluggable extensions such as memory debugging, memory usage control and profiling, cpu profiling, controlling web/webservice and telnet consoles, distributed crawling cluster, scraping statistics and lot more. * Built-in event signaling. * Spiders quality test tools.