The Apache Hadoop project develops open-source software for reliable, scalable, distributed computing, including:
* Hadoop Core, our flagship sub-project, provides a distributed filesystem (HDFS) and support for the MapReduce distributed computing framework.
* HBase builds on Hadoop Core to provide a scalable, distributed database.
* Pig is a high-level data-flow language and execution framework for parallel computation. It is built on top of Hadoop Core.
* ZooKeeper is a highly available and reliable coordination system. Distributed applications use ZooKeeper to store and mediate updates for critical shared state.
* Hive is a data warehouse infrastructure built on Hadoop Core that provides data summarization, adhoc querying and analysis of datasets.
It is from this operational asymmetry that complexity in event processing is required. In other words, as distributed networks grow in complexity, it is difficult to determine causal dependence when trying to diagnosis a distributed networked system. Most who work in a large distributed network ecosystem (cyberspace) understand this. The CEP notion of “the event cloud” was an attempt to express this complexity and uncertainly (in cyberspace).
The goal of the Condor® Project is to develop, implement, deploy, and evaluate mechanisms and policies that support High Throughput Computing (HTC) on large collections of distributively owned computing resources. Guided by both the technological and sociological challenges of such a computing environment, the Condor Team has been building software tools that enable scientists and engineers to increase their computing throughput
J. Weerasinghe, F. Abel, C. Hagleitner, and A. Herkersdorf. Ubiquitous Intelligence and Computing and 2015 IEEE 12th Intl Conf on Autonomic and Trusted Computing and 2015 IEEE 15th Intl Conf on Scalable Computing and Communications and Its Associated Workshops (UIC-ATC-ScalCom), 2015 IEEE 12th Intl Conf on, page 1078--1086. IEEE, (2015)