A full stack, reactive architecture for general purpose programming. Algebraic and monadically composable primitives for concurrency, parallelism, event handling, transactions, multithreading, Web, and distributed computing with complete de-inversion of control (No callbacks, no blocking, pure state)
In an earlier post I mentioned that one goal of the new introductory curriculum at Carnegie Mellon is to teach parallelism as the general case of computing, rather than an esoteric, specialized subject for advanced students. Many people are incredulous when I tell them this, because it immediately conjures in their mind the myriad complexities…
I participated in the design and development of a couple of concurrency libraries for shared-memory multiprocessors long before such machines were popular. So when I started using java.util.concurrent I was already somewhat comfortable with the concepts.
JPPF enables applications with large processing power requirements to be run on any number of computers, in order to dramatically reduce their processing time. This is done by splitting an application into smaller parts that can be executed simultaneously on different machines.
with Philippa Gardner, Term 1, 2007/2008. Recommended Textbooks R. Milner. Communicating and Mobile Systems: the pi-Calculus. Cambridge University Press, various editions. (Introductory) D. Sangiorgi and D. Walker. The pi-Calculus: a Theory of Mobile Processes. Cambridge University Press, 2001. Online References and Tutorials A Calculus for Mobile Processes (Parts I/II) (by Robin Milner, Joachim Parrow, and David Walker). Also available from this site: [ Part I] [ Part II] The Polyadic Pi-Calculus: A Tutorial (by Robin Milner). Also available from this site: [(Postscript)] An Introduction to the Pi-calculus (by Joachim Parrow) A Brief Introduction to Applied Pi (by Peter Sewell) Asynchronous process calculi: the first-order and higher-order paradigms (Tutorial) (by Davide Sangiorgi)
The Little Book of Semaphores is a free (in both senses of the word) textbook that introduces the principles of synchronization for concurrent programming. In most computer science curricula, synchronization is a module in an Operating Systems class. OS textbooks present a standard set of problems with a standard set of solutions, but most students don't get a good understanding of the material or the ability to solve similar problems. The approach of this book is to identify patterns that are useful for a variety of synchronization problems and then show how they can be assembled into solutions. After each problem, the book offers a hint before showing a solution, giving students a better chance of discovering solutions on their own. The book covers the classical problems, including "Readers-writers," "Producer-consumer", and "Dining Philosophers." In addition, it collects a number of not-so-classical problems
Kamaelia - Concurrency made useful, fun In Kamaelia you build systems from simple components that talk to each other. This speeds development, massively aids maintenance and also means you build naturally concurrent software. It's intended to be accessible by any developer, including novices. What sort of systems? Network servers, clients, desktop applications, pygame based games, transcode systems and pipelines, digital TV systems, spam eradicators, teaching tools, and a fair amount more :)
This site will show how to write the concurrency section of A Tour of Go in Haskell. A Tour of Go is a famous tutorial of Go. Haskell has concurrency features similar to Go: lightweight thread, channel, etc.. So it should be interesting to compare equivalent concurrent programs in Haskell and Go.
This book aims to explain green threads by using a small example where we implement a simple but working program where we use our own green threads to execute code.
The C++ standardization committee is hard at work standardizing threads for the next version of C++. Some members recently met to discuss the issues, and The C++ Source was there. Read on to learn what the world’s leading experts on concurrency are plan
Parallel4 is a easy-to-use multi-threading API for Java an other JVM based languages like Groovy. It offers parallel versions of the "for" and "foreach" loops to leverage the full power of todays multi-core CPUs.
Parallel4's goals are:
* Simple API: Hide as many details of multi-threaded programming as possible from the API. Although it does not offer transparent/implicit multi-threading in a strict sense, it tries to come as close to this as a non-functional programming language allows.
* Familiar API: Offer well known constructs, e.g. the "for" loop and add parallelism to it.
* Easy adaptation of parallel programming: Based on a familiar API, it is easy to introduce multi-threading. Often, two additional lines of code are enough without imposing any structural changes.
* Adaptive: Let the "framework" make reasonable defaults to adapt to the execution environment, e.g. use as many threads as CPU cores are available.
* Performance: A low overhead makes it easy to decide whether to use parallel processing or not.
* Reliable: Multi-threading gets tricky when things go wrong unexpectedly. A well defined exception handling makes this a bit easier.
Akka is the platform for the next generation event-driven, scalable and fault-tolerant architectures on the JVM
We believe that writing correct concurrent, fault-tolerant and scalable applications is too hard. Most of the time it's because we are using the wrong tools and the wrong level of abstraction.
Akka is here to change that.
Using the Actor Model together with Software Transactional Memory we raise the abstraction level and provides a better platform to build correct concurrent and scalable applications.
For fault-tolerance we adopt the "Let it crash" / "Embrace failure" model which have been used with great success in the telecom industry to build applications that self-heals, systems that never stop.
Actors also provides the abstraction for transparent distribution and the basis for truly scalable and fault-tolerant applications.
Akka is Open Source and available under the Apache 2 License.
JavaScript is single threaded language but multi threading can be achieved in JavaScript using HTML5 Web Workers API. This will enable JavaScript code to run in background AKA parallel programming.
MINA is a simple yet full-featured network application framework which provides:
* Unified API for various transport types:
o TCP/IP & UDP/IP via Java NIO
o Serial communication (RS232) via RXTX
o In-VM pipe communication
o You can implement your own!
* Filter interface as an extension point; similar to Servlet filters
* Low-level and high-level API:
o Low-level: uses ByteBuffers
o High-level: uses user-defined message objects and codecs
* Highly customizable thread model:
o Single thread
o One thread pool
o More than one thread pools (i.e. SEDA)
* Out-of-the-box SSL · TLS · StartTLS support using Java 5 SSLEngine
* Overload shielding & traffic throttling
* Unit testability using mock objects
* JMX managability
* Stream-based I/O support via StreamIoHandler
* Integration with well known containers such as PicoContainer and Spring
* Smooth migration from Netty, an ancestor of Apache MINA.
We have designed, implemented, and evaluated AtomCaml, an extension to Objective Caml that provides a synchronization primitive for atomic (transactional) execution of code. A first-class primitive function of type (unit->'a)->'a evaluates its argument (which may call other functions, even external C functions) as though no other thread has interleaved execution. Our design ensures fair scheduling and obstruction-freedom. Our implementation extends the Objective Caml bytecode compiler and run-time system to support atomicity. A logging-and-rollback approach lets us undo uncompleted atomic blocks upon thread pre-emption, and retry them when the thread is rescheduled. The mostly functional nature of the Caml language and the Objective Caml implementation's commitment to a uniprocessor execution model (i.e., threads are interleaved, not executed simultaneously) allow particularly efficient logging.
I didn't see anyone post them yet, so here are the slides from Tim Sweeney's POPL talk entitled "The Next Mainstream Programming Languages: A Game Developer's Perspective". I know Tim and I aren't the only game developers who follow LtU, and I figure even non-game developers might find them quite interesting!
RP has extremely good performance and scalability properties. Many uses of RP in the Linux Kernel have resulted in several orders of magnitude performance improvement compared to locking and transactional memory. Is it easy to program with? RP is not difficult to program with. Allowing each execution sequence to proceed using its own view of memory, by default, simplifies concurrent programming because it prevents memory from changing spontaneously. Threads are guaranteed to observe coherent memory, i.e., memory will contain values that were actually written at some time in the past. Read paths also exhibit deterministic performance characteristics, since they can not block or retry. This feature simplifies programming of time-sensitive systems. Nevertheless, RP is a new programming paradigm with a new interface and there are several ways to misuse it. Read Copy Update (RCU), an early example of RP, has seen extensive use in the Linux Kernel at over 2000 uses
One of those things I have to do fairly often in multithreaded programming is send off a whole bunch of threads to do their thing while I do something else on the main thread until they’re done. For example, imagine you’re downloading a bunch of images from the web, you don’t want to call httpGet one image right after another, because network resources are slow and processing them takes up almost no CPU time. But on the other hand, forkIO doesn’t return anything, so a thread thunk will have to put its contents somewhere you can access them later. Thus, my short, simple solution, far too small to bother putting up on Hackage: module Control.Concurrent.Future where import Control.Concurrent future :: IO a -> IO (MVar a) future thunk = do ref <- newEmptyMVar forkIO $ thunk >>= putMVar ref return ref forceAll :: [MVar a] -> IO [a] forceAll = mapM takeMVar
CUDA lets you work with familiar programming concepts while developing software that can run on a GP This is the first of a series of articles to introduce you to the power of CUDA -- through working code -- and to the thought process to help you map applications onto multi-threaded hardware (such as GPUs) to get big performance increases. Of course, not all problems can be mapped efficiently onto multi-threaded hardware, so part of my thought process will be to distinguish what will and what won't work, plus provide a common-sense idea of what might work "well-enough". "CUDA programming" and "GPGPU programming" are not the same (although CUDA runs on GPUs). CUDA permits working with familiar programming concepts while developing software that can run on a GPU. It also avoids the performance overhead of graphics layer APIs by compiling your software directly to the hardware (GPU assembly language, for instance), thereby providing great performance.
We've discussed differences between parallel, computational systems and concurrent, event handling systems. Areas of differences include:
Determinism: desirable vs impossible
Sign of parallel safety: same results vs correct results
Parallelism bugs: easy to pinpoint vs impossible to define
Queues: implementation detail vs part of the interface
Preemption: nearly useless vs nearly essential
In an earlier post Over on the Twisted blog, Duncan McGreggor has asked us to expand a bit on where we think Twisted may be lacking in it’s support for concurrency. I’m afraid this has turned into a meandering essay, since I needed to reference so muc
Kilim is a message-passing framwork for Java that provides ultra-lightweight threads and facilities for fast, safe, zero-copy messaging between these threads.
Jetlang provides a high performance java threading library. The library is based upon Retlang.
The library is a complement to the java.util.concurrent package introduced in 1.5 and should be used for message based concurrency similar to event based actors in Scala.
The library does not provide remote messaging capabilities. It is designed specifically for high performance in-memory messaging.
Features¶
* All messages to a particular Fiber are delivered sequentially. Components can easily keep state without synchronizing data access or worrying about thread races.
* Single Fiber interface that can be backed by a dedicated thread or a thread pool.
* Supports single or multiple subscribers for messages.
* Subscriptions for single events or event batching
* Single or recurring event scheduling
* High performance design optimized for low latency and high scalability
* Publishing is thread safe, allowing easy integration with other threading models.
* Low Lock Contention - Minimizing lock contention is critical for performance. Other concurrency solutions are limited by a single lock typically on a central thread pool or message queue. Jetlang is optimized for low lock contention. Without a central bottleneck, performance easily scales to the needs of the application.
* Powerful Async Request/Reply Support
* Single jar with no dependencies except the jdk (1.6+)
* Integrates with any JVM language - jruby, scala, clojure, groovy, etc
Communication and Concurrency Lectures take place Monday and Thursday 14.00-14.50 in Appleton Tower, room M1, and are given by Colin Stirling. The syllabus of this module can be viewed in HTML through the Course Guide Background Reading: Milner's book "Communication and Concurrency, Prentice-Hall 1989" is important. Here is some background reading on ordinary, list, and tree (=structural) induction: ps pdf The Wikipedia article on Modal Logic makes interesting background reading. Note that there is a pointer to, but no article for, Hennessy-Milner logic: maybe you would like to write it?
Disco is an oss implementation of the Map-Reduce framework for distributed computing. Disco supports parallel computations over large data sets on unreliable cluster of computers. The Disco core is written in Erlang. Users of Disco typically write jobs in Python, which makes it possible to express even complex algorithms or data processing tasks often only in tens of lines of code. This means that you can quickly write scripts to process massive amounts of data. Disco was started at Nokia Research Center as a lightweight framework for rapid scripting of distributed data processing tasks. This far Disco has been succesfully used, for instance, in parsing and reformatting data, data clustering, probabilistic modelling, data mining, full-text indexing, and log analysis with hundreds of gigabytes of real-world data. Linux is the only supported platform but you can run Disco in the Amazon's Elastic Computing Cloud.
Peng Li's 2008 PhD dissertation, First, this dissertation presents a Haskell solution based on concurrency monads. This approach provides clean interfaces to both multithreaded programming and event-driven programming in the same application, but it also does not require native support of continuations from compilers or runtime systems. Then, this dissertation investigates for a generic solution to support lightweight concurrency in Haskell, compares several possible concurrency configurations and summarizes the lessons learned. The paper's summary explains what I like most about it: the project ... solves a systems problem using a language-based approach. Systems programmers, Haskell implementors and programming language designers may each find their own interests in this dissertation. Even if concurrency isn't your thing, section 6.3 describes the author's findings on the pros and cons of both purity and laziness in a systems programming context.
Various concurrency primitives have been added to sequential programming languages, in order to turn them concurrent. Prominent examples are concurrent buffers for Haskell, channels in Concurrent ML, joins in JoCaml, and handled futures in Alice ML. Even though one might conjecture that all these primitives provide the same expressiveness, proving this equivalence is an open challenge in the area of program semantics. In this paper, we establish a first instance of this conjecture. We show that concurrent buffers can be encoded in the lambda calculus with futures underlying Alice ML. Our correctness proof results from a systematic method, based on observational semantics with respect to may and must convergence.
We introduce a new lambda calculus with futures, Lambda(fut), that models the operational semantics of concurrent statically typed functional programming languages with mixed eager and lazy threads such as Alice ML, a concurrent extension of Standard ML. Lambda(fut) is a minimalist extension of the call-by-value lambda-calculus that is sufficiently expressive to define and combine a variety of standard concurrency abstractions, such as channels, semaphores, and ports. Despite its minimality, the basic machinery of Lambda(fut) is sufficiently powerful to support explicit recursion and call-by-need evaluation. We present a static type system for Lambda(fut) and distinguish a fragment of Lambda(fut) that we prove to be uniformly confluent. This result confirms our intuition that reference cells are the sole source of indeterminism.
ATS is a PL with a highly expressive type system from the framework Applied Type System. Both dependent types and linear types are available in ATS. The current implementation of ATS (ATS/Anairiats) is written in ATS itself. It can be as efficient as C/C++ and supports * Functional programming. ATS uses eager call-by-value eval, it also supports lazy call-by-need evaluation. Linear types in ATS can often make FP run with high efficiency * Imperative programming. While features considered dangerous in other languages (e.g., explicit pointer arithmetic and explicit memory mgmt) are allowed in ATS, the type system of ATS is still able to guarantee that no run-time errors can occur * Concurrent programming. ATS, equipped with a multicore-safe implementation of garbage collection, can support multithreaded programming through the use of pthreads and parallel let * Modular programming. The module system of ATS is largely infuenced by that of Modula-3
The result is a unified concurrency model providing both thread abstractions and event abstractions. We implemented the unified concurrency model in Haskell Our implementation demonstrates how to use these techniques by building an application-level thread library with support for multiprocessing and asynchronous I/O mechanisms in Linux. The thread library is type-safe, is relatively simple to implement, and has good performance. Application-level threads are extremely lightweight (scaling to 10,000,000 threads!) and our scheduler, which is implemented as a modular and extensible event-driven system, outperforms NPTL in I/O benchmarks.
S. Abramsky. Electronic Notes in Theoretical Computer Science, (2006)Proceedings of the Workshop "Essays on Algebraic Process Calculi" (APC 25)Proceedings of the Workshop "Essays on Algebraic Process Calculi" (APC 25).
D. Aumayr, S. Marr, E. Gonzalez Boix, and H. Mössenböck. Proceedings of the 16th ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes, page 157--171. ACM, (October 2019)
J. Bezivin. OOPSLA '87: Conference proceedings on Object-oriented programming systems, languages and applications, page 394--405. New York, NY, USA, ACM, (1987)