Squeryl
Home Learn more Scaladoc Community Follow _squeryl on TwitterTwitter
A Scala ORM and DSL for talking with Databases with minimum verbosity and maximum type safety
Write compiler validated statements.
Squeryl statements that pass compilation won’t fail at runtime. Refactor your schema as often as is required, the Scala compiler and your IDE will tell you exactly which lines of code are affected.
Never repeat yourself
The Composability of Squeryl statements allows you to define them
once and reuse them as sub queries within other statements.
Write declaratively
Write as declaratively as SQL, only with less boilerplate. SQL’s declarativeness is preserved, not encapsulated in a lower level API that requires imperative and procedural code to get things done.
Explicitly control retrieval granularity and laziness
A significant part of optimizing a database abstraction layer is to choose for every situation the right balance between fine and large grained retrieval, and the optimal mix of laziness and eagerness. Data retrieval strategies are explicit in Squeryl rather than driven by configuration like current generation Java ORMs read more
Sausalito is a product that enables you to develop, test, deploy, and host scalable web applications in the cloud. On this page, you find some reading material in case you're interested in
* technical details about Sausalito,
* user experience reports, or
* presentations and demos that we gave.
FlockDB is a distributed graph database for storing adjancency lists, with goals of supporting:
* a high rate of add/update/remove operations
* potientially complex set arithmetic queries
* paging through query result sets containing millions of entries
* ability to "archive" and later restore archived edges
* horizontal scaling including replication
* online data migration
Non-goals include:
* multi-hop queries (or graph-walking queries)
* automatic shard migrations
FlockDB is much simpler than other graph databases such as neo4j because it tries to solve fewer problems. It scales horizontally and is designed for on-line, low-latency, high throughput environments such as web-sites.
Twitter uses FlockDB to store social graphs (who follows whom, who blocks whom) and secondary indices. As of April 2010, the Twitter FlockDB cluster stores 13+ billion edges and sustains peak traffic of 20k writes/second and 100k reads/second.
SYMPLiK EXZELLENZ is a Java program to manipulate table data in Oracle database by using Microsoft Excel file. Unlike Oracle WebADI, More4Apps or other Excel-driven solution, this program does not use Office VBA (Macro) to process the data. It is a pure Java solution incorporated with Apache POI and Oracle JDBC driver to download, upload, insert, update and delete data from Oracle database tables.
Sqoop is a tool designed to import data from relational databases into Hadoop. Sqoop uses JDBC to connect to a database. It examines each table’s schema and automatically generates the necessary classes to import data into the Hadoop Distributed File System (HDFS). Sqoop then creates and launches a MapReduce job to read tables from the database via DBInputFormat, the JDBC-based InputFormat. Tables are read into a set of files in HDFS. Sqoop supports both SequenceFile and text-based target and includes performance enhancements for loading data from MySQL.
AutoPatch was born from the needs of using an agile development process while working on systems that have persistent storage. Without AutoPatch, developers usually can't afford the maintenance headache of their own database, and DBAs are required just to apply changes to all of the various environments a serious development effort requires.
The very application of database changes becomes an inefficient, error-prone, expensive process, all conspiring to discourage any refactoring that touches the model, or being a bottleneck when model changes are made.
AutoPatch solves this problem, completely.
With AutoPatch, an agile development process that requires a database change looks like this:
* Developer alters the model, which requires a change to the database
* Developer possibly consults a DBA, and develops a SQL patch against their personal database that implements the alteration
* Developer commits the patch to source control at the same time as they commit their dependent code
* Other developers' and environments' databases are automatically updated by AutoPatch the next time the new source is run
This represents streamlined environment maintenance, allowing developers to cheaply have their own databases and all databases to stay in synch with massively lower costs and no environment skew.
That's what AutoPatch does.
Clusters with one database? Multiple schemas? Logical migrations, instead of just DDL changes? Need to do something special/custom? Need to distribute your changes commercially? All without paying anything? No problem.
Very interesting approach!
"Apache Empire-db is an Open Source relational data persistence component which allows database vendor independent dynamic query definition as well as safe and simple data retrieval and updating. Compared to most other solutions like e.g. Hibernate, TopLink, iBATIS or JPA implementations, Empire-db takes a considerably different approach, with a special focus on compile-time safety, reduced redundancies and improved developer productivity."
Das Problem …
PHP-Scripte werden nach einer bestimmten Laufzeit abgebrochen (normalerweise nach 30 Sekunden), und so funktioniert ein Backup mit diversen Tools nur bis zu einer bestimmten Größe.
Braucht das Script länger als die 30 Sekunden, so wird es vom Server einfach abgebrochen, und man kommt nicht mehr an sein Backup heran. Gleiches gilt für das Wiedereinspielen eines Backups.
Wer einmal ein Dumpfile von Hand in viele kleinere Einzelabschnitte zerlegt hat, um eine Datenbank wieder herzustellen, der weiß genau, wovon wir reden.
MySQLDumper ist die Lösung …
MySQLDumper umgeht den Timeout-Error mit Hilfe eines kleinen Tricks: Er liest nur eine bestimmte Anzahl von Datensätzen aus der Tabelle aus, merkt sich, wie weit er gekommen ist, und ruft sich anschließend selbst auf. Dadurch erhält das Script bei jedem Aufruf wieder die vollen 30 Sekunden und umgeht so den Timeout-Error. Das gleiche Prinzip benutzt MySQLDumper auch beim Wiederherstellen der Daten.
MySQLDumper kann die Daten beim Sichern sofort packen. Auch das Wiederherstellungsscript kann direkt aus dieser gepackten Datei lesen, ohne dass sie auf dem Server entpackt werden muss! Natürlich kann man die Datei auch ungepackt lassen, aber spätestens beim Hochladen eines Backups weiß man dies zu schätzen.
DbFit is a set of FIT fixtures which enables FIT/FitNesse tests to execute directly against a database. This enables developers to manipulate database objects in a relational tabular form, making database testing and management much easier then with xUnit-style tools. The library is free to use, released under GNU GPL.
Spring, JPA, and JTA with Hibernate and JOTM
2007-04-24 20:35
have been struggling for a couple of hours today to modify a Spring JPA configuration with a single datasource, Hibernate as the JPA provider and the JpaTransactionManager to a configuration with two XA datasources, Hibernate as the JPA provider, and the JtaTransactionManager with JOTM as the standalone JTA provider.
since the Spring and Hibernate reference manual and Javadoc documentation merely contain a number of hints on how to configure JPA with a JTA transaction manager and others are struggling as well i decided to post how i finally got it to work.
With the addition of generics in Java 5, writing a custom DAO for each domain object is no longer required. There are a wide variety of articles on creating generic DAOs, but my current project uses the approach from this IBM DeveloperWorks article. This approach was choses mainly because of the clearly written article and the integration with Spring. You should be able to extend any generic DAO based on Spring to implement the stored procedure configuration.
The agimatec tools contain some utilities to enable generation of files, source code etc. from sql files and annotated java classes.
The agimatec-dbmigrate tool is a database migration/script execution framework that can execute sql scripts, groovy scripts and provides a lot of features to assist in database schema migration. (tested with Oracle and Postgres)
Hattori is a Java framework that simplifies Data Transfer Object population in n-tier web applications by using Java 5 annotations.
The framework is meant to be used at service layer and presentation layer. Given a domain object graph, a DTO object model subset is populated by Hattori which becomes available for modification which will could be later taken to the original domain object graph. Normally, the DTOs are populated at the service layer and sent to the presentation layer for that modification.
The population process is done by evaluating each Data Transfer Object annotated with the ObjectPopulation annotation, identifying operations to be executed on each one using one of these options: implicit rules, annotation rules or your own specific per-object defined java code.
About
AutoDAO is a Generic DAO on steroids implementation for Java.
This project was inspired by Don't repeat the DAO! article by Per Mellqvist.
Main features
* Ready to use CRUD operations
* Zero persistence code for common DAO queries
* Annotation-driven auto-configuration
* Spring Framework custom namespace for easy to use configuration
* Hibernate/JPA support
What does it do?
Given an accessible database schema, the Hibernate POJO Generator produces all the Java code necessary to access each field in each table via the Hibernate persistence framework. Additionally, the generator also creates all the necessary helper classes and test units for each component.
The DataNucleus project provides products for the management of application data in a Java environment. Our aim is to provide good quality open source products to handle data in all of its forms, wherever it is stored. This goes from persistence of data into heterogeneous datastores, to providing methods of retrieval using a range of query languages, and eventually on to the analysis of data and tools for managing data quality. Your use of DataNucleus products will mean that you don't need to take significant time in learning the oddities of particular datastores, or query languages and instead use a single common interface for all of your data, and instead your team can concentrate their application development time on adding business logic and let DataNucleus take care of data management issues.
The Mogwai ERDesigner is a entity relation modeling tool such as ERWin and co. The only difference is that it is Open Source and does not cost anything. It was designed to make database modeling as easy as it can be and to support the developer in the whole development process, from database design to schema and code generation. This tool was also designed to support a flexible plug in architecture, to extend the system simply by installing a new plug in. This way, everybody can implement new features and tools to make ERDesigner fit the requirements.
ERDesigner NG
* is based on Java and can be run on Windows and Unix systems
* has a powerfull WYSIWYG for physical database design
* handles tables, relations, indexes and comments
* supports subject areas
* supports MySQL, Oracle, Microsoft SQLServer and Postgres
* creates the SQL DDL statements for schema creation
* has an integrated schema version control system
* can generate schema migration scripts for every change
* stores the database definition as XML files for further processing
* can export the database schema as GIF, BMP, JPEG or SVG files
* has an integrated reverse engineering module for existing schemas
* it is based on GPL license
* support is available by authors and newsgroups
The Eclipse Persistence Services Project (EclipseLink) project's goal is to provide a complete persistence framework that is both comprehensive and universal. It will run in any Java environment and read and write objects to virtually any type of data source, including relational databases, XML, or EIS systems. EclipseLink will focus on providing leading edge support, including advanced feature extensions, for the dominant persistence standards for each target data source; Java Persistence API (JPA) for relational databases, Java Architecture for XML Binding (JAXB) for XML, J2EE Connector Architecture (JCA) for EIS and other types of legacy systems, and Service Data Objects (SDO).
The Envers project aims to enable easy versioning of persistent JPA classes. All that you have to do is annotate your persistent class or some of its properties, that you want to version, with @Versioned. For each versioned entity, a table will be created, which will hold the history of changes made to the entity. You can then retrieve and query historical data without much effort.
Similarly to Subversion, the library has a concept of revisions. Basically, one transaction commit is one revision (unless the transaction didn't modify any versioned entities). As the revisions are global, having a revision number, you can query for various entities at that revision, retrieving a (partial) view of the database at that revision.
The goal of Autofetch is reduce the modularity penalty and programmer burden of specifying associations which should be loaded with an object query. These specifications are sometimes are called fetch profiles, prefetch directives, or joins. These specifications are an important performance optimization because they reduce the number of round-trips to a persistence store whether that be a relational database, object database, or flat file. Autofetch is a library which integrates with object persistence tools and automatically handles prefetching data. Using dynamic program profiling, Autofetch determines the right prefetch directives for each query a program executes.
What is Ebean?
Ebean is a Object Relational Mapping Persistence Layer written in Java (Open Source LGPL license).
* Providing the features of EJB3's JPA (and more)
* No container required
* JPA compatible ORM mapping (@Entity, @OneToMany, ...)
Why use Ebean?
Ebean provides a simpler approach to Object Relational Mapping. It does this by not requiring session objects such as JPA EntityManager, JDO PersistenceManager, Hibernate Session, Toplink ClientSession.
dt-Source
This is a taglib providing connection between Displaytag taglib and data sources, allowing to load partial lists from different sources, like Hibernate, JDBC, etc. Requires Spring framework.
During the project growth, it's getting new functionality and features like filters, searches, etc. completely customizable with CSS.
The most of the code was written by Olexiy Prokhorenko with some help from Olexandr Prokhorenko and later was donated to Open Source Community. We support this project free of charge but we welcome donations. :)
Remember when the Object database was going to kill the Relational database?
OOP was the sexy programming model, and relational set theory seemed so quaint. Once you are using Objects, why wouldn’t you just want to persist them instead of having to drop down to this crazy SQL? Inner joins instead of just person.name.first? Fools.
SymmetricDS is web-enabled, database independent, data synchronization/replication software. It uses web and database technologies to replicate tables between relational databases in near real time. The software was designed to scale for a large number of databases, work across low-bandwidth connections, and withstand periods of network outage.
jLynx is a simple, lite, hi-performance layer over the JDBC API. Persist and retrieve POJO and Map objects directly. Designed for developer productivity. Simpler, easy to learn, easy to configure as compared to Hibernate, JPA, etc.
jLynx generates JavaBean objects based on your database schema. For example if you have 24 tables and views in your database, jLynx will create 24 objects with properties that map to the field names in each table. The jLynx Generator creates the objects and compiles them. You can also use the generated source instead if you prefer to customize the objects. Using the API to persist objects is trivial.
jLynx Framework implements the Relational interface, which has all necessary methods to persist objects and fetch entire collections of objects.
jLynx leverages the strengths of SQL and the strengths of Java without having to write/maintain J2EE code. It is truly the best of both worlds.
log4jdbc is a JDBC driver that can log SQL and/or JDBC calls (and optionally SQL timing information) for other jdbc drivers using the Simple Logging Facade For Java (SLF4J) logging system.
SQL Workbench/J is a free, DBMS-independent, cross-platform SQL query tool. It is written in Java and thus it should run on any operating system that provides a Java Runtime Environment.
It's main focus is on running SQL scripts (either interactively or as a batch), graphical query building or administration of the database is not the focus.
Do you hate starting on a new project and having to try to figure out someone else's idea of a database? Or are you in QA and the developers expect you to understand all the relationships in their schema? If so then this tool's for you.
SchemaSpy is a Java-based tool (requires Java 1.4 or higher) that analyzes the metadata of a schema in a database and generates a visual representation of it in a browser-displayable format. It lets you click through the hierarchy of database tables via child and parent table relationships. The browsing through relationships can occur though HTML links and/or though the graphical representation of the relationships. It's also designed to help resolve the obtuse errors that a database sometimes gives related to failures due to constraints.
DB Schema Viewer is an Eclipse plugin that builds a graphical view of an existing database. Reverse Engineering is performed through JDBC. The most popular databases are currently supported:
* MySQL
* PostgreSQL
* Firebird
QuantumDB is a simple but powerful database access plug-in for the Eclipse Development Platform. QuantumDB allows you to:
* connect to databases using standard JDBC drivers
* review schemas, tables, views and sequences
* look up column, index and foreign key information
* issue ad-hoc queries or other SQL statements against the database
* manage, edit, and work with SQL files (*.sql)
* issue updates, deletes, and inserts using simple, easy-to-use wizards
Eclipse SQL Explorer is a thin SQL client that allows you to query and browse any JDBC compliant database. It supports plugins with specialized functionality for individual databases (Oracle, DB2 and MySQL) and can be extended to include specialized support for other databases.
This article presents with an solution on how to page through large amountsofdata effectively without taking down your database and without keeping it all in memory.
Scriptella is an open source ETL (Extract-Transform-Load) and script execution tool written in Java.
Our primary focus is simplicity. You don't have to study yet another complex XML-based language - use SQL (or other scripting language suitable for the data source) to perform required transformations.
SchemaCrawler is a command-line tool to output your database schema and data in a readable form. The output is designed to be diff-ed with previous versions of your database schema. SchemaCrawler is also an API that improves on the standard JDBC metadata.
Application Ermodeller is a free CASE tool for data modelling which is built in Java. This tool supports conceptual and logical data modelling and in the final step, it allows to create physical model that is specific for particular DBMS.
The basic idea is to support development of database via ER modelling in three levels of abstraction:
* conceptual
* logical
* physical
This separation allows to create database model independent on the particular database system. You may create conceptual model and logical model that are independent on any DBMS. Then, you may generate physical model for each DBMS you will use. Afterwards, physical models allows you to generate SQL data definition for your target DBMS.
Develop a java-based tool that manages the life-cycle of the database schema and generates code and documentation artifacts. The tool transforms an XML description of a database schema into an HTML data dictionary, SQL scripts to create the database, and Java data access code. The tool automatically updates the documentation, scripts, and code as the schema is changed and updated. The tool also provides a simple but powerful data access programming model and framework for developing custom data access code. The generated Java code conforms to the Data Access Object (DAO) and Value Object (AKA Data Transfer Object) patterns that are described in the J2EE Java Blueprints.
Extrema is a powerful visualization and data analysis tool that enables researchers to quickly distill their large, complex data sets into meaningful information. Its flexibility, sophistication, and power allow you to easily develop your own commands and create highly customized graphs.
The Sundog Database Refactoring Tool (SDRT) is an open source (LGPL), DBMS-independent library that eases database development, particularly in an agile environment.
MIGRATEdb will parse an XML file of database changes and load them to the targeted database if they are not already there.
A database change consists of tests to determine whether the change has already been applied and a set of change actions.
dbdeploy is a Database Change Management tool. It helps developers and
DBAs change their database in a simple, controlled, flexible and
frequent manner
JPOX is a free and fully compliant implementation of the JDO1, JDO2 specifications, providing transparent persistence of Java objects. It supports persistence to all of the major RDBMS on the market today, persistence to the DB4O object datastore, supporting all of the main Object-Relational Mapping (ORM) patterns demanded by today's applications, allows querying using either JDOQL, SQL, or JPQL, and comes with its own byte-code enhancer. JPOX is available under the Open Source Apache 2 license, allowing access to not just a top quality Java persistence implementation but also to the source code, allowing you to contribute to the success story of the principal standards-compliant Open Source persistence implementation in the world today.
SuperQuaiL is a Java SQL query builder. The goal of SuperQuaiL is to help Java developers who code SQL run and create queries easier. Other database tools focus on making it easy to do database administration and it is nice to be able to inspect the database through these tools. PhpMyAdmin, for example, a is nice web front end that you can run from multiple computers, but the interface is clumsy for the developer. For development, building queries is the main task and having a small program that loads quickly is more helpful than having a complicated interface where you can find out almost anything about the database. SuperQuaiL provides nice features to aid in the process of developing queries quick and easy.
Jackcess is a pure Java library for reading from and writing to MS Access databases. It is not an application. There is no GUI. It's a library, intended for other developers to use to build Java applications. Jackcess is licensed under the GNU Lesser General Public License. Take a look at our Frequently Asked Questions for more info.