The Experience API (xAPI) allows us to collect data about any type of learning experience or activity, but does that mean we should? Should we generate massive amounts of xAPI data for every possible type of interaction and then expect to make sense of it all later? This approach can be costly in terms of data storage, but also in terms of your time.
Sequence alignment data is often ordered by coordinate (id of the reference sequence plus position on the sequence where the fragment was mapped) when stored in BAM files, as this simplifies the extraction of variants between the mapped data and the reference or of variants within the mapped data. In this order paired reads are usually separated in the file, which complicates some other applications like duplicate marking or conversion to the FastQ format which require to access the full information of the pairs. In this paper we introduce biobambam, a set of tools based on the efficient collation of alignments in BAM files by read name. The employed collation algorithm avoids time and space consuming sorting of alignments by read name where this is possible without using more than a specified amount of main memory. Using this algorithm tasks like duplicate marking in BAM files and conversion of BAM files to the FastQ format can be performed very efficiently with limited resources. We also make the collation algorithm available in the form of an API for other projects. This API is part of the libmaus package. In comparison with previous approaches to problems involving the collation of alignments by read name like the BAM to FastQ or duplication marking utilities our approach can often perform an equivalent task more efficiently in terms of the required main memory and run-time. Our BAM to FastQ conversion is faster than all widely known alternatives including Picard and bamUtil. Our duplicate marking is about as fast as the closest competitor bamUtil for small data sets and faster than all known alternatives on large and complex data sets.
Recently we have been doing some experiments to cluster semantically similar messages, by leveraging pre-trained models so we can get something off the ground using no labelled data. Task here is…
Presentation from Marko Teräs at the 28th ICDE World Conference on Online Learning of a national-level learning analytics research and development project funded by the Finnish Ministry of Education and Culture. Student and teacher needs analysis results for LA pilot development and for policy recommendations.