Advanced Training: Developing Workflows 2022

Click here for the training survey

Course overview:

High-throughput laboratory machines generate large quantities of raw biological data. Processing the data to generate useful information involves using several tools and sometimes setting up complex bioinformatics analysis environments across heterogeneous computing devices. Bioinformatics workflows and workflow managers allow reproducibility and portability of analysis plans and optimization of available computing resources. The Advanced Training: Developing Workflows workshop was aimed at introducing, deploying Nextflow, a too which enables scalable and reproducible scientific workflows.

Intended audience:

Beginner - Intermediate level Bioinformaticians working with next-generation sequencing data in East-Africa.

Pre-requisite:

For the course, participants do not need a high-performance computer since the trainers will provide access to KEMRI's cluster

The course will cover:

Logistics:

Venue of course: KEMRI / WELLCOME TRUST Kilifi, Kenya
Start date:2022-04-25
End date:2022-04-29

Course curriculum:

Day 1 HPC and introduction to containers
Day 2 Introduction to git and file organisation
Day 3 Introduction to snakemake
Day 4 Advanced workflows (Nextflow, CWL)
Day 5 Practical application

Course schedule and Materials:

Day 1 Training Videos

Session	Course materials	Training partner	Interactive Sessions with Instructor/s
9:00am - 10:00am	Introduction to the Course 1. Draw Scientific Journey (Includes your current needs for intermediate/advanced training) 1.1 Get setup working on HPC	Malawi-Liverpool-Wellcome Trust KEMRI	Dr Philip Ashton Dr George Githinji
10:30am - 1:00pm	2. Introduction to Conda and Conda environments 2.1 Miniconda Installation 2.2 Use of mamba 2.3 Conda environments 2.4 Different ways of specifying (cmd line, yaml file) 2.5 Ways of saving/exporting/sharing 2.5.1 Practical 1: Conda installation, mamba installation and activation of virtual environments	Malawi-Liverpool-Wellcome Trust KEMRI	Dr Philip Ashton Dr George Githinji
2:00pm - 4:00pm	3. Introduction to Containers. 3.1 Introduction to docker 3.2 Finding and rolling out containers 3.3 Run docker 3.4 Build a docker container 3.5 Modifying a docker container 3.6 Theory session on Singularity, similarities and differences with docker 3.6.1 Practical 2: Docker.	Malawi-Liverpool-Wellcome Trust KEMRI	Dr Philip Ashton Dr George Githinji

Day 2 Training Videos

Session	Course materials	Training partner	Interactive Sessions with Instructor/s
9:00am - 10:00am	1. Recap from yesterday 1.1 Introduction to docker mounts and volumes 1.2 Working with volumes and persisting data 1.3 Introduction to singularity	Malawi-Liverpool-Wellcome Trust KEMRI	Dr Philip Ashton Dr George Githinji
10:30am - 1:00pm	2. Introduction to Git 2.1 Software carpentry (Basic Unix and Git)	Malawi-Liverpool-Wellcome Trust KEMRI	Dr Philip Ashton Dr George Githinji
2:00pm - 3:00pm	3. File organisation. 3.1 Guide to organisation of Computational Biology projects 3.2 Mordern file organisation 3.2.1 Practical 1 3.2.2 Practical 2 3.2.3 Practical 3	KEMRI	Dr George Githinji

Day 3 Training Videos

Session	Course materials	Training partner	Interactive Sessions with Instructor/s
9:00am - 1:00pm	1. Introduction to Snakemake 1.1 Scripting 1.2 Basic concerpts 1.3 Running a basic pipeline 1.4 Configurations 1.5 Reproducible, scalable, and shareable analysis pipelines with bioinformatics workflow managers	Malawi-Liverpool-Wellcome Trust KEMRI	Dr Philip Ashton, Belson Kutambe Dr George Githinji
2:00pm - 4:00pm	2. Running snakemake. 2.1 Run snakemake with conda 2.2 Running snakemake on HPC 2.3 Practical: Snakemake	Malawi-Liverpool-Wellcome Trust KEMRI	Dr Philip Ashton, Belson Dr George Githinji

Day 4 Training Videos

Session	Course materials	Training partner	Interactive Sessions with Instructor/s
9:00am - 10:00am	1. Running Snakemate cont...	ILRI KEMRI Malawi-Liverpool-Wellcome Trust	John Juma Dr George Githinji Belson Kutambe
10:30am - 1:00pm	2. Sharing pipelines	Malawi-Liverpool-Wellcome Trust KEMRI ILRI	Dr Philip Ashton, Belson Kutambe John Juma
2:00pm - 4:00pm	3. Introduction to Bioinformatics workflows with Nextflow and nf-core 3.1 Advanced workflows (Nextflow and Common Workflow Language) (Nextflow Exercises) 3.2 Compare and contrast with Snakemake 3.3 Simple worklfows 3.4 Using docker in worklfows 3.5 Using HPC 3.6 Introduction to Common Workflow language (CWL)	ILRI KEMRI	John Juma Dr George Githinji

Session	Course materials	Training partner	Interactive Sessions with Instructor/s
9:00am - 1:00pm	Practical project on implementing your own workflow	KEMRI Malawi-Liverpool-Wellcome Trust	Dr George Githinji Dr Philip Ashton