Big Data Using the MetaOmGraph

MetaOmGraph: Metabolic Network Exchange

Introduction

MetaOmGraph is a tool for plotting and analyzing large sets of data while using as little memory as possible. It was designed with biological experiment data in mind. Some uses for MetaOmGraph include:

Visualizing gene expression patterns
Finding functional groups of genes
Determining which genes have expression patterns most/least correlated to a gene of interest.

MOG

Urminder Singh, Manhoi Hur, Karin Dorman, Eve Syrkin Wurtele, MetaOmGraph: a workbench for interactive exploratory data analysis of large expression datasets, Nucleic Acids Research.

Project Overview

The project aims to develop novel features for interactive analysis of big data using the MetaOmGraph software.

Main goals:

Java Upgrade
Limma Analysis
Filter Improvements
Logging and Playback
Quality of Life Improvements
Performance Improvements
Increasing Test Coverage

Technologies Used

Development:

Java 11
Maven Framework
Stan4j; Log4j
Javascript
Swing
Renjin (R in JVM)

Project Management:

Notion
When2Meet
GitLab Issues
Discord
Webex
Google Sheets/Docs/Drive

Development Practices

Agile workflow
Assigning time estimates to tasks
Tracking known issues from discovery to resolution
Pairing and Mobbing Sessions
Cross-Platform Testing

Development Challenges

Problems:

Difficult to reach client for the first few weeks
Confusing project goals
1. Web or desktop?
2. Unclear specifications

Solutions:

Scheduled weekly meetings early in the semester
Meet frequently to discuss thoughts with client, professor, and team

Other Slowdowns:

Large existing codebase
Technical debt
Existing technology

How we handled them:

Stan4j/Code Coverage Tools
Learning how to use their technology

Bugs:

Started appearing as we used MOG more
Found in our code, and theirs
Troubleshooting was sometimes difficult