Progress Reports

Small drone flying over a cornfield

Project Title: sUAS Safety Case Tool

Team Members:

  • Brady Bargren: Project Manager / Backend Developer
  • Benjamin Kelly: Frontend Developer
  • Gabriel Perez: Backend Developer
  • Gautham Suresh: Backend Developer
  • Blake Bryan: Frontend/Backend Developer

Stage I Report

Introduction

The increasing use of small, Uncrewed Aerial Systems (sUAS) for various purposes, such as recreation, emergency services, and commercial activities, has highlighted the need for standardized safety procedures. Unlike traditional aviation, there is a lack of formalized safety case generation for sUAS pilots, leading to potential risks due to hardware failures, software bugs, human errors, adverse weather conditions, and radio interference. Our project aims to address this gap by developing a web-based sUAS Safety Case Tool that automates the generation of safety cases based on specific input parameters related to the pilot, vehicle, and environmental conditions.

Problem Statement

  • Safety Case Complexity: Creating safety cases is a complex and time-consuming process, especially for drones with diverse configurations.
  • Lack of Standardized Pre-Flight Checks: sUAS pilots currently lack a formal process for performing pre-flight safety checks.
  • Diverse Configurations: The numerous permutations of drone types, capabilities, and environmental conditions make it challenging to generate accurate safety cases manually.

Project Overview

Our solution is a web-based application that automates safety case generation through a customizable front end and a robust backend that processes flight and pilot data.

Key Components:

  1. Dynamic Questionnaire Generation:
    • Parses an XML-based model (model.xml) representing various flight and pilot configurations.
    • Generates dynamic questions for the user based on the parsed XML structure.
  2. User-Friendly Interface:
    • Built with React and TypeScript for responsiveness and portability.
    • Allows users to input data in a format similar to the FAA's Flight Risk Assessment Tool (FRAT).
  3. Data Storage and Retrieval:
    • Utilizes MongoDB to store user responses for future retrieval and analysis.
    • Ensures compliance with potential regulatory requirements by maintaining records of safety assessments.
  4. Safety Case Generation:
    • Converts user inputs and the XML model into a safety case represented in YAML format.
    • Processes the YAML file to generate graphical safety case diagrams using Goal Structuring Notation (GSN).
  5. Risk Assessment Integration:
    • Incorporates a risk assessment algorithm that calculates the overall risk level based on user inputs.
    • Visually represents risk levels in the safety case diagrams, highlighting areas that require attention.

Current Status (Stage I)

Backend:

  • MongoDB Integration: Successfully connected to store and retrieve user responses.
  • Dynamic Question Generation:
    • Parsed model.xml to generate dynamic questions based on drone configurations.
    • Handled complex XML structures, including AND, OR, and ALT nodes, through recursive processing.
  • Safety Case Generation:
    • Converted user inputs and model.xml data into a structured YAML file.
    • Generated GSN diagrams from the YAML file.

Frontend:

  • Form Display and Management:
    • Rendered questions and potential answers dynamically.
    • Implemented form management with state handling and validation.
  • Cookies Implementation:
    • Used cookies to save the state of the form for user convenience.
    • Aided in pre-filling previous answers and handling page refreshes.
  • Results Display:
    • Began implementing the display of survey results and the generated safety case to the user.

Challenges Faced

Backend:

  • XML Parsing:
    • Handling the complex structure of model.xml and converting it into a dynamic form.
  • Data Flow:
    • Ensuring proper communication between the frontend and backend.
    • Managing user inputs and converting them into structured formats.
  • YAML File Generation:
    • Dynamically generating YAML files to accurately represent various drone configurations.

Frontend:

  • Data Management:
    • Managing state and ensuring components re-render appropriately.
  • Component Integration:
    • Getting various frontend components to work together seamlessly.
  • Version Control:
    • Coordinating development efforts and managing merges in GitLab.

Tools and Technologies Used

Backend:

  • Node.js & Express: Backend framework for handling API requests and server operations.
  • MongoDB: Database for storing and retrieving user responses.
  • xml2js: Library for parsing XML files into JavaScript objects.
  • js-yaml: Library for converting data into YAML format.
  • Axios: Used for handling HTTP requests between the frontend and backend.

Frontend:

  • React.js with TypeScript: For building a responsive and dynamic user interface.
  • React-Router: To manage navigation between different pages.
  • Bootstrap: For consistent styling across the application.
  • Cookies: To persist user data and enhance user experience.

Team Roles and Responsibilities

  • Brady Bargren:
    • Project Manager and Backend Developer.
    • Coordinated overall project efforts.
    • Developed the YAML generator and integrated the backend with MongoDB.
  • Benjamin Kelly:
    • Frontend Developer.
    • Created the dynamic user interface using React and TypeScript.
    • Managed state handling, form validation, and cookie implementation.
  • Gabriel Perez:
    • Backend Developer.
    • Worked on parsing model.xml and generating dynamic questions.
    • Ensured proper data flow between frontend and backend.
  • Gautham Suresh:
    • Backend Developer.
    • Managed the MongoDB database interactions.
    • Restructured the backend for better modularity and maintainability.
  • Blake Bryan:
    • Frontend/Backend Developer.
    • Developed system architecture diagrams.
    • Assisted with frontend development and integration tasks.

Communication and Collaboration

  • Regular Meetings:
    • Held weekly meetings to discuss progress and plan tasks.
  • Discord Communication:
    • Used Discord for daily communication and issue resolution.
    • Shared updates, challenges, and solutions promptly.

Remaining Work and Future Plans

Backend:

  • Enhance YAML Generation:
    • Fine-tune the YAML structure for more complex safety case representations.
  • Error Handling:
    • Improve error logging and handling in form submission and YAML generation processes.
  • Performance Optimization:
    • Ensure efficient data handling with large XML files and multiple users.
  • Integration Testing:
    • Finalize end-to-end tests for seamless interaction between all components.

Frontend:

  • Results Display Enhancement:
    • Improve the rendering of processed form results and safety case diagrams.
  • File Download Feature:
    • Allow users to download the generated safety case files.
  • Form Validation:
    • Implement checks and preprocessing on the form before submission.

Conclusion

By the end of Stage I, the team has successfully developed a functional prototype of the sUAS Safety Case Tool. The key components are integrated, and the system can generate safety cases based on user input. The focus moving forward will be on refining the system, enhancing user experience, and ensuring robustness and compliance with potential regulatory requirements.

Appendix

Demonstration Video

A demonstration video showcasing the current functionality is available at Demo 1 Video

Stage II Report

Introduction

The increasing use of small Uncrewed Aircraft Systems (sUAS) for various purposes—such as recreation, emergency services, and commercial activities—has led to a rise in incidents within shared airspace. These incidents are often due to hardware failures, software bugs, human errors, adverse weather conditions, and radio interference. Unlike traditional aviation, sUAS pilots lack formalized safety case generation, resulting in potential risks. Our project aims to address this gap by developing a web-based sUAS Safety Case Tool that automates the generation of safety assessments and risk diagrams based on specific input parameters related to the pilot, vehicle, and environmental conditions.

Problem Statement

  • Safety Case Complexity: Manually creating safety cases is complex and time-consuming, especially given the diverse configurations of sUAS.
  • Lack of Standardized Pre-Flight Checks: sUAS pilots currently lack a formal process for performing pre-flight safety checks.
  • Diverse Configurations: The numerous permutations of drone types, capabilities, and environmental conditions make accurate manual safety case generation challenging.

Project Overview

Our solution is a web-based application that automates safety case generation through a customizable front end and a robust backend that processes flight and pilot data.

Key Components:

  • Dynamic Risk & Safety Diagrams: Generates diagrams based on user input, color-coded by risk level (green: low, yellow: moderate, red: high).
  • Colorization Module: Developed an SVG colorizer for clear visualization of calculated risks.
  • Timestamp Handling: Ensures all files display properly in the frontend by handling timestamped files.
  • Installer Batch File: Streamlines dependency installation, easing the project setup.
  • Argument Parser Pipeline: Automates the parsing, structuring, and visualization of arguments in the Goal Structuring Notation (GSN) framework.
  • Mapping Feature Model Form and Safety Case: Matches user responses to safety case nodes, adjusting node statuses and propagating failures as needed.

Current Status (Stage II)

Backend:

  • Dynamic Risk & Safety Diagrams: Implemented generation of safety diagrams based on user input, with risk levels color-coded for immediate visual feedback.
  • Colorization Module: Developed an SVG colorizer that identifies node classes and assigns specific colors based on risk levels.
  • Timestamp Handling: Updated backend to manage timestamped files, ensuring chronological ordering and accurate display of submissions.
  • Installer Batch File: Created to simplify project setup by automating dependency installations.
  • Argument Parser Pipeline:
    • Tokenizer: Processes the argument file, extracting safety conditions.
    • Parser: Converts the argument file into a structured JSON format.
    • GSN YAML Generation: Transforms JSON data into GSN YAML files.
    • GSN Visualization: Produces graphical safety case diagrams.
  • Mapping Feature Model Form and Safety Case:
    • Process Overview: Automates the mapping of user responses to safety case nodes.
    • Extracting Leaf Nodes: Identifies nodes without children to map responses.
    • Mapping Responses: Uses similarity matching to associate user answers with safety case nodes.
    • Adjusting Node Status: Updates node statuses based on user responses.
    • Propagating Failures: Propagates failed statuses up the tree to affect parent nodes.

Frontend:

  • Robust Form Display: Enhanced dynamic question rendering and user input handling.
  • Image Generation Display: Implemented functionality to display generated safety diagrams within the application.
  • Historical Data Access: Enabled viewing of previous and most recent submissions, aiding in tracking and analysis.

Challenges Faced

Backend:

  • Managing Timestamps: Ensured proper chronological ordering by storing files locally to prevent overwriting and maintain data integrity.
  • Safety Case Diagram Generation: Overcame complexities in dynamically generating diagrams from argument files.
  • Color Display Issues: Resolved inaccuracies in risk-level color displays by refining the SVG colorization process.
  • Frontend Integration: Adjusted APIs to ensure that timestamped diagrams display correctly in the frontend.

Frontend:

  • NOTAMs API Integration: Encountered difficulties due to API access limitations and CAPTCHA requirements, delaying the integration of real-time airspace data.
  • Backend Consistency: Ensured that frontend components remained consistent with backend changes, requiring careful coordination.
  • Image Generation Display: Addressed challenges in asynchronously loading and displaying generated images within the user interface.

Tools and Technologies Used

Backend:

  • Node.js & Express: Framework for handling API requests and server operations.
  • MongoDB: Database for storing and retrieving user responses.
  • xml2js: Library for parsing XML files into JavaScript objects.
  • js-yaml: Library for converting data into YAML format.
  • Axios: For handling HTTP requests between frontend and backend.
  • Goal Structuring Notation (GSN): Used for safety case diagram representation.
  • SVG Manipulation: For colorizing and customizing SVG images to reflect risk levels.

Frontend:

  • React.js with TypeScript: For building a responsive and dynamic user interface.
  • React-Router: Manages navigation between different pages.
  • Bootstrap: Provides consistent styling across the application.
  • Cookies: Used to persist user data and enhance user experience.

Team Roles and Responsibilities

  • Brady R Bargren:
    • Project Manager and Backend Developer.
    • Developed the risk assessment and safety case diagrams.
    • Implemented the colorization module for SVG images.
    • Handled timestamp management and created the installer batch file.
  • Benjamin C Kelly:
    • Frontend Developer.
    • Enhanced form display and management.
    • Integrated image generation display within the frontend.
    • Implemented viewing of historical and most recent responses.
  • Gabriel A Perez:
    • Backend Developer.
    • Worked on parsing the argument file and mapping feature models.
    • Assisted in developing the argument parser pipeline.
  • Gautham Suresh:
    • Backend Developer.
    • Managed MongoDB database interactions.
    • Adjusted the backend API for frontend integration.
  • Blake M Bryan:
    • Frontend/Backend Developer.
    • Assisted with fixing image generation display issues.
    • Worked on handling previous responses and timestamped data.

Communication and Collaboration

  • Regular Meetings:
    • Held weekly meetings to discuss progress, plan tasks, and address challenges.
  • Discord Communication:
    • Utilized Discord for daily communication, sharing updates, and resolving issues promptly.
  • Version Control:
    • Coordinated development efforts through GitLab, managing merges and ensuring codebase integrity.

Remaining Work and Future Plans

Backend:

  • NOTAM Integration:
    • Plan to integrate Notice to Air Missions (NOTAM) data into risk calculations and displays, providing pilots with real-time airspace updates.
  • Safety Case Simplification:
    • Adjust diagrams to focus on problematic nodes, allowing users to quickly identify and address specific risks.
  • Increase Result Clarity:
    • Improve readability of safety case results by removing irrelevant information and highlighting critical risk factors.
  • Highlight Risk Justifications:
    • Provide concise explanations for assigned risk levels to enhance pilot understanding and decision-making.
  • Codebase Flexibility:
    • Refactor code to accommodate varied inputs, enabling greater adaptability for different flight scenarios.
  • Expanded Risk Algorithm:
    • Enhance the flight risk algorithm to integrate additional factors, making it more comprehensive and adaptable.

Frontend:

  • Update Displays for Results:
    • Enhance the display of survey results and generated safety cases for better user experience.
  • Input Validation:
    • Implement better form input validation to ensure data integrity and prevent errors.
  • Downloadable Documents:
    • Allow users to download generated safety case files for record-keeping and compliance purposes.
  • NOTAMs Integration:
    • Continue efforts to integrate NOTAMs into the frontend, overcoming API access challenges.

Conclusion

By the end of Stage II, the team has made significant progress in developing the sUAS Safety Case Tool. The application now dynamically generates color-coded safety diagrams based on user input, providing immediate visual feedback on risk levels. The backend and frontend components are more robust and integrated, with improved user interface elements and data handling.

Moving forward, the focus will be on refining the user experience, integrating real-time data such as NOTAMs, and enhancing the risk assessment algorithms to provide more comprehensive safety evaluations. The team remains committed to addressing the challenges and achieving the project's objectives.

Appendix

Demonstration Video

A demonstration video showcasing the current functionality is available at Demo 2 Video

Stage III Report

Introduction

The increasing use of small Uncrewed Aircraft Systems (sUAS) for various applications continues to pose risks in shared airspace. This project aims to address the lack of formalized safety case generation for sUAS by providing a web-based tool that automates the process of safety assessment and risk evaluation. Through user inputs and dynamic safety case formats, the tool assists pilots in understanding and mitigating potential risks.

This report outlines the progress achieved during Stage III, focusing on the final additions and features implemented for the third demo.

Problem Statement

  1. Safety Case Complexity: Generating safety cases for diverse sUAS configurations remains a challenging, manual process.
  2. Limited Pre-Flight Tools: Current tools lack standardization and cannot dynamically adapt to different scenarios.
  3. Data Accessibility: Real-time and contextual data, such as Notices to Air Missions (NOTAMs), are often inaccessible in safety tools.

Stage III Final Additions

1. New Safety Case Generator

  • A new JSON-based safety case generator was added, enhancing the tool's flexibility by introducing a third format alongside existing YAML and .argument outputs.
  • Impact: Users now have three distinct file formats to choose from, catering to various downstream applications and improving compatibility with client workflows.

2. Pruning Functionality

  • Developed two pruners for safety cases, one for YAML-based safety cases and another for JSON-based safety cases.
  • These tools allow users to visualize the remaining nodes in a safety case after removing nodes associated with failures.
  • Impact: Improves clarity and focuses user attention on actionable areas within the safety case.

3. NOTAMs Integration

  • Integrated NOTAM (Notice to Air Missions) functionality lets users view real-time information about airspace conditions and restrictions.
  • Impact: Enhances situational awareness by providing essential pre-flight data directly within the tool.

4. Expanded Admin Page

  • Added an admin page with features allowing administrators to adjust the system.
  • Impact: Supports ongoing maintenance, customization, and scalability of the application by client teams.

5. Enhanced Model and Questionnaire

  • Significantly expanded the model.xml used for safety case generation, incorporating a more comprehensive set of questions.
  • Impact: Provides a more robust safety case generation process with detailed user inputs, ensuring higher fidelity in risk analysis.

6. Logic Expansion for Risk Calculations

  • Began addressing challenges posed by non-numerical determinations in the expanded model, with plans to refine the logic for handling these scenarios.
  • Impact: Improves the accuracy of the safety case risk level calculator, although some calculations require further development.

7. Testing and Validation

  • Added Jest tests and other test files to validate the system, ensuring reliability and robustness across the expanded features.
  • Impact: Streamlines future updates and provides a foundation for testing by the Spring 2025 team.

8. Project Continuation Approval

  • NASA has approved the continuation of the project for Spring 2025, ensuring the work will be carried forward by a new team.

Key Files Updated

  • JSON to YAML Conversion Script (json_xml_to_yaml.js): Handles conversions, pruning logic, and risk level propagation for JSON safety cases.
  • API Routes (apiRoutes.js): Expanded to support additional endpoints for generating and retrieving new safety cases, pruning, and NOTAM integration.
  • Admin Thresholds and Safety Cases (treeDataController.js): Added functions for saving, retrieving, and pruning safety case data, ensuring seamless functionality for administrators.
  • NOTAMs Integration (notamsContainer.tsx): Displays real-time NOTAMs data with a user-friendly interface and supports navigation back to the home page.
  • Survey Results (SurveyResults.tsx): Improved interface for viewing generated safety cases, pruned trees, and risk diagrams.

Challenges Faced

  1. Logic Expansion:
    • Non-numerical determinations in the expanded model.xml introduced complexities in risk calculations. Full support for these determinations remains a work in progress.
  2. Pruning Logic:
    • Developing and validating pruning mechanisms across both JSON and YAML formats required iterative testing and adjustments.
  3. NOTAM API Constraints:
    • Limited access and API challenges required creative solutions to ensure reliable data retrieval and display.

Tools and Technologies

  • Backend: Node.js, Express, MongoDB, js-yaml, xml2js, Jest.
  • Frontend: React, React Router, TypeScript, Bootstrap.
  • Safety Case Formats: JSON, YAML, .argument.
  • Visualization: Goal Structuring Notation (GSN), custom SVG generation.

Future Work and Recommendations

  1. Complete Logic Expansion:
    • Finalize the risk level calculator to fully support expanded safety case models.
  2. NOTAM Integration Enhancements:
    • Integrate NOTAMs directly into risk calculations for a more comprehensive pre-flight assessment.
  3. User Interface Improvements:
    • Refine the display of pruned safety cases and risk diagrams for better clarity and usability.
  4. Codebase Refinements:
    • Ensure modularity and documentation to facilitate the project's handoff to the Spring 2025 team.

Conclusion

Stage III marks significant progress in transforming the sUAS Safety Case Tool into a robust, user-centric solution. Adding new safety case formats, pruners, NOTAM integration, and admin functionalities highlights the team's commitment to addressing the needs of sUAS pilots and stakeholders. With the project's green light for continuation, it is positioned to improve its impact in the next phase.

Appendix

  • Demonstration Video: [Link to Demo 3 Video]