All Projects

1. GraphGrad

   Custom C++ tensor library with pybinding support that allows many standard tensor operations.

   • C++
   • Python
   • CUDA
   • OpenMP
   • Pybind

   GraphGrad

   Github Repo

   Description

   A custom-built C++ tensor library to perform matrix and tensor operations efficiently with automatic differentiation support for machine learning algorithms (a simplified version of PyTorch/TensorFlow). The library allows for multiple operations such as add, subtract, matmul, etc. and supports Python usage through pybindings. Multiple optimization techniques were implemented such as Lazy Evaluation, OpenMP CPU optimizations, SIMD computation, Common Subexpression Elimination, and GPU kernels. For more information and documentation for the library, check out the GitHub repository linked above.

   Why?

   This was a semester group project for a course on Machine Learning System Implementations and our group wanted to better understand what it took to implement the machine learning libraries we've so often used.

   My Contributions

   We collaborated to come up with the overall architecture design and I implemented many of the standard operations, some GPU kernels, OpenMP optimizations, a couple of the backpropogation functions, the majority of the pytests, and initial benchmark comparisons to evaluate our library's efficiency.

2. Text Recovery from Palimpsests

   Spatial revising variational autoencoder for the purposes of recovering text of from Palimpsests, or old manuscripts which were chemically altered to be reused, using multi-spectral images.

   • Python
   • PyTorch
   • Computer Vision

   Text Recovery from Palimpsests

   Github Repo

   Description

   A PyTorch implementation of the Spatial Revising Variational Autoencoder from Yu et al. for extracting useful features from hyperspectral image data. After implementation, this model was then used to extract features from multi-spectral images of palimpsests in order to perform segmentation on images and recover hidden or damaged text.

   Why?

   This was a class project for a Neural Networks and Machine Learning course done in a group. For our project, we chose our topic to be text recovery from palimpsests as one of the group members was an image science major working on a Capstone project in the same topic and had access to multi-spectral image data for training and inference. While searching for potential model architectures that would work well, we came across a Spatial Revising Variational Autoencoder used for segmentation of hyperspectral images. Since this was exactly what we wanted to do, we began our project as an application study to apply this model to our topic.

   My Contributions

   I implemented the CNN module, parts of the LSTM module, modified the loss function, built most of the encoder and decoder, and trained and evaluated the model on our data.

3. Speaker Diarization in Expressive Dialogues

   Master's Capstone Project

   Joint, end-to-end, speaker diarization and automatic speech recognition system trained and evaluated on expressive dialogues and compared to traditional speaker diarization methods.

   • Python
   • PyTorch
   • Speech Processing
   • Research

   Speaker Diarization in Expressive Dialogues

   Description

   An end-to-end, joint Speaker Diarization and Automatic Speech Recognition model implementation for recognizing the words spoken in audio recordings as well as who spoke them. This was done as my Master's capstone project with the main goal to evaluate how a newer end-to-end system performed in comparison to traditional methods on more difficult dialogue containing expressive, or emotional, speech.

   Why?

   While attending Rochester Institute of Technology, I was exposed to a lot of deaf or hard of hearing students as it was also the home of the National Technical Institute of the Deaf so topics like automatic speech recognition came up often. From one of my peers, I also learned more about speaker diarization and how it could be used to automate transcriptions and provide more rich ASR output. Combining these two pieces with my interest for natural language processing and speech processing, I took this as an opportunity to learn more about how to create these types of systems.

   My Contributions

   I performed all of the work on this project (model implementation, training, and evaluation, technical report, and poster) with the help of my Capstone advisor, Cecilia Alm.

4. Web Checkers

   A web application for users to play checkers against one another or against a simple AI.

   • Java
   • Agile
   • Spark
   • JavaScript
   • Maven

   Web Checkers

   Description

   A web application users can sign onto and play checkers against one another or a simple artificial intelligence bot. Able to handle multiple simultaneous games on the server and allowed users to select who they wished to play against.

   Why?

   This was a semester long group project for a Software Engineering course learning about the Agile development process and developing a full "product" (the checkers app) for our "client" (the professor). It involved daily stand-ups, weekly sprints, demos and presentations, etc.

   My Contributions

   I worked on a lot of the backend logic for the checkers application, helped to work out a simple AI bot, and helped design the API and communication between the server and clients.

5. Traffic Camera Reprojection to Supplement Autonomous Perception

   A lightweight system for cooperative perception between LiDARs on autonomous vehicles and monocular traffic cameras.

   • Python
   • Computer Vision
   • CARLA
   • Research

   Traffic Camera Reprojection to Supplement Autonomous Perception

   Description

   A cooperative perception system attempting to combine data from monocular traffic cameras with LiDARs on autonomous vehicles in a Bird's Eye View format in order to improve autonomous vehicle perception and visibility. This uses the pinhole camera method to first estimate the objects positions for the camera view and then further refined by using a point alignment algorithm such as ICP to match the camera estimations with the vehicle's detections.

   Why?

   Traffic intersections are one of the most accident-prone areas on the road and contain many potential occlusions, or blind spots, for both human drivers and autonomous vehicles alike. One way to improve autonomous vehicle visibility is through the use of cooperative perception and combining what the vehicle sees with what another third party sensor can see. Since traffic cameras are already commonly found at many intersections, it would be widely beneficial to use these for cooperative perception.

   My Contributions

   This project started as a group class project with two others for an Autonomous Vehicles course and I expanded upon it for an independent study. I wrote the CARLA simulation code for collecting data and evaluating the system, implemented point alignment, and modified the pinhole camera code.

6. Speech Emotion Recognizer

   An emotion recognition model trained to identify 7 different emotions.

   • Python
   • TensorFlow
   • Speech Processing
   • Librosa
   • PyGame

   Speech Emotion Recognizer

   Github Repo

   Description

   An emotion recognition model for recognizing 7 different emotions - anger, disgust, fear, sadness, happiness, neutrality, and surprise - with a PyGame UI for uploading or recording voice clips for the model to evaluate. Model was created using TensorFlow/Keras and trained using the RAVDESS and TESS datasets. Data was preprocessed with the numpy, pandas, and librosa Python libraries.

   Why?

   How computers process and understand language has always been an interest of mine, and I am more specifically interested in the intersection between language and emotion as it is a particularly difficult area to interpret, even for humans. Therefore, I used this project as an opportunity to learn more about emotion and some of the features that can be used to detect it for computers.

   My Contributions

   This was an individual project where I implemented, trained, and created the interface to interact with the model.

7. Twitter Depression Detection

   A model to predict a person's likelihood for depression based on the content and activity of their Tweets on Twitter (now X).

   • Python
   • Sci-kit Learn
   • Feature Engineering
   • Natural Language Processing

   Twitter Depression Detection

   Github Repo

   Description

   A depression detection system for Twitter users based on Tweet history and activity. The focus of this project was on feature engineering and analysis or which features were more predictive for the task through the use of the pandas, scikit-learn, and nltk Python libraries.

   Why?

   This was a group class project for a natural language processing class, and we wanted to explore an application involving "messier" text such as that in tweets with slang, improper grammar, and emojis. Since I have an interest in emotion in language, we decided our application would be depression detection and researching features that would be suitable for this task.

   My Contributions

   I contributed the feature extraction, model creation, model evaluation, some report writing/reviewing, and some presentation writing for this project.

8. Voice Cloning Resume Website

   A resume website demo with a ChatGPT-powered chatbot that answers questions about the provided resume and plays responses using a cloned voice.

   • Python
   • Speech Processing
   • ChatGPT

   Voice Cloning Resume Website

   Project Information

   Description

   A website demo for using ChatGPT to create a question-answer experience for asking questions about one's resume and receiving answers in a text-to-speech system using voice cloning to replicate the voice of the resume owner. Voice cloning was performed using the Tortoise-TTS model set up on HuggingFace Space.

   Why?

   As a group project for a speech processing course, we wanted to explore the topic of voice cloning and as we were nearing graduation, a practical application for a resume website was a fun use case to try it on. This allowed us to learn about the topic while also creating something a system to potentially use for ourselves in the future.

   My Contributions

   I managed the model interactions and designed the evaluation methodology for determining how well our voices were cloned. I additionally helped with report and presentation writing.