9:00 AM Engineering Breakout VI: Panel B
Wednesday, August 2 9:00AM – 10:00AM
Location: Pathways
Victor Diaz
California Polytechnic State University, Humboldt
Enhancing Deception Detection: A Multimodal Approach Using Supervised Machine Learning with Visual Features
Deception detection is a complex challenge. Research has demonstrated that the accuracy of the latest computerized polygraph testing techniques is 98% accurate [8.]. Several human-controlled variables help to achieve this level of accuracy[8.]; hence there is a lack of availability when implementing these techniques. This is where this research aims to reduce the requirements of lie detection by relying on Visual Features that are tracked with computer vision. The proposed multi-modal will track movements of the face and body to detect when a person is trying to detective. The model proposed will use data consisting of videos collected from public court trials[15.] and a variety of videos from Youtube. The classifier features to be tracked: Movement and Ground Truth micro expressions with Improved Density Trajectory (IDT), Facial Action Units (AU) with OpenFace, and MUMIN coding with 3D-CNN. By using fusion, the features will be processed utilizing a variety of algorithms: Decision Tree, Random Forest, Support Vector Machine (SVM), and Logistic Regression. This research aims to find a model with a methodology that increases or matches the current deception detection accuracy using only Visual Features.
Ethan Feldman
Rochester Institute of Technology
Detecting Review Manipulation using Transfer Learning in Heterogeneous Knowledge Graphs
Recent studies have revealed an increasing trend of "fake" reviews in the vast array of products available on e-commerce platforms. Consequently, consumer confidence in online rating platforms has plummeted to an unprecedented low, causing a trust crisis between individuals and companies. Recognizing the significance of this issue, researchers and practitioners have devoted considerable time and effort to detecting these deceptive reviews. Graph neural network (GNN) machine learning models have become a popular tool for detecting this manipulation; however, most existing methods suffer from a variety of limitations. Numerous models predominantly depend on homogeneous datasets that focus on only a single type of manipulation. Furthermore, a significant number of models struggle to obtain genuine fake reviews for training purposes, leading to the use of artificially generated reviews. This approach introduces bias into the models and hampers their ability to consider broader contextual information beyond their specific manipulation focus. To address these shortcomings, our study employs a novel zero-shot transfer learning approach using a heterogeneous GNN to detect review manipulation. By utilizing a network of interconnected review-reviewer-product data, we can provide valuable context for training the model. Additionally, leveraging the plentifully labeled product nodes allows us to transfer insights to review and reviewer observations that lack known labels. Our method manages the limitations of previous approaches and creates a model more robust to deception. We hope to improve manipulation detection and increase trust in online rating platforms.
Javon Hickmon
University of Washington
Thinking Beyond Images: Using Chain-of-Thought Prompting to Harness the Power of Language in Multimodal Models
Image classification is a fundamental problem in Computer Vision, and recent progress in Multimodal Machine Learning has enabled researchers to train large models using both images and text as input. Alongside the improvements in Multimodal Learning, recent work in Natural Language Processing has demonstrated the effectiveness of chain-of-thought prompting — a technique that allows the model to generate its own series of intermediate steps — in improving the performance of Large Language Models, particularly for tasks that involve common sense and symbolic reasoning. Despite these recent advancements, little work has been done to understand how improvements for Large Language Models affect the performance of Multimodal Models. I aim to demonstrate that incorporating chain-of-thought prompting into Multimodal Models can lead to significant improvements in accuracy for the task of few-shot image classification. I leverage OpenFlamingo, a powerful open-source 9 billion parameter Vision Language Model, to generate intermediate descriptions for the classification results, eliciting a chain-of-thought. Preliminary results indicate chain-of-thought improves the accuracy of few-shot image classification. Improving the task of image classification furthers the generalizability of Multimodal Machine Learning, effectively reducing the impact of hidden biases from a single modality of data which will result in fairer and much more representative systems.
Jimena Jimenez
University of Minnesota Twin Cities
Applying the Theory of Stochastic Computing to Networks of Spike-Based Neural Networks for Ultra Low Power Applications
Spiking neural networks (SNNs) have emerged as a third-generation artificial neural network, employing spiking neurons to encode data through spikes, closely resembling biological neurons. SNNs offer promising potential for efficient computing, leveraging event-driven, parallel processing, and are being explored for modeling the dynamics of the human brain and implementing compact deep learning neural networks. This research introduces a novel approach to implementing SNNs for ultra-low-power applications using stochastic computing. Stochastic computing (SC) is a unique computing paradigm that operates on probabilities rather than traditional binary numbers. Probabilities are encoded via streams of 0s and 1s. One of the major advantages of SC is its simplicity in performing multiplication operations. Unlike traditional algorithms that require complex multiplication circuits, SC achieves multiplication using just a single AND gate, which significantly simplifies the computation process. This research presents two models for a digital representation of the Izhikevich model spiking neuron synthesized for a field-programmable gate array (FPGA). One model employs a classical binary radix representation, while the other integrates stochastic computing theory. The stochastic computing-based design reduces the number of logic gates required for computation, resulting in lower power consumption and improved scalability. These findings contribute to advancing energy-efficient neuromorphic computing architectures by thoroughly investigating the application of stochastic computing in spiking neural networks.