Vehicle tracking technology holds diverse applications across various industries worldwide. Most commonly, vehicle tracking technology uses a combination of sensor nodes, which capture relevant location data, and computational units, which process the sensor data, to execute vehicle tracking. The underlying assumption in the context of vehicle tracking is the availability of appropriate sensor nodes. While the simplest approach to fulfill this assumption is to activate all sensors at all times, doing this proves impractical due to cost and efficiency concerns, especially if the area of interest is expansive. The lab thus focuses on exploring algorithms that decide when to activate specific sensors and has shown that the Maximum A Posteriori with Arm Elimination (MAP_e) algorithm outperforms other multi-armed bandit algorithms in terms of sample cost, accuracy, and latency. Our work involved simulating a vehicle tracking scenario using Python. The simulation depicted a grid representing the area of interest, with a combination of available and blocked (unavailable) cells, allowing the vehicle to move in specific directions. We provided extensive simulation results where we explored a number of scenarios and algorithms, including our proposed improvements to the algorithm as well as the consideration of error propagation while tracking vehicles.

As electronic equipment such as atomic clocks, transmission cables, and microprocessor-controlled devices become increasingly miniaturized, the magnetic fields they produce will interact in ways never seen before. Problems such as shifts in atomic transition frequencies within chip-scale atomic devices arise when we compact systems to the micro-scale. My research focuses on using alternating layers of ferromagnetic and diamagnetic material to produce a micro-scale shield that prevents low-frequency magnetic fields from entering the protected area. Permalloy has lower magnetic reluctance due to its higher magnetic permeability and thus transmits the magnetic field through materials more easily, effectively shielding the inner components by redirecting the field lines. My research extends the work of Wu et al., which demonstrated that large numbers of thin shields can provide a much higher shielding factor than a single shield with the same material thickness. By parameterizing the dimensions of shield components within COMSOL Multiphysics, I ran various simulations that accurately portray how physical shields would behave under predetermined conditions. Specifically, using the “Magnetic Field, No Currents” package, I simulated a magnetic field on shield designs to determine the resulting magnetic flux density on the shield and surrounding areas. Through these simulations, I found that the optimal solution is a five-layer slanted shield with 300μm x 300μm through silicon vias (TSVs) every 200μm. Ultimately, the miniaturization of shielding allows for the isolation of superconducting electronic chips from each other and the external environment, which opens up a multitude of possibilities for microelectronics.

While intelligent tutoring systems (ITSs) can use information from past students to personalize instruction, each new student is unique—with their own learning style. ITSs should therefore be able to interact with students in a way that maximizes their learning success while respecting their time. By turning to reinforcement learning (RL) algorithms, ITSs can both learn and adapt to new students. Practically, the education problem is partially observed and partial observability typically increases the difficulty of RL significantly; so, we explore what happens when we can use probing interventions to get more information. Gathering information through these interventions reduces the difficulty of final estimation, but it also introduces a cost-benefit decision on how often we want to probe versus help. As a result, our solution seeks to find a balance between probing enough to get accurate estimates and probing so often that it becomes disruptive to the student. We develop a dynamic, time-series environment to simulate a classroom setting, with student-teacher interventions—including tutoring sessions, lectures, and exams. We evaluate the efficacy of standard RL algorithms under several degrees of partial observability. Our results across Q-learning, Deep Q-learning, XGBoost, and Random Decision Forests (RDFs) are varying but demonstrate effective learning algorithms. In addition, Deep Q-learning, XGBoost, and RDFs are more resistant to changes in partial observability than Q Learning. The models that we develop using these learning algorithms can be used to project when a student should obtain assistance based on their learning trajectory.

Communication channels are imperfect due to noise interfering with the transmission, resulting in distorted messages. Ensuring reliable communication systems that minimize the Codeword Error Rate (CER) is important in such scenarios. One effective method to combat this issue is the use of a serially concatenated code with a Cyclic Redundancy Check (CRC) as the outer code and a Tail-biting Convolutional Code (TBCC) as the inner code. This concatenated code leverages the strength of different error-detecting codes to improve overall performance. An Expurgating Linear Function (ELF) is a generalization of the CRC that doesn’t restrict the outer code to be cyclic. For a variety of interesting cases, there are no cyclic codes available so the ELF generalization provides an important insight. This project focuses on the specific case of ELFs used as outer codes for a TBCC and seeks to understand how cyclic codes perform within the larger space of ELFs. By their nature, cyclic codes used for the expurgation of TBCCs will remove or retain all cyclic shifts of a codeword. We proceed to examine the full set of ELFs for cases where the TBCC and ELF redundancy are fixed. Performance is evaluated using union bounds on CER. Our results suggest that if cyclic codes exist in the set of possible ELFs when the TBCC and ELF redundancy are fixed, then the best ELF is a cyclic code. We haven’t found a counterexample to this conjecture, which significantly reduces the search space by restricting attention to cyclic ELFs when available.

Contextual linear bandits is an important problem with diverse applications in online advertising, recommendation systems, and healthcare interventions. In contextual bandits, a learner interacts with an environment by sequentially selecting actions in the changing action sets based on the context and receiving rewards accordingly, which are measured by the dot product of the chosen action and an unknown parameter. The objective is to minimize the regret, which is defined as the cumulative difference between the rewards obtained by the learner's chosen actions and the optimal rewards in hindsight. The problem is considered more challenging than linear bandits, where the action set remains fixed in each iteration. As a result, more simpler and straightforward algorithms have been developed for linear bandits compared to contextual bandits. In previous work done by the lab, Osama devised the reduction algorithm to address the contextual linear bandit problem by simplifying it to a linear bandit problem. This reduction algorithm enables all developed and future algorithms for linear bandits to solve contextual bandits. However, the reduction framework was only proved theoretically, with its practicality unknown. In this project, we implement the reduction framework from scratch and compare its performance with conventional algorithms. By applying the reduction, we attain similar regret for contextual bandits, as state-of-the-art algorithms, using simpler algorithms. Furthermore, we observe that the regret obtained from the reduction method appears to be even lower than conventional approaches for a number of instances.