Reinforcement learning (RL) has been a growing subset of machine learning with increasing success and promise - but it has just begun to be used in complex, multiplayer environments and games. Through trial and error, an agent begins from fully random trials and finishes with sophisticated actions. We apply RL to the imperfect information game known as Liar’s Dice, which presents a challenging mix of two-player dynamics and partial information to explore. The game forces players to call bluffs and doubt opponents while reading others’ potential actions. Implementing reinforcement learning to imperfect information games allows us to find successful strategies and models in dynamic Markov Decision Process (MDP) environments that require sequential decision making. We employed the popular Q-learning method of RL to train agents that begin with random actions or to use a combination of fixed strategies against others. An agent employing Q-learning improved its win rate from 50% to only 65% within 1,000,000 episodes against a simple agent. Another agent that made decisions based on various fixed strategies available increased its win rate from 11% to 77.7% with 100,000 episodes. These rates demonstrate the variability of Q-learning in a game with partial information. In future work, we may compare counterfactual regret minimization and more state-of-the-art RL algorithms, which would expand our understanding of various methods on partially observed, dynamic environments. By studying this game, we hope to broaden our results to the education space, a similar Markov process where individuals also make decisions sequentially.

Electrowetting-on-dielectric (EWOD) is a mechanism that enables the manipulation of droplets through electrical signals alone. Due to its advantages for droplet-based microfluidics, such as the simplicity in both device design and fabrication, EWOD has demonstrated its utility in numerous biochemical and biomedical applications, especially lab-on-a-chip. However, currently the EWOD technology is being utilized by only a small number of labs who have proper engineering backgrounds, enough resources to design and fabricate devices to fulfill their goals, and control systems and software to operate the devices. To combat this barrier, the UCLA Micro and Nano Manufacturing Laboratory is developing a cloud-based cybermanufacturing platform for common users to gain easy access to the EWOD technology. In furthering the mission, the lab is exploring a remote operation of the EWOD control system by introducing an Internet of Things (IoT)-based intermediate system. This remote operation will serve as a gateway to an envisioned lab-on-cloud that will help democratize EWOD technology. Acting as a broker for information exchange between a backend server and EWOD control system, the intermediate system is designed to run users’ instruction files, communicate its messages to the EWOD control system, and operate alongside a camera for users to remotely monitor their experiments. My work has involved developing a proof-of-concept demonstration that one can operate an EWOD control system by sending the necessary commands from a remote location.

As the world rapidly turns to robots, it is important to make robot creation ubiquitous, but there are barriers – accessibility and limitation of resources – that inhibit such phenomena. In this study, we proposed and tested a possible solution that can lessen those barriers by building origami robots made with accessible resources. Origami is defined as the Japanese art of folding. To test our hypothesis, we implemented a process that fabricates origami robot cars made from single-layered materials like cardstocks while ensuring that these cars can still carry out several robotic behaviors. The process includes gathering accessible hardware, programming behaviors using Arduino, designing the origami-inspired body of the car using LEMUR’s RoCo, and testing the car to determine if its origami-structure can handle the programmed robotic functionalities: driving on a variety of surfaces, using differential and pivot steering, detecting obstacles, implementing PID control through IMU and visual sensors, and enabling communication with other robots through mesh networking. The scheme yielded robot cars that are useful and fairly accessible with cardstock-made bodies and with a variety of robotic behaviors as mentioned above. The result of our study justifies that the implementation of origami robots has a huge potential in terms of lessening the resource-related barriers of robot creation which increases both the number of people who can build robots and the probability of robot creation being ubiquitous.

Macrocycles represent attractive synthetic structures as they can modulate “difficult-to-drug” targets such as proteases, GPCRs, and protein-protein interactions. Despite their increasing popularity among chemists, a clear guideline to assess the passive permeability of this class of compounds does not currently exist as their physicochemical properties fall outside traditional permeation rules. In order to rationalize the molecular determinants of cell permeability for macrocycles, we primarily analyzed three-dimensional factors obtained from energy-minimized conformers using Principal Component Analysis (PCA). Our studies revealed that rod- and disk-shaped compounds were the strongest predictors of permeability after accounting for molecular weight, flexibility, and polarity. By fine-tuning physiochemical and structural properties, medicinal chemists are able to design and select cell-permeable macrocycles with a rational approach.

Practically speaking, most communications channels are imperfect; noise will interfere with the communications and corrupt transmitted data. In order to combat this, many communications systems utilize feedback- the practice of relaying information regarding received data back to the transmitter- in order to efficiently transmit and decode messages. Research in this field, arguably set in motion by Michael Horstein in 1963, studies this phenomena and ways to mitigate interference. This research builds off an existing algorithm created by Amaael Antonini and Rita Gimelshein, which uses causal encoding over the the BSC (Binary Symmetric Channel), a channel through which binary messages can be transmitted with an equal crossover probability of zeros and ones. It modifies the algorithm to utilize sparse feedback instead of bitwise feedback- sending feedback after a specially determined number of bits have been sent through the channel, rather than after every bit, aiming to increase efficiency without loss in performance.

Toxoplasma gondii is an obligate intracellular parasite that infects nearly one-third of the human population, making it one of the most common parasites in the world. However, little is known about the secretory pathways that mediates intracellular trafficking between the ER-Golgi-Plasma Membrane of T. gondii. To better understand secretory protein trafficking, we are focusing on a series of Tre2–Bub2–Cdc16 (TBC)-domain containing proteins. These proteins inactivate RAB-GTPases, which are involved in vesicle fusion and determining their precise localization and function will help us to understand secretory protein trafficking in the parasite. To explore Golgi trafficking mechanisms, candidate Golgi proteins were identified using in vivo biotinylation and subsequently localized by epitope tagging. Four novel TBC-domain containing proteins were discovered to localize to the Golgi complex. These four TBC-domain Golgi proteins were then colocalized against each other to address similarities and differences in their localization in relation to the Golgi and post-Golgi structures. Together these studies provide new insight into the Golgi complex in T. gondii and identify putative targets for the design of novel therapeutics that can specifically target the parasite.

Thermoplastic polyurethane (TPU) is a class of material that combines the desirable elastic properties of rubbers with the ease of manufacturing of plastics, and are widely used in various industries sectors. The objective of this study is to fabricate TPU lattice structures as light-weight energy-absorbing materials by fused deposition modeling (FDM) 3D printing. Since the extrusion of TPU from the nozzle and adhesion on the surface is fraught with challenges, dog-bone samples were first printed to characterize the correct printing parameters, such as temperature, printing height, and printing speed. It was observed that increasing the print temperature and reducing the extrusion rate improve the print quality, (i.e. reduction in missing material, burning and clogging). To further print TPU lattices full of overhanging structures, water-soluble Polyvinyl alcohol (PVA) is used as the supporting material. Unit cells of an octet truss lattice were printed, and it was found that by increasing layer thickness, disabling retraction, and enabling sacrificial nozzle wiping structures provide the best prints.

Terahertz light has demonstrated the ability to identify complex molecules via spectroscopy. However, its potential to do so has been untapped due to a lack of capable, broadband, non-dispersive sources. The double ridge design of the metasurface is a viable path toward bridging this gap by broadening the amplification bandwidth of metasurfaces to create widely tunable quantum cascade external-cavity lasers with low group delay dispersion (GDD). The design consists of repeating units of two differently sized ridges, each of which corresponds to a resonant frequency at which there is a peak in reflectance (a measure of light amplification). Altering the widths of the ridges and the separation between them allows for manipulation of the resonant frequencies so that they are near each other, creating a continuous range over which amplification is high. Previously, design dimensions were determined by running simulations where tested parameters were manually input, and collected data was then analyzed to determine favorable geometries. To more efficiently and accurately find an optimal geometry, MATLAB functions representing bandwidth and GDD were written in order to quantify the values to be optimized. The simulation software COMSOL Multiphysics as well as its optimization module were then used with these functions to create optimization studies that determined the dimensions that minimize dispersion while maintaining broadband performance. Initial results suggest that GDD could be improved upon by 1.72% while maintaining the same broadband performance as a previous design, and 7.48% at a slight cost (0.125 THz) to the bandwidth.

Chikungunya virus (CHIKV) is a re-emerging alphavirus that causes fever, rash, and arthralgia. Despite its low mortality rate, more than half of the patients experience incurable chronic arthralgia and muscle pain even years after infection. Previous studies implicate CHIKV infection of macrophages in the onset and persistence of arthralgia, and preliminary data from our lab suggest that CHIKV has a more robust infection and higher virus titers in THP-1 monocyte-derived macrophages (MDMs) than other alphaviruses, including the closely related o’nyong-nyong virus (ONNV). A better understanding of how CHIKV can strongly infect macrophages could help us develop novel treatments to lower the prevalence of CHIKV-induced arthralgia. To identify the viral proteins that allow CHIKV to infect MDMs, we constructed a repertoire of CHIKV and ONNV chimera viruses and determined that the structural proteins were responsible for the enhanced infection of CHIKV in MDMs. We are generating chimeras of the individual structural genes and swapping the domains of the structural genes to determine which specific region of which structural protein is responsible for strong CHIKV infection. In addition, we are working to design CHIKV and ONNV E2 reporter viruses to visualize and compare the viral entry and exit pathways. We aim to use a mass spectrometry screen to identify host factors that interact with the structural proteins responsible for CHIKV’s enhanced infection in MDMs; through this screen, we hope to identify novel host factors that could eventually be used as therapeutic targets against CHIKV infection and chronic arthralgia.