In October 2022, Professor David Griffeath of UW-Madison approached me with an interesting recreational mathematics question: how efficiently could a computer find square palindromes, i.e. numbers which are perfect squares and which also read the same forwards and backwards (when written in base 10)? There are certain infinite families of square palindromes which are well-known. For example, , ,…
Travel is a powerful yet fleeting experience that can shape personal perspectives and support self-reflection. To recapture the essence of travel, we explored the use of VR as a medium for immersive re-experiencing with an emphasis on storytelling. We developed TravelGalleria, a VR authoring tool that allows users to curate personalized digital galleries. TravelGalleria encourages creative…
As augmented reality devices (e.g., smartphones and headsets) proliferate in the market, multi-user AR scenarios are set to become more common. Co-located users will want to share coherent and synchronized AR experiences, but this is surprisingly cumbersome with current methods. In response, we developed PatternTrack, a novel tracking approach that repurposes the structured infrared light patterns…
Sensing touch on arbitrary surfaces has long been a goal of ubiquitous computing, but often requires instrumenting the surface. Depth camera-based systems have emerged as a promising solution for minimizing instrumentation, but at the cost of high touch-down detection error rates, high touch latency, and high minimum hover distance, limiting them to basic tasks. We developed HaloTouch, a…
Virtual reality (VR) can immerse users into engaging experiences, affording opportunities to study behaviour in simulated contexts such as decision-making processes. However, methodological research into designing meaningful VR experiences – experiences that promote appreciation and deeper understanding of a work – is still underdeveloped. In this two-part study, we investigate how visual…
Non-Playable Characters (NPCs) are important in games, as they can provide guidance to the player, create social engagement, and advance the game’s narrative. Although much research exists regarding NPC interactions for traditional gaming environments, e.g. desktop or console, fewer works have considered this from a virtual reality (VR) perspective. Our work first uncovers the salient and unique…
Asymmetric AR/VR collaboration systems bring a remote VR user to a local AR user’s physical environment, allowing them to communicate and work within a shared virtual/physical space. Such systems often display the remote environment through 3D reconstructions or 360° videos. While 360° cameras stream an environment in higher quality, they lack spatial information, making them less interactable. We…
Lying and deception are important parts of social interaction; when applied to storytelling mediums such as video games, such elements can add complexity and intrigue. We developed a game, “AlphaBetaCity”, in which non-playable characters (NPCs) made various false statements, and used this game to investigate perceptions of deceptive behaviour. We used a mix of human-written dialogue incorporating…
Shared Mixed Reality experiences allow two co-located users to collaborate on both physical and digital tasks with familiar social protocols. However, extending the same to remote collaboration is limited by cumbersome setups for aligning distinct physical environments and the lack of access to remote physical artifacts. We present SurfShare, a general-purpose symmetric remote collaboration system…
UIUCTF 2023 had a challenge called Rattler Read, in which the goal was to exploit a Python sandbox, which ran user-provided code under RestrictedPython. RestrictedPython is a library which can be used to create safer subsets of Python, in order to execute untrusted user code within a larger process. For example, it can be used to provide a limited set… Continue reading "CVE-2023-37271:…
As the use of hand gestures becomes increasingly prevalent in virtual reality (VR) applications, prototyping Compound Freehand Interactions (CFIs) effectively and efficiently has become a critical need in the design process. Compound Freehand Interaction (CFI) is a sequence of freehand interactions where each sub-interaction in the sequence conditions the next. Despite the need for interactive…
Real-time communication with immersive 360° video can enable users to be telepresent within a remotely streamed environment. Increasingly, users are shifting to mobile devices and connecting to the Internet via mobile-cellular networks. As the ideal media for 360° videos, some VR headsets now also come with cellular capacity, giving them potential for mobile applications. However, streaming…
Sleep is important for humans, and past research has considered methods of improving sleep through technologies such as virtual reality (VR). However, there has been limited research on how such VR technology may affect the experiential and practical aspects of sleep, especially outside of a clinical lab setting. We consider this research gap through the lens of individuals that voluntarily……
Another year, another SSTIC challenge! It was great fun this year and I was introduced to a lot of new concepts in a short time. This year’s challenge featured a more parallel structure, with two short intro challenges leading up to a set of four “key recovery” challenges that could be completed in any order, and culminating in a final… Continue reading "[SSTIC 2023] Solving the SSTIC CTF…
Touch sensing on ad-hoc surfaces has the potential to transform everyday surfaces in the environment – desks, tables and walls – into tactile, touch-interactive surfaces, creating large, comfortable interactive spaces without the cost of large touch sensors. Depth sensors are a promising way to provide touch sensing on arbitrary surfaces, but past systems have suffered from high latency and poor……
I solved the inaugural Hexacon 2022 challenge and came in first place – earning a ticket to the conference and the chance to meet some of the best hackers in offensive security. The challenge was an epic five-part solo adventure through reversing, web exploitation, Linux exploitation, Hyper-V VM escape, and finally cryptography, in a similar vein to the SSTIC challenges… Continue reading "[Hexacon…
Narrative-rich video games provide opportunities for players to make choices at key points in the game, generating malleability within the game world and its characters. In this study, we explore the types of choices that exist in such games, how choices affect player experience, and how players make decisions when presented with choice. We first conduct interviews with game developers… Continue…
Many algorithmic approaches to 3D acoustic imaging have been devised which rely on a large abundance of receiving elements to produce images with delay-and-sum techniques, but these have found little use in air due to hardware complexity and low accuracy. Recent learning-based approaches to one-shot in-air acoustic reconstruction attempt to overcome these limitations using simple hardware and…
Random Reward Mechanisms (RRMs) in video games are systems in which rewards are issued probabilistically upon certain trigger conditions, such as completing gameplay tasks, exceeding a playtime quota, or making in-game purchases. We investigated the relationship between RRM implementations and user experience. Video analysis of 35 RRM systems allowed for the creation of a classification system…
Introduction I participated again in the SSTIC challenge this year, and had a blast. This year’s challenge consisted of six stages with individual flags, with the final stage providing an email address that had to be emailed to complete the challenge. The stages involved a diverse set of skills: file format forensics, reverse engineering (AVR assembly and 32/64-bit Intel Linux… Continue reading…
Problem Description SnECCanography Solves: 5 Score: 1000 Tags: forensics I hid a secret message in this picture of my pet snECC, but I don’t know what happened to it :(( Some of the pixels got corrupted… how do fix ?_? By oops (@oops on discord) Attachments: encode.py, snek2.png Introduction We’re given an 896 x 396 PNG image of a cute… Continue reading "[UTCTF 2022] SnECCanography"
Immersive head-mounted augmented reality allows users to overlay 3D digital content on a user’s view of the world. Current-generation devices primarily support interaction modalities such as gesture, gaze and voice, which are readily available to most users yet lack precision and tactility, rendering them fatiguing for extended interactions. We propose using smartphones, which are also readily…
Maple Bacon participated in TSG CTF 2021, which was organized by the Theoretical Science Group of the University of Tokyo. We placed 13th overall. During the competition, I worked on a problem called Catastrophe, which was about exploiting a deliberately introduced bug in the OCaml compiler. We ended up being the only solvers for this challenge! I chose this problem… Continue reading "[TSG 2021]…
TL;DR: We solved the “Ethernet from Above” hardware challenge at Pwn2Win CTF, which went unsolved during the main event. This challenge required reversing an FPGA design given only its binary configuration (bitfile). The organizers left it open as a bonus afterwards, so Daniel (@______esc______) and I collaborated to solve this challenge and claim the offered prize of 1 XMR. Read… Continue reading…
tl;dr: I solved the annual SSTIC Challenge solo, an epic 5-part challenge involving USB protocol forensics, Windows binary exploitation, whitebox cryptography, reversing an unknown instruction set, and Linux kernel exploitation, in the span of 75 hours. Read on to learn all of the gory details! All of the files used in this writeup may be found on the accompanying GitHub… Continue reading "[SSTIC…
Rapid iteration is crucial to effective prototyping; yet making certain objects – large, smoothly curved and/or of specific material – requires specialized equipment or considerable time. To improve access to casting such objects, we developed FoldMold: a low-cost, simply-resourced and eco-friendly technique for creating scalable, curved mold shapes (any developable surface) with wax-stiffened…
Inertial Measurement Units (IMUs) with gyroscopic sensors are standard in today’s mobile devices. We show that these sensors can be co-opted for vibroacoustic data reception. Our approach, called VibroComm, requires direct physical contact to a transmitting (i.e., vibrating) surface. This makes interactions targeted and explicit in nature, making it well suited for contexts with many targets or…
When sketching, we must choose between paper (expressive ease, ruler and eraser) and computational assistance (parametric support, a digital record). PHysically Assisted SKetching (Phasking) provides both, with a pen that displays force constraints with which the sketcher interacts as they draw on paper. Phasking provides passive, “bound” constraints (like a ruler); or actively “brings” the…
Disclaimer: Building strong cryptosystems is way harder than breaking them. Props to the system author for going for it and publishing the system – it’s a nice and simple design which is useful for learning cryptographic concepts. I hope this post helps inform future designs! I recently heard about the Sarah2 pen-and-paper cipher, touted as a “strong pen-and-paper cipher” and… Continue reading…
Augmented reality requires precise and instant overlay of digital information onto everyday objects. We present our work on LightAnchors, a new method for displaying spatially-anchored data. We take advantage of pervasive point lights – such as LEDs and light bulbs – for both in-view anchoring and data transmission. These lights are blinked at high speed to encode data. We built… Continue reading…
Low-cost, smartphone-powered VR/AR headsets are becoming more popular. These basic devices – little more than plastic or cardboard shells – lack advanced features, such as controllers for the hands, limiting their interactive capability. Moreover, even high-end consumer headsets lack the ability to track the body and face. For this reason, interactive experiences like social VR are underdeveloped.…
or: reverse-engineering a custom, unknown CPU from a single program tl;dr: We reverse-engineered a program written for a completely custom, unknown CPU architecture, without any documentation for the CPU (no emulator, no ISA reference, nothing) in the span of ten hours. Read on to find out how we did it… This past weekend, I participated in Dragon Sector’s 2019 Teaser… Continue reading "[DSCTF…
The UBC CTF team Maple Bacon participated in CyBRICS 2019; as part of this competition, I helped solve a challenge called Construction. Our team wound up being the only solvers on this challenge, and we solved it in an unintended way by effectively exploiting a “zero-day” bug in a Linux utility known as ss. Description We’re given SSH credentials to… Continue reading "[CyBRICS 2019] Construction –…
I played with PPP in the DEFCON CTF 2018 competition, held in Las Vegas from August 10 to August 12. The competition was fierce this year, with 24 teams in the finals (up from just 12 the previous year). This year, it was organized by a new group of people – the Order of the Overflow – taking over from… Continue reading "[DEFCON 2018] Doublethink – 8-Architecture Assembly Polyglot"
On May 16th, I found a vulnerability in the LocationSmart website which allowed anyone, with no prior authentication or consent, to obtain the realtime location of any cellphone in the US to within a few hundred feet. I immediately moved to contact US CERT to coordinate disclosure, and worked with Brian Krebs to publish the story after the vulnerability was… Continue reading "LocationSmart API…
Compact, worn computers with projected, on-skin touch interfaces have been a long-standing yet elusive goal, largely written off as science fiction. Such devices offer the potential to mitigate the significant human input/output bottleneck inherent in worn devices with small screens. In this work, we present the first, fully-functional and self-contained projection smartwatch implementation,…
We present MRTouch, a novel multitouch input solution for head-mounted mixed reality systems. Our system enables users to reach out and directly manipulate virtual interfaces affixed to surfaces in their environment, as though they were touchscreens. Touch input offers precise, tactile and comfortable user input, and naturally complements existing popular modalities, such as voice and hand…
Systems for providing mixed physical-virtual interaction on desktop surfaces have been proposed for decades, though no such systems have achieved widespread use. One major factor contributing to this lack of acceptance may be that these systems are not designed for the variety and complexity of actual work surfaces, which are often in flux and cluttered with physical objects. In this… Continue…
Homes, offices and many other environments will be increasingly saturated with connected, computational appliances, forming the “Internet of Things” (IoT). At present, most of these devices rely on mechanical inputs, webpages, or smartphone apps for control. However, as IoT devices proliferate, these existing interaction methods will become increasingly cumbersome. Will future smart-home owners…
insomni’hack is an annual security conference in Geneva, Switzerland. Each year, they host a teaser competition prior to the conference. Here’s a writeup from one of the problems, which was to reverse engineer a cryptographic algorithm from a binary executable, worth 400 points (a hard problem). Our team was the only team out of 300+ teams to solve this challenge.… Continue reading "[Insomni’hack…
We present CapCam, a novel technique that enables smartphones (and similar devices) to establish quick, ad-hoc connections with a host touchscreen device, simply by pressing a device to the screen’s surface. Pairing data, used to bootstrap a conventional wireless connection, is transmitted optically to the phone’s rear camera. This approach utilizes the near-ubiquitous rear camera on smart…
Several generations of inexpensive depth cameras have opened the possibility for new kinds of interaction on everyday surfaces. A number of research systems have demonstrated that depth cameras, combined with projectors for output, can turn nearly any reasonably flat surface into a touch-sensitive display. However, even with the latest generation of depth cameras, it has been difficult to obtain…
Tomography is an imaging technique that estimates the cross-sectional interior structure of objects through the use of an external, penetrating signal. In this research, we use Electrical Impedance Tomography (EIT), which uses surface electrodes and high frequency alternating current (AC) to measure internal electrical impedance. By placing many electrodes around an object, it is possible to…
Watches are unique among computing devices in that they are worn, offering great potential to transform arms and hands into expressive input and sensing platforms. As people use their hands, tiny micro-vibrations propagate through the arm, carrying information about the objects they interact with and the activities they perform throughout the day. Smartwatches are ideally situated to capture these…
We explore the concept of a sparse peripheral display, which augments the field-of-view of a head-mounted display with a lightweight, low-resolution, inexpensively produced array of LEDs surrounding the central high-resolution display. We show that sparse peripheral displays expand the available field-of-view up to 190º horizontal, nearly filling the human field-of-view. We prototyped two…
The Plaid Parliament of Pwning participated in (and won) BCTF 2016, hosted by Tsinghua University. Here’s a writeup of one of the problems, which was to exploit a Linux binary. It was worth 400 points (a medium-hard problem). We were given a vulnerable 32-bit x86 binary built for Linux called knurd, the libc binary, and the address of the server… Continue reading "[BCTF 2016] Knurd – Linux/Windows…
The Plaid Parliament of Pwning participated in (and won) BCTF 2016, hosted by Tsinghua University. Here’s a writeup of one of the problems, which was to decrypt a ciphertext encrypted with a variant of RSA. It was categorized as a cryptography problem and worth 500 points (a hard problem). We are provided a zipfile containing a censored picture of one… Continue reading "[BCTF 2016] Hyper RSA"
Today’s touch interfaces are primarily driven by the 2D location of touch events. However, there are many other dimensions of touch that can be captured and utilized interactively. By increasing the richness of touch input, users can do more in the same small space, potentially enabling richer applications. In this research, we describe a new method that estimates a finger’s… Continue reading…
Understanding who is interacting with a computing device has many implications and applications. It is most often used for authentication, where only certain users are permitted to use the system or privileged functionality. It can also be used for personalization, where different users can have custom settings, browser bookmarks, email accounts, favorite applications and so on. Furthermore,…
Gaze interaction is particularly well suited to rapid, coarse, absolute pointing, but lacks natural and expressive mechanisms to support modal actions. Conversely, free space hand gesturing is slow and imprecise for pointing, but has unparalleled strength in gesturing, which can be used to trigger a wide variety of interactive functions. Thus, these two modalities are highly complementary. By…