Detecting out of distribution text with surprisal and entropy

When large language models refuse to help with harmful tasks, attackers sometimes try to confuse them by adding bizarre strings of text called “adversarial suffixes”. While reproducing a study on using perplexity to detect these attacks, I discovered that looking at individual tokens reveals a striking pattern: adversarial tokens aren’t just surprising — they’re surprisingly surprising given their context.

This led to the development of a novel visualisation technique and metric that makes these patterns immediately obvious to human analysts. The approach reveals how adversarial tokens can be identified by their unexpectedly high surprisal even in contexts where the model is confident about what should come next.

Read the full report on LessWrong →

Building digital spaces for the physical world

Customers today expect simple and seamless experiences. How do you create a digital tool that manages the complexities of designing a 3D space, that provides not only utility in planning and design, but inspiration, enabling users to explore their ideas and boost their confidence in what they’re designing? In this talk, Micah Sargisson and I present some considerations and challenges when designing 3D experiences, and how to approach product development and structure teams to solve difficult challenges.

Landblade

Landblade was a web app for performing search and analysis of continent-scale raster and vector data. It was a groundbreaking way to work with geospatial data.

The Raster Storage Archive

The Raster Storage Archive is a geospatial image storage system and query engine. This video whitepaper introduces the design and motivation of the storage system, and shows how the query engine can be used to analyse large datasets.

VPAC Engineering Showcase

A visualisation I developed at VPAC Innovations to demonstrate our engineering capabilities. The video shows a complex assembly (a limited-slip differential) with various physical simulations applied to it. A novel cross-sectional method was used for the transitions between shots. I modelled, animated, rendered and edited the video in Blender, with direction from Ariana Mugica Morales. The video was shown at the 2014 Grand Prix in Melbourne.

S. Cargo

S. Cargo is a 3D adventure game about a snail who has been robbed. The player controls Cargo, exploring the environment from a snail’s point of view. The snail integrates tightly with the environment; wrapping around objects, clibing walls, and brushing grass out of the way. The game focuses on immersion and story telling, with high-quality animation for the non-player characters.

I have a decent reputation on StackOverflow (top 5%).