Fitting study in around work

I’ve been pretty engrossed with my new job, so studying took a back seat for a while. I’m getting back into it now, and I’ve found several good resources to help me to catch up.

The firehose of research

My most consistent study method is to listen to podcasts while commuting. Actually mostly YouTube videos; these channels are my staples:

• Dwarkesh Patel, 80,000 Hours, and The AGI Show¹ for deep, long-form interviews with some incredible people right in the thick of it²
• Tunadorable and AI Coffee Break for curated paper readings
• Rob Miles and Rational Animations for my daily dose of doom
• MattVidProAI and Curious Refuge to keep one foot on the hype train
• AI Explained and Sam Witteveen for analysis of the rate of capabilities development
• Andrej Karpathy and 3Blue1Brown for deep dives into the code and maths.

Homework

In terms of actual pen-to-paper work, I recently read a summary paper on alignment challenges³. I marked it up on a paper tablet⁴, and had fun writing up a mock research proposal for addressing one of the challenges.

I’ve also been messing around with transformers in Colab, seeing what happens when rearranging the architecture a little. It’s not my first time writing deep neural network code, but it is the first time I’ve done more than inference with LLMs. I’d like to share it but I need to consider whether it’s an info-hazard. It’s pretty small-scale stuff though, so it’s probably fine.

1. The AGI Show has ceased, but there are some good interviews in the catalogue. ↩

2. Never had I listened to a 4.5h interview before, but we live in the future now and it’s weird here. ↩

3. Anwar, Usman, et al. “Foundational challenges in assuring alignment and safety of large language models.” arXiv preprint arXiv:2404.09932 (2024). ↩

4. It was a borrowed ReMarkable 2, but it may now be my preferred way to read papers. I might have to get one. Not with the default pen though, it’s too heavy. ↩