When We Did Research By Hand

How ML/NLP Research Changed in the Course of My PhD

I’m approaching the end of my time at CMU and I’ve been reflecting on my PhD experience. In hindsight, though my PhD itself technically didn’t last an absurdly long time, I feel like I’ve witnessed a lot of change in both the research landscape and myself. Things have changed a lot, and I find that my mentors are also uncertain on what the future holds when I ask for advice. I find myself also hedging when asked by younger students: what should they focus their work on nowadays? Does it make sense to commit themselves to a 5 year or longer PhD anymore? Will their intellectual labour still hold value in the next decade?

Continue reading →

How to apply for and get compute grants (for students)

Every PhD student has complained about compute sometime in their lives. At this point, it’s the academic equivalent of complaining about not having enough time – treated as an immutable fact of life, rather than a problem that could actually feasibly be solved. Unlike time however, compute exists in (relative) abundance, if one is willing to look for it.

Continue reading →

Underrated life hack: working outdoors

Continue reading →

The Promises and Pitfalls of AI Scientists

Recently, I saw news that an AI-agent generated paper was accepted to an ICLR workshop. I’ve been interested in this topic for a while, and some masters students I’m working with are currently building a benchmark for end-to-end scientific reasoning in LMs (from idea generation to coding/execution), so I was curious to read the paper. I’m not actually skeptical that LM-based agents can eventually automate parts of research or serve as assistants in many aspects of research. In fact, I often ask LLMs to fetch literature related to research ideas, draw plots, critique ideas, and more. If you haven’t tried this yet, you should! Sometimes it’s not very helpful, but the LMs tend to call every idea you pitch brilliant and innovative, which is very good for building confidence (NOTE: this is not referring to gpt-4o’s recent update, which verges into sycophantic). I explain this to say that I wasn’t looking for flaws at all, and was rather thinking about how this particular system could be benchmarked.

Continue reading →

I'm an NLP Researcher. I didn't check Twitter for a year. It was fine.

(Before anyone who knows me says “wait a minute, I’ve seen you post things on Twitter!”, I have occasionally posted and reposted papers for work reasons more or less, but for more or less a year, I haven’t opened Twitter to browse through new posts at all. This is what I mean by “I didn’t check Twitter for a year”.)

Continue reading →