tl;dr Here’s my cv

This website has two parts: the part that makes sense, and the part that is my blog.

Highest level, coolest ideas

I study language models (LMs). They thrill, baffle, frustrate, charm, horrify, and elude me (just like any good friend!). I have many interests but the one I think I care most about is using language models to help humans to think, feel, decide, etc. more deliberately. Can language models help you decide whether you buy the argument that (bayesian > frequentist)? Can they help you decide whether you want to stay in a hard relationship? Can they help you decide whether to take or quit a job? Well yes, they definitely can; but should they? I don’t know if it’s crazy to see them as having the potential to be any number of friends and mentors and colleagues (students?) for us (all of whom, I’d point out, would be very well read).

BUT, that’s not a very concrete research agenda, now is it?

Concrete research agenda

Past: For the last few years, my research agenda has centered around identifying the extent of algorithmic fidelity in LMs, that is, to what extent have they modeled the ideas and behaviors of the humans who generated their training data? Can they channel real humans, take surveys like them, act like them, help other humans on opposite sides of the political aisle respect each other more in a conversation, etc.?

Future: Right now, I’m looking for my next concrete research agenda (read, job). This could go lots of ways. For example:

• Language models also still aren’t really being used in social science. I spent a fair amount of time in my PhD arguing that they could be used as proxies for human beings, in specific applications and domains. But I think all this study was just margin of possibility. Imagine if we could simulate RCTs on simulated humans in a way that reflected real human outcomes. What if we could simulate populations that are inaccessible in the real world? LMs allowing us to do something that was formerly impossible is, in my opinion, much more exciting than simply making it cheaper to do what we already can.

• Alignment is a big, philosophically fraught field of problems and questions. What does alignment even mean? To whom are we aligning? At what point in time? These are pretty philosophical questions, but there are practical aspects, too. I think it’s important to estimate quality of completions, certainty of models, think about how well-calibrated they are, etc.

• Literature review is something that LMs still cannot do well, let alone reliably. Semi-parametric models are cool because they can be thought of as decoupling the memory and reasoning parts of reading large text corpora. This feels like a reasonable solution to temporal generalization problems of LMs given that you can store your knowledge about the world in a dynamic search index, instead of in a transformer’s weights. But there are still plenty of problems to solve, like the expense of the approach, the building of the index, and all the typical failure modes of language models.

I can’t adequately stress how inadequate this short, sad list feels relative to the full one, but hopefully this gives you some sense.