Gravitational waves from black hole binaries bring us a unique window into extreme gravity that will allow us to perform high-precision tests of Einstein’s theory of General Relativity (GR). However, in this talk I will address a more ambitious question: could we observe signs of modified gravity using gravitational waves? I will focus on the case of higher-derivative corrections to Einstein’s theory, which are well-motivated theories from the viewpoints of effective and fundamental theory. I will first introduce a general family of higher-derivative extensions of GR and I will show how the Kerr metric, which represents a rotating black hole in GR, is modified by these corrections. Then I will discuss recent efforts towards understanding the observational signatures of these modified Kerr black holes.