The Special Collections department of the library at University of California at Santa Barbara has an ongoing project of digitizing pre-1920 wax cylinder recordings, and making the digital files available online. I found out about this project from the blog Ukulelia, who reported on some great early recordings of Hawaiian music. But you’ll find music from classical to popular song to early jazz as well.
My favorite recording so far? Well, there are some good recordings by the Ford Hawaiians, but I really like “Clarinet Squawk,” by the Louisiana Five, hot jazz from 1920.