The Sermon was the second of two albums recorded on two dates at The Manhattan Towers Hotel Ballroom, the first was Smith's previous album, House Party (1958). Rudy Van Gelder used the ballroom as a recording studio for recording sessions in 1957-1958, while he was still using his parents' Hackensack, New Jersey, home studio to record artists for Blue Note. He mainly used it for larger groups of musicians that would not fit in his parents' living room, or when New York was a more convenient location to record the artists involved.