After their marriage Pierce and Cornelia set up home in Natchez, Mississippi, where Pierce had accepted the rectorship of Trinity Episcopal church. They lived for three years at White Cottage where their first two children, Mercer and Adeline, were born. In August 1835 Pierce resigned his pastorate in order to explore the claims of the Roman Catholic Church.

For both Pierce and Cornelia religion held a place of importance. They were young believers who searched for answers to the most serious questions of their lives