Born on June 7, 1821, in Sheridan, Chautauqua County, New York, Keziah was the daughter of Ezra Washburn and Phoebe (Brown) Nason. She was the third of six children. Between 1827 and 1832, Ezra moved his family to Greenfield, Erie County, Pennsylvania, where their youngest child, Phoebe, was born in March 1832. The family was enumerated in that place on the 1840 Federal Census. However, they are found in French Creek, Chautauqua County, New York, on the New York State Census in 1845.
Keziah was married to Silas Thomas Bemiss on April 27, 1853, in Greenfield, Pennsylvania. They had two children born to them: William Silas and Phoebe Silinda. Both of the children were born in Pennsylvania. They lived in Erie County until 1884, when they removed to Gove County in western Kansas.
Keziah died in Grinnell, Gove County, Kansas, on December 1, 1888. She is buried in the Grinnell Cemetery.