James Smith

Little is known about James Smith beyond his work at the White House under President James Madison. He is mentioned in Paul Jenning’s memoir, A Colored Man’s Reminiscences of James Madison. Jennings, an enslaved worker in President James Madison’s household, identifies Smith as “a free colored man” who “accompanied Mr. Madison to Bladensburg” to observe the battle between American troops and the invading British Army during the War of 1812. Jennings states that after the battle, on the night of August 24, 1814, Smith “galloped up to the [White] house, waving his hat, and cried out, ‘Clear out, clear out!’” He informed White House workers that the American army was in retreat, after which he left to rendezvous back with President Madison. First Lady Dolley Madison then ordered the White House to evacuate, leading to its subsequent burning at the hands of British soldiers.¹ When politician and writer Charles Ingersoll described the event in his 1849 history of the War of 1812, he referred to him as Madison’s “faithful, brave young slave, Jim” and says he delivered a note from the president to his wife.² Given the fact that Jennings worked with Smith in the White House and Ingersoll did not, it seems likely that Ingersoll was mistaken when he labeled Smith an enslaved worker. Whether Smith was free or not, the episode reveals that he held an important position within President Madison’s household, perhaps as his attendant or messenger.³ 

Three days later, President Madison wrote to Dolley to inform her of his whereabouts. He then said: “Jamey will give you some particulars which I have not time to write,” which indicates he intended Jamey to act as a courier. However, Madison added a postscript before sending the letter: “Since the above it is found necessary to detain Jamey, & sent a Trooper,” or soldier, instead. Given the similarity in names and roles, it is highly likely that Jamey and James Smith are the same person.⁴ 

James may appear in two other documents. White House Steward Henry F. Doyhar mentions a “Jimes” in a letter to President Madison in 1814, though it is unclear if this is the same person as James. Doyhar states that Jimes worked at the White House and purchased groceries for the household.⁵ In 1817, as President Madison was preparing to leave the White House and return to Virginia upon the end of his term in office, his nephew Robert Madison reported to the president that three of his enslaved workers named “Jim, Abram & Paul” were planning on escaping by taking work as cooks on merchant ships docked in Washington, D.C. It is unclear if this “Jim” is the same person as James Smith, however, since Jennings referred to him as free in 1814 which would mean he had no reason to escape onboard a merchant ship.⁶ 

Thank you to the Papers of James Madison, to Hilarie M. Hicks, Senior Research Historian, James Madison's Montpelier, and to Holly Shulman, editor of the Dolley Madison Digital Edition, for providing research assistance with this biography.