He left LSU to enlist in the United States Army during World War II and served with a B-24 bomber unit in England and Africa.
In 1946, after military service, Gilbert married the former Barbara Jane Peck (October 26, 1922 – May 1985) of the Ferry Place Plantation in Sicily Island, the daughter of William Smith Peck, Sr.
Many cultures lived in the area before Catahoula was established.
The peoples of the Marksville, Troyville, Coles Creek, and Plaquemine cultures built villages and mounds thoughout the site.
Gilbert was a great-nephew of former State Senator Thomas Benjamin Gilbert, II (1864–1931), who served from 1904 to 1908 and again from 1916 to 1932. Instead she moved into one of the Gilbert homes in Wisner and for many years afterwards operated a boarding house with three meals daily for her patrons.
Jess and Fannie shared an October 27 birthday but a year apart.Fannie was a daughter of William Hughlett Adams, a Franklin Parish sheriff.Gilbert's sister, Frances Virginia Gilbert Martin was his elder and only sibling.(1) Though he is a member of a pioneer Catahoula Parish family, Gilbert grew up in a boardinghouse operated by his mother, who was widowed at the age of twenty-eight. "Sonny" Gilbert (March 6, 1922 – November 21, 2014), was a cotton farmer and a former Democratic member of both houses of the Louisiana State Legislature from the town of Sicily Island in Catahoula Parish in northeastern Louisiana.(2) In 1976, after his term in the Louisiana House of Representatives, Gilbert was named a director of the Louisiana Department of Wildlife and Fisheries under appointment of then Governor Edwin Edwards. Gilbert served three consecutive terms in the Louisiana State Senate from 1960 to 1972, having represented Franklin, Richland, and Catahoula parishes.