Giles Joly de Lotbinière's family rose to eminence in France and Canada and through his cousin, the Earl of Devon, the family can trace their line back to Hugh Capet, King of France in 987.
In 1787, whilst in Canada, the family was granted arms as Marquis de Lotbinière by Louis XVI of France. Rene Louis Chartier de Lotbinière had already, in 1672, been created Seigneur de Lotbinière of Quebec by Louis XIV.
His descendant Sir Henry Gustave Joly de Lotbinière became the sixth Seigneur de Lotbiniere and was also the First Minster of Quebec and Governor of British Columbia. His grandson, Brigadier General Henry Gustave Joly de Lotbinière DSO, established the British line of the family and settled at Brandon Hall in Suffolk. His son, Sir Edmond Joly de Lotbinière's first wife, The Hon Elizabeth Joliffe, was a direct descendant of Henry VII, and, through her, Giles can trace his line back to the great families of Tudor times. Veronica Joly de Lotbinière (nee Levett-Scrivener) is descended from a prominent Suffolk family who bought 20,000 acres from the Duke of Norfolk in the 16th century to add to their estates in East Suffolk. Her ancestors include Bishop John Fisher, patron of John Constable and close confidante of George III. Veronica's great great grandfather, Sir Harry Parkes, was Ambassador in Beijing in 1883, and his wedding present to his daughter Mable Parkes Levett-Scrivener was a tiger claw necklace from the Forbidden City, where he resided. The necklace is still worn by her great grand daughter Veronica. Veronica's ancestors include Admiral William Bligh (1754-1817) who was Captain of the Bounty, and its infamous mutiny has been made into several films. Also General Augustine Prevost (1723-1782) who commanded the British Army in the invasion of North Carolina in 1779 and his son Sir George (1769-1816) who became Governor General and Commander in Chief of the British Forces in North America during the War of Independence.