The first of the three Joshuas was an eccentric character and he lived in an age of great laxity in Church matters. It is related how, during the incumbency of Isaac Wilson, who was for 56 years the non-resident vicar of Caistor, a certain curate who was responsible for carrying out the duties which ought to have been performed by the absent parish priest, used to be very fond of taking a pipe of tobacco in the vestry before the sermon, while the congregation sang a psalm. On one occasion the psalm ended before the pipe and Joshua Foster came in to tell the clergyman that the people were waiting ; "Let them sing another," said the curate. Presently, back came Joshua to announce that the second psalm was finished ; "Let them sing the 119th" said the curate ! Eventually, however, the pipe was done and the parson began to put on his large, black preaching gown but the folds were troublesome and he called Joshua to his aid. "I think the devil must be in this gown," he remarked, as he wrestled with it. Old Foster waited until the garment was duly on and then dryly remarked,  "I should say as how he is, Sir." This anecdote is related by the late P. H. Ditchfield in his book, The Parish Clerk, but it is only fair to state that the same story may be found, told of other places also.
At the time of the enclosure of Caistor a small field on the Brigg road was allotted to the parish clerk by virtue of his office and is consequently known as "The Clerk's Piece."

THIS MAN, whose name is sometimes spelt Bernard, was sufficiently prominent in English history to be accorded a place in the Dictionary of National Biography. He was the son of a lawyer and was born at Caistor in 1628. Having received his early education at the recently founded Grammar School here, he was admitted at Queens' College, Cambridge, in 1645 and migrated to Oxford three years later. He graduated B.A. and became a Fellow of Lincoln College, Oxford in 1650 and eventually proceeded to the degrees of M.A., B.D., and D.D. In 1656 he became Rector of Waddington and in 1683, Vicar of Gedney and Prebendary of Asgarby in Lincoln