It is not generally known that the body of Colonel Collins, the first Lieutenant Governor of this colony, lies interred in the corner of the churchyard next the town. About a year after his death a small wooden church was erected over the grave, and the altar, such as it was, was built directly over it. This building however did not remain long, for being got up in a hurry for the reception of Governor Macquarie to attend divine service in during his visit here, the stays were not properly fixed, and in a sudden gale of wind it was blown over.
Hobart Town Courier, 21 March 1829

On the St David’s cathedral website, there is an artist’s impression of that building, placed in context (use the slider below the image).

And this Monument long projected
was Erected to his memory in 1838,
By Direction of His Excellency
Sir John Franklin R.C.H.K.R
Site of the first Church erected in 1810 in
Van Dieman’s Land.
Built over the grace of
LIEUTENANT GOVERNOR COLLINS
whose body resided beneath the altar.