St Johns Ave, New Town. Google Maps. The church is flanked by the buildings of the King/Queen’s Orphan school which were opened two years earlier.
Foundation stone laid 1834.
Opened 1835, to provide a second Anglican church for the residents of Hobart Town and surrounds.
Consecrated 1838.

Some background (and better photos)
Organ (and some nice internal photos)
Friends of the Orphan Schools: background, interesting photos (inc. inside of tower) and associated buildings.
Memorial plaques/stones

Other images:
Plans (1 or 2) and cross-section
Drawing, 1831
Paiting, 1840s
Interior, stereograph c. 1865
Interior
More

St Johns Avenue leading up to church, with  orphan school on either side. (c. 1880, from my collection).

When, in our last, we incidentally mentioned the erection of a new Church in Hobart Town, as one, among other under takings, which we thought the Government would do well to commence, with part of the surplusage of last year’s revenue, we were aware that a project was on the carpet for something of the sort, by connecting a Church with the proposed new buildings for the King’s Orphan Schools, near New Town, but had not been made acquainted with the precise plan, and therefore reserved our observations until the matter was more fully before us. Looking at the very great, the paramount importance to all classes, of religious instruction, and the opportunities of attending public worship, we are always glad when we see any thing on foot that has the facilitating or the promotion of these objects in view ; and so far, we would encourage and support the designs of the gentlemen who have promised to lend their aid in the manner now promulgated; but when we Consider the present extent of Hobart Town its constant and rapid increase and its still more rapid increase of population, -we confess we do not think the mere co-operation of a few private individuals with the Government towards building a Church, chiefly for the convenience of New Town, or of that part of the Hobart Town suburbs which border that village, ought to withdraw the attention, either of the Public Authorities or of the inhabitants of Hobart Town, from the main point; and we hope, therefore, yet to learn that the design, which we understand has been, long under consideration, will be persevered in, and that we shall shortly have a second Church in Hobart Town. We are very willing to ascribe it to the zeal and other fitting qualities of the gentlemen who so respectably fill our pulpits, but certainly, as a body, the inhabitants of Hobart Town mark the Sabbath in as becoming a manner as is perhaps, to be met with in any town of the same size in the World. If therefore, the vineyard be so prolific, give us say we, an adequate supply of husbandmen let not the fruit perish for the want of being gathered; and, since speaking in round numbers, the different places of Public Worship in this town if all filled at the same moment could not accommodate one-third of our inhabitants, were they all desirous of attending, let them not hereafter, be allowed the opportunities of saying, in extenuation of their neglect, that there was no room for them in any house set aside to the service of their Maker.
Colonial Times, 25 June 1830

We are glad to see that a commencement has been made on the new Orphan Schools and Church at New town. We have had the pleasure to see the elevation of it, and very much approve of the design, which if executed agreeably to the plan will be a striking ornament to the architecture of’ the colony. The Church is to be of Gothic architecture, with a handsome spire and turrets corresponding. It will form a striking object as the parish Church at New town, situated about 300 yards on the left side of the road, and on this side of the rivulet fronting the river. It is anticipated that two years will suffice to complete the whole edifice, but divine worship will be celebrated by the Rev. Mr. Norman as soon as the south wing of the Orphan School is finished, to accommodate the inhabitants.
Hobart Town Courier, 4 September 1830

We are happy to inform the inhabitants of New town and its vicinity, that until the parish church is completed arrangements have been made for the performance of Divine Service every Sunday afternoon, at the Orphan School, beginning at 3 o’clock.
Hobart Town Courier, 19 April 1833

The regular attendance of Crown prisoners at Divine Service at the Barracks will be much felt at St. David’s. Church, as it will throw open the gallery in accommodating many more of the inhabitants, who are prevented from attending the Established Church for the want of a second Church. The Penitentiary Chapel will also afford some trifling further accommodation to the Public, until the New Church shall be built ; which will not only be a vast improvement to the appearance of Hobart Town, but be instrumental, in preventing one half the population thereof in becoming dissenters, or making their own homes places of public worship. We, therefore, hope that the temporary accommodation in the gallery of St. David’s Church, or the opening, of the Penitentiary Chapel, will not delay or, prevent the erection of the New Church.
The Colonist, 9 July 1833

Right hand/northern wing of orphanage.

Ceremony
Of laying the Foundation Stone of the Church, at New Town.
The very interesting ceremony of laying the Foundation-stone of the New Church, at the Orphan School, took place yesterday. –A vast con course of the most respectable inhabitants assembled on the occasion. The 21st Regiment and Band arrived about 2 o’clock — the Officers and men of that Gallant Corps being in full costume; His Excellency, and Suite arrived about 3 o’clock. The following details of the Procession and Ceremony has been given us by a Correspondent :–

The Procession being formed, it approached the spot where the, Stone was to be laid in the following order:–

TWO CHAPLAINS
The Treasurer (with the Coins) and the Architect with the Trowel.
Magistrates, Public Officers, and Gentlemen,
Two and Two.
THE COMMITTEE.
His Excellency the Lieutenant Governor ;
Supported by Colonel Leahy, the Chief Justice, and Colonial Secretary.
The Clergy.

The Procession filed off on each side, when His Excellency approached the stone. The Architect then presented his plan, and recommended that the Foundation-stone be laid at the north-east corner of the building; this being assented to by His Excellency, the Architect proceeded to fix the lower part of the Stone* in its proper situation. The upper part of the Stone was then raised by the engine and one of the Chaplains repeated the following short prayer:–

“May the Great Architect of the Universe deign to assist us in our present convention ; and may we be enabled to raise on the Foundation-stone we are now about to lay, such a superstructure as may tend to promote His Honor and Glory !!”

An Anthem was then Sung ;

During which the Treasurer proceeded to de posit the Coins in the cavity of the lower part of the Stone, and the Architect fixed the Plate, containing the inscription. His Excellency, then descended to the foundation, and assisted by the Architect, proceeded to spread the mortar. The upper part of the stone having been let down to its proper situation by three regular steps, (and the anthem concluded), His Excellency applied the square, plumb-rule, and level, to the stone, and giving it three knocks with the maul, concluded this part of the ceremony by saying—

“As we have now laid the foundation-stone, may the Almighty Architect of the Universe, by his kind providence, enable us to carry on and finish the work which we have began ; may he be a guard to this building, and .may he preserve it from decay to the latest posterity.”

Three Cheers !!! were then given. When His Excellency directed the Architect to complete the building in exact accordance with a design in a firm and substantial manner, and perfect in all parts. The Chairman of the Committee then addressed His Excellency in a short speech, and returning thanks for the honor conferred on the occasion. His Excellency, directed one of the Chaplains to reply to the Chairman, and address the spectators generally. Another Anthem was sung. One of the Chaplains concluded the Ceremony with a short prayer and the usual blessing. The procession being resumed returned in reversed order, amidst the acclamations of the spectators.

* The Foundation-stone is usually composed of two separate pieces, hollow in the centre, which, when united, appear as one Stone.
The Colonist, 7 January 1834

Rear of building in previous photo, with church in background.

The new Orphan-school Church at New town is expected, we rejoice to learn, to be ready for the public performance of divine service on Christmas day. The edifice is a credit to every one connected with it, excepting the small pinnacles on the top of the tower, which we hope to see soon removed or replaced by something more appropriate and tasteful.
Hobart Town Courier, 4 December 1835

Rear of church.

Hobart Town Courier, 11 December 1835.

On Sunday week, Mr. Lee Archer, the Civil engineer, according to advertisement, opened the Church at New Town, when a very appropriate sermon, was preached on the occasion.
Colonial Times, 29 December 1835

Hobart Town Courier, 4 May 1838.

There was both an Anglican & a Catholic cemetery on site until the 1870s. Children from the orphans schools were buried here, but it was also the parish cemetery for residents of New Town. The stones were cleared in the 1960s, and many taken to Cornelian Bay.

Friends of the Orphan Schools has much better photos.
Transcriptions & photos of stones moved to Cornelian Bay
Volunteer historians in bid to shed light on orphan graves, unearthing Hobart history (ABC News)

This is the only extant memorial stone.

Some photos of cemetery when it was still in use:
1860s
c. 1860
c. 1860