I did it with something like this.  Just use the old <xpath> tags like you do with forms, tree views etc.

<openerp>

<data>

<template id="report_custom_invoice_document" inherit_id="account.report_invoice_document">
    <xpath expr="//div[@class='page']" position="replace">
        <div class="page">

          <!-- etc -->

        </div>
    </xpath>
</template>

</data>

</openerp>