There is no way to edit rml in OpenOffice without the plugin. Furthermore, there exists no wysiwyg editor application for rml.
If you open an rml file with OpenOffice then you will see the rml source code. This is the same as opening the file in a plain text editor.
If you are a programmer then all of OpenERP is modifiable using a text editor. If you are a non-programmer and the plugin doesn't work for you (bug) then you can either learn what programming skills you need for the task or hire someone who already does. Either way, it's a large investment if all you want is to change a small thing.
Keep in mind that the rml files are located at the heart of OpenERP. They are not within a database. Sometimes, you can have a dummy database where you can tryout things before applying them in the production database. In the case of the rml files, if you make a change and save it then the change goes live to anyone using OpenERP from then on. If you make the change using a text editor then you will be going by trial and error - make a change, save it, generate a report within OpenERP, make a note of needed changes, start again. Under those conditions, the editing process is significantly longer than with a wysiwyg editor.
Some more information here:
http://stackoverflow.com/questions/5247452/does-a-wysiwyg-editor-for-report-labs-rml-exist
