HI,
Please attach your db logs as per the following link it will list which files.
http://wiki.scn.sap.com/wiki/display/MaxDB/MaxDB+log+files
Also list more detailed error. It seems it says service in use have you restarted the server or services perhaps? is there perhaps already an instance of the db running?
Need more details.
Kind Regards,
Johan