I am not a programmer.
There should be an option, to just display text of speech in real time, without recording. Though many programs work by uploading to a cloud server the sound for analysis and transcription before downloading and display!
HOWEVER, I think you might find someone who could modify or enlighten you on this, in a forum of Smarthphone programmers. Specially Android.
IF the software automatically did record. BUT The location of that recording - the specified folder, had properties that did not allow changing, then the recording could not be saved. So would the transcription still occur. It might. Would the recording still be made. No. Unless the software had programming to write to a default or parent folder or to create a folder. But that would be further options that could be modified perhaps.
You should be prepared to explain that the recordings are deleted automatically, without specifying the time frame. No doubt some who observe your use of the phone, as for text display will express concern. You are a participant. They will observe this. That is your opportunity to mention it or to allow that UN HIDDEN behavior to be all the notification needed 
EDIT: I just checked my phone and I have a number of such transcribers that I never use on the iPhone. Found one called Voice Translator. If you set it to the same language for both input and output screens it acts like a transcriber. It says nothing about recording the audio. I am seeing that it has a voice speed adjustment, which if it did record, might be of interest to you… I dunno? Check it out.
