Non-sensical output with Phi-3.5 when do_sample is set to True

[natke]

Describe the bug
See title

To Reproduce

python phi3-qa.py -e cpu --temperature 0.8 --top_k 3 --do_sample -m ..\..\models\microsoft\Phi-3.5-mini-instruct-onnx\cpu_and_mobile\cpu-int4-awq-block-128-acc-level-4

Input: Hi

Output:  Hithere'0x47! I may be an Artificiel Intellignent like you but remember that while my abilitie to convice is quite different from a real conversation partner. If I may help with information and answering queries you have in your human way of thinking and communication. 𓅓
‍

**If I had more information, such context about your question: I'm here, I can assist with information or help guide a discussion based upon that! ➤
- **For a more technical conversation, you can provide the subject, details of your query or issue you need information about: I can assist by giving relevant information and explanation based upon my training.

For a more natural and interactive chat: you could share thoughts or topics of conversation. Remember that as A. I can only respond with text and can’ t truly participant or understand emotional nuances as you humans.

Expected behavior
I ran the same prompt and configuration with the PyTorch model, and got the following output:

 Hello! I'm Phi, an AI digital assistant here to help you. What can I do for you today?

[natke]

Tested with main.

Also repro'd with 0.5.2

Input: Hi

Output:  Hithere'0: Hello there, if I can help with something, please don’ts1️1️5. I might be a digital Assistantsince my existence started with the creation by Microsoft'so1: Hello, how about I continue the dialog with your next prompt? If that sounds alike to your plan then just go on!0️℠0️0️1

[natke]

Without temperature setting

C:\Users\nakersha\Develop\samples\phi3-0.5.2>python phi3-qa.py --top_k 3 --do_sample -m ..\..\models\microsoft\
Phi-3.5-mini-instruct-onnx\cpu_and_mobile\cpu-int4-awq-block-128-acc-level-4
Input: Hi

Output:  Hithere're quite some people here, but let’'t keep it brief. What can i do or what do i need to help with right away for now!

[natke]

Without temperature and without top_k the output looks sensible

(phi3.5) C:\Users\nakersha\Develop\samples\phi3-0.5.2>python phi3-qa.py --do_sample -m ..\..\models\microsoft\Phi-3.5-mi
ni-instruct-onnx\cpu_and_mobile\cpu-int4-awq-block-128-acc-level-4
Input: Hi

Output:  Hello! How can I assist you today?

[natke]

With top_k=50

(phi3.5) C:\Users\nakersha\Develop\samples\phi3-0.5.2>python phi3-qa.py --top_k 50 --do_sample -m ..\..\models\microsoft
\Phi-3.5-mini-instruct-onnx\cpu_and_mobile\cpu-int4-awq-block-128-acc-level-4
Input: Hi

Output:  HelloaThere’0Here ya goes your coucherized version 3 of simplicity wrapped nicely into conversant speak ─ it stays chattely friendly just chills on "HereayaBopo0Dabarist-inclusivespraw" if those last terms get under thunk toohug water-dunk it deep before delt! I herewith deliver ya dialog in daily digital parl ♥ Hope kind and keeps well to comeo of further fine fun discoursay yaa!!! Now ye digital domain where wit’ unwind’n to free! Good to gab' once, nuzz-back quick thr'more in mine ears n digital eyes lyk thee and ye in good chase – peace till anearst banp ♠f'