different keyword but get same description response

[SongYao2]

Hi all, I have come across a problem and trying to seek your help. I am going to fine-tune parler-tts in Chinese, use Chinese dataset and choose chatglm-6b as the generating model for text description. The prompt format I am using basically corresponds to the English version. But when the annotation of the keywords are different, all my wav files get the same LM output. this has been bothering me for a long time, and I would like to know how to solve it.
（also I've checked that the keywords are all replaced correctly,and my keywords annotation are also chinese.)

[ylacombe]

Hey @SongYao2 , have you been able to solve this ? If not, what LLM do you use?

[SongYao2]

Hi, I'm surprised you noticed and answered my question. 😄🎉🎉 Actually I chose chatglm-6b in order to generate Chinese descriptions with more diverse syntax. And chatglm-6b is proposed by a team in China and supports both English and Chinese.

In the end, I didn't dwell on the data construction part, I specified a fixed description syntax, replacing keyword tags to the specified position. I used such a text description to train parler-tts from scratch with a 1000-hour Chinese dataset. the generated tones were very similar.

I did not perform text-controlled comparison experiments due to misreading and omission of words in the generated speech.

[ylacombe]

Thanks for the details!

I did not perform text-controlled comparison experiments due to misreading and omission of words in the generated speech.

I've realized that the tokenizer we used doesn't use byte fallbacks, which means that some tokens might be omitted. Expect a new release soon