Teuken-7B-Base and Teuken-7B-Instruct: Towards European LLMs (2024)

When I ask smaller models a question in English, the model does well. When I ask the same model a question in Turkish, the answer is mediocre. When I ask the model to translate my question into English, get the answer, and translate the answer back to Turkish, the model again does well.

For example, I tried the above with Llama 3.3 70B, and asked it to plan me a 3-day trip to Istanbul. When I asked Llama to do the translations between English <> Turkish, the answer was notably better.

Anyone else observed a similar behavior?

Would be interesting to see whether they actually score better in leetcode questions when using python.

Presumably there is a lot more public info about, and code in Javascript and Python, hence this "preference"

Maybe the LLM preferring English is because of a similar phenomenon - it has been trained on mostly western, English speaking internet?

This likely plays a major - probably dominant - role.

It's interesting to think of other factors too though. The relatively concise syntax of those languages might make them easier for LLMs to work with. If resources are in any way token limited then reading and writing Spring Boot apps is going to be burdensome.

Those languages also have a lot of single file applications, which might make them easier for LLMs to learn. So much of iOS development for example is split across many files and I wonder if that affects the quality of the training data.