This here needs to be done for an AI to "understand" you. https://en.wikipedia.org/wiki/Lexical_analysis
What the actual tokens are, is different for each model. But we are talking about large language models here. They deal with language and anything that can be seen as language. They are not unlike a compiler. But instead of converting source code into machine code, it converts natural language into whatever chunks the model operates on. Probabilisic meaning carrier units. Call them what you want. The process is called tokenisation and it was not invented for ai, so there already was a name to use, and that's tokens. That's how I understand the situation.
Also, each model can and probably will use different tokens and tokenisation.
Each token would influence the outcome, so it follows that counting used tokens would correlate with computational effort and therefore the costs. How accurate is this? I do not know. But since it correlates with the costs, it is reasonable to use this as a measuring unit for charging and access limitations.
Interesting bit about that Morse code thing. Seems those AI developers have forgotton to always keep user input in quotes all the way down. That was bascially an eval attack. That's what you get, when you let AI do things, instead of use them for information and generation. That was not gen ai use, that was ai agent usage.
There were or are some interesting experimental games that explore that concept a bit. Tricking the ai. I remember this one, that is on Itch. You can watch yt videos how people try to survive, which would be by tricking the ai (it's an escape horror game).
you don't even need tokens at all
I do not really grasp what you are doing there. But I have a hunch you embedded some type of optimisation problem into some kind of neural network. Which means you need no prior training data and not really a complex input to the system. If you have no natural language input, you do not need a tokenisation process. You also do not have a natural language output. And if you use a feedback loop to "learn" or optimise, you can use whatever internal native state descriptions you use for your optimisation or navigating the hyperspace of the problem territory your system tries to solve by probability.
AI was a really bad choice of words for the things that are now called AI. AI was and is used for anything a computer calculates. If it is a tad more comples than a calculator, you would say, the ai did this, the ai did that. For procedural processes. Especially in games and interactive systems.
But the ai systems everyone is talking about are gen ai systems. They are made for generating things. Like texts and images. The chatbot types are llm, large language models.