Skip to main content

Indie game storeFree gamesFun gamesHorror games
Game developmentAssetsComics
SalesBundles
Jobs
TagsGame Engines

This here needs to be done for an AI to "understand" you.  https://en.wikipedia.org/wiki/Lexical_analysis 

What the actual tokens are, is different for each model. But we are talking about large language models here. They deal with language and anything that can be seen as language. They are not unlike a compiler. But instead of converting source code into machine code, it converts natural language into whatever chunks the model operates on. Probabilisic meaning carrier units. Call them what you want. The process is called tokenisation and it was not invented for ai, so there already was a name to use, and that's tokens. That's how I understand the situation.

Also, each model can and probably will use different tokens and tokenisation.

Each token would influence the outcome, so it follows that counting used tokens would correlate with computational effort and therefore the costs. How accurate is this? I do not know. But since it correlates with the costs, it is reasonable to use this as a measuring unit for charging and access limitations.

Interesting bit about that Morse code thing. Seems those AI developers have forgotton to always keep user input in quotes all the way down. That was bascially an eval attack. That's what you get, when you let AI do things, instead of use them for information and generation. That was not gen ai use, that was ai agent usage.

There were or are some interesting experimental games that explore that concept a bit. Tricking the ai. I remember this one, that is on Itch. You can watch yt videos how people try to survive, which would be by tricking the ai (it's an escape horror game).

you don't even need tokens at all

I do not really grasp what you are doing there. But I have a hunch you embedded some type of optimisation problem into some kind of neural network. Which means you need no prior training data and not really a complex input to the system. If you have no natural language input, you do not need a tokenisation process. You also do not have a natural language output. And if you use a feedback loop to "learn" or optimise, you can use whatever internal native state descriptions you use for your optimisation or navigating the hyperspace of the problem territory your system tries to solve by probability.

AI was a really bad choice of words for the things that are now called AI. AI was and is used for anything a computer calculates. If it is a tad more comples than a calculator, you would say, the ai did this, the ai did that. For procedural processes. Especially in games and interactive systems. 

But the ai systems everyone is talking about are gen ai systems. They are made for generating things. Like texts and images. The chatbot types are llm, large language models. 

Hang on, I don't mean to keep bothering you--I'm just trying to wrap my mind around this to make sure I'm not actually trying to say something else and am just too "new" to use the correct terminology.

So, let's strip away the model. Strip away the interface, the LLM, all of it. Strip it all down until all you have is the feed-forward neural network calculations. That... that right there is what I am talking about. Beyond the UI, beyond the code logic, beyond the framework that even makes the neural network useful at all... That's the part that screams at me for my attention.

And that's the part I am talking about. So maybe I'm not really talking about "AI" but actually neural network technology itself?

But whatever it is I noticed, it worked to make MUNCH.

22 vector "outer input"

8 vector "inner input"

The only embedding is in index 0 and index 7 of the 8 vector input. The higher context layer reflects data back into the system >> 20 higher context triangles to indices 1-20. But external world objects act like "errors" or "disembodied functions" by simply injecting additional data directly into the 22 vector. The disruptions in calculation produce different levels of activation.

The two separate vectors are combined into a 30 dimensional vector. MUNCH uses feed-forward and simplicial neural network calculations, so the triangles lighting up are not just for show. The triangle surfaces graph data from the physics motion of the physical node vibrations (simplicial graphing of frequencies). The feed-forward weighted matrices happen along the physical lines of the architecture. Hence the "embodied" dependency. It has to actually be physically arranged in a physics sandbox like Unity or Unreal.

The initial vector embedding is used as a sort of "ignition" because the system is a closed loop. Hence the "autonomy". The architecture cannot help but receive data simply by being online.

The movement is a result of the output. Technically speaking, this is probably closer to "Artificial Perception" rather than "Artificial Intelligence" but the philosophy of function is the same. So the output can be re-expressed directly into movement in the same way modern AI can re-express output into words, images, what-have-you. Just minus the softMax because we aren't worried about probability.

So nothing tells the agent to "move" except that it "perceives" something it's calculations express as conflict between the Convergence and Divergence nodes. That video, the weights were still defaulted to 0.15 and never changed because I did not turn on the dual-learning (outcome and association). So it can learn, but it does not need to yet. I think it will become necessary once I start coaxing out more complex behaviors and exposing it to different outside environments.

But to be honest, I don't fully know if this will work yet... I mean, it works obviously, but right now I am working with more combat-oriented tests and I am starting to suspect I may need to change a few things regarding how the inputs are handled. What's more, I can run this thing in a "bullet hell" simulation at 250+ fps and ~3ms, which is encouraging, but I think I might be outpacing my own comprehension at this point. So I've gotta slow down and move more deliberately now.

I hope at least some of this makes sense. I'm not trying to be like "I KNOW", it's more I'm trying to say "hey, look at this thing I noticed" by using my comprehension to do the same thing in a different way, so to speak.

So surely, I'm on to something here... Maybe I'm not articulating it correctly?

But yeah, MUNCH is fully functional solely due to it's architecture. Potential for harnessing behavior is purely an emergent result of the mathematical process. I haven't tried to get MUNCH to understand me yet, I've been focusing on understanding MUNCH so I can figure out the best way to train different behavioral profiles (saved weights).

Hang on, I don't mean to keep bothering you

I am not bothered by this, but I think we are going offtopic here.

let's strip away the model. Strip away the interface, the LLM, all of it

The ai that is being talked about is generative ai. It is a misnomer, but that's how it is called. If your thingy does not produce output by considering an input in relation to a model that has been trained with the type of data you want as an output, then you do not talk about a gen ai system.

There are nieche gen ai applications that are trained on special things and thus generate special things. Like maths. Or even code, though that is not a nieche. And since you can consider a lot of things as "language", those chatbots can do a surprising amount of things.

I suggest you talk to a chatbot about this. They are good at definitions and disecting statements. If you give specs and dig and bother it, it will tell you what makes a system a gen ai system and what type of system you made. For example, the "ai" that is used for translation is not a gen ai. It's not a llm, but a nmtBut it does use a neural network. For such questions I prefer duck.ai , and I see myself using this for quick answers a lot, after failing to find it quickly with a regular search.

(+1)

Ahh, that's where I am indeed tripping up then! I'm being too broad in my assessment. I think you are pointing out an important distinction with Generative AI that I am not taking into account. That makes sense. And it's food for thought!

Thank you, man. I really appreciate you taking the time for this back-and-forth. It helped me differentiate some stuff I had blended together in my own comprehension, which gives me direction to tighten my own understanding. Right on.