Post by Durance Gaming in Does AI Hatred Hurts Artists? My Experience...

Viewing post in Does AI Hatred Hurts Artists? My Experience...

I don't know if the OP still reads this but, here goes:

To the OP, your logic with AI is a little off--and I don't mean that in a bad way. You should definitely look into "linear algebra" and find out exactly what AI is doing. It will blow your mind.

I say this because I finally got tired of the arguing, I figured I would find out the truth for myself. By my own hands and eyes, I wanted to KNOW. So I spent 3 full days pouring over the mathematics. It all looked so simple, I thought it couldn't be true. So I worked out the high-dimensional mathematics for single 6x6 encoder neuron pass--by hand. Sure enough... You could literally do anything AI does by writing numbers on a piece of paper. The computer code is just automating the math problems at incredible speeds.

I can now literally, not maybe, I can literally build my own ChatGPT from a blank sheet of paper. I don't have to ask anyone anymore... I know.

IS AI ART STEALING?

Yes and no... but mostly? NO.

WHEN IS IT STEALING?

There is literally only 2 instances in which generating something with AI can be reasonably called "stealing" and that is:

1.) When you use a LORA or an Embedding to mimic a particular style.

2.) If you use or mention anyone else's work in the initial prompt. (Like using img2video to animate someone else's artwork)

There is no "output", only the input transformed. These words, all this terminology, it's just lies. Or wishful thinking or marketing hype or something. There is no "training", there is no "knowledge base", there is no "pattern recognition".

"AI" isn't even AI. What we do after the process, that is the "intelligence" aspect. In truth, an AI model does not have to "speak" in words like a chat bot. It does not have to "speak" in images or video. It can utilize anything. ANYTHING.

Because a feed-forward neural network is way cooler than anyone can possibly imagine if they don't already know.

It is a High-Dimensional Probability Drive. And whatever mathematical expression that navigates that hyperspace becomes forever changed.

It's not "intelligence". It's a navigation system. The only reason it even "cares" to do anything other than that is because after the navigation, we use a softMax function to reduce the final vector (coordinates) into small enough numbers that they can fit into probability calculations. Then, each final neuron is compared against the token dictionary like a massive game of Battleship.

E-3... Hit!

Whichever neuron produces a probability closest to the intended token, that's the one that is chosen. An AI's "temperature" is literally like saying how "hot or cold" it is allowed to be from the desired result.

So until someone steps up with the mathematics, ignore them. Because they do not know. It will help your heart and mind to know for yourself.

Like Reply

A Fals Fiction9 days ago(+1)

How you figured out the math is really cool. I know AI developers who would have liked to talk with ten years ago.

But to a couple of your points: “Is it stealing?” is a social problem, not a mathematical one. Your breakdown left out significant chunks about what stealing means.

A mathematician doesn’t get the final say on the topic, because “Does AI hatred hurt artists?” is a sociological question, not a mathematical problem.

Like Reply

Durance Gaming9 days ago

"Your breakdown left out significant chunks about what stealing means."

Agreed. That was intentional. I'm not gonna lie, I heard a little fan-girl scream in my head when you pointed that out, haha. There's no one to talk to about any of this stuff so I'm learning to just leave stuff out and try to say the bare minimum. Yet still, I ramble. lol

My point to the OP was not so much "math says this and that's that" as it was "math is a necessary part of the argument and until someone can truly comprehend that, there is no frame to structure the argument without chaos. It's like a triangle with no 3rd."

As for me, my view is my own perception--and perception is often subject to change. But in this stage in my understanding, there is too much to consider. The only way to honestly, truly begin the conversation (in my opinion) is to start by the numbers and then go from there.

Like Reply

redonihunter9 days ago

in which generating something with AI can be reasonably called "stealing"

Basically never. But since stealing sounds worse than plagiarism, people arguing against it, are using the worse sounding offense. Also, the stealing argument is usually used for the training data and not the output. Some models were trained with data that was not quite meant for public access. Plus the whole debate about giving consent for machines to learn from your work. All these are not stealing as such. That word has lost a lot of meaning over the years. If I steal something from you, you no longer have it.

There is no "training"

I am not sure how you mean that. Your hypderdimensional math is using matrix multiplication or something like that, is it not? How are the values in those matrixes filled? To calculate probabilities, you need data to derive the chances. And you need a mapping from natural language to whatever the system uses internally. Guess that's tokens.

But maybe you were very specific for this movie transformation thing. Maybe it does not even count as "ai" in the sense it is so often used these days. And that is "generative ai" and means using a large language model or something related to it. Whatever is under the hood there, those things are trained to have a "model" which would be the map to navigate your hyperspace.

There is no "output", only the input transformed.

i * m = o

There is an m. You need a model to transform your input into an output. And you are right in the way that if you keep record of your random seed and your exact parameters and the model, you will reproduce the exact same output. So in a way, it's just like "there is no 21", only a transformed 7.

Like Reply

Durance Gaming9 days ago

@redonihunter - So forgive me, but also correct me, on my jargon here. My entry point into higher mathematics was probably the most round-about way possible. I am still adapting my vocabulary. What I speak to here, I am referring to a sense of "invariability".

Just from my own perspective, the term "training" implies something I have not yet witnessed. The weight calibrations feel like exactly that, calibrations. When I sit an think about it, I suppose I was harsh on this term. I think I may have sort of "flash-formed" direct connections to the overall epiphany because it all hit me at once. When I look at modern AI, all I can seem to see is something like this:

Vectors ~= Coordinates

Matrix ~= Map

Weighted Matrix ~= Re-drawing or re-calibrating the map

Weights ~= Calibration

Bias ~= Threshold or Condition

Knowledge Base ~= Packaged Guidance System

Temperature ~= How Far You Are Allowed To Arrive From An Intended Destination

Then you add in the softMax? That looks like some kind of Sci-Fi high-dimensional probability drive where every "neuron" is like a different landing point and every "synapse" is the jump through hyperspace. Then, at the final layer, whichever little spaceship warped closest to the desired token space wins. I mean, we're talking passing the mathematical expression of a thing through high-dimensional space, nudged along by little weighted bumps and then locked into better and better patterns of navigation. This isn't even pattern recognition, this is something else entirely... Like... It reminds me of how water can erode rock and carve channels. Almost like the frozen weights represent some sort of "tunneling" of some kind.

But anyways, it appears that nothing ever actually touches the network itself. Like, no one ever let it upload files and chew up color patterns to mimic stuff with common pattern recognition. None of that stuff. You know what I mean?

And by "there is no output" I know that's a bit of a stretch but literally speaking, the output is the input transformed by this process. Take a ChatGPT conversation for example. Every token (let's say word) activated is a lightning-fast automation of this mathematical process. Every word it prints, it has fed the entire conversation back in over and over. Including the words it, itself generated.

Talking to a chat AI is not a conversation. It is a composition. This is why context management is so important. It's why it "lies". It's why it has "memory" problems... There is no memory. There's nothing to remember.

Like Reply

redonihunter9 days ago

Just from my own perspective, the term "training" implies something I have not yet witnessed. The weight calibrations feel like exactly that, calibrations.

Calibration is a feedback loop of fitting your model to the data. In statistics, that's a bad thing. For ai, that is how they are trained. The end result is the calibration values.

If you calibrate your model on weather patterns or if you calibrate it on how words appear in texts, those will be different values inside your hard coded model, that now can do this hyper dimensional math things to predit what comes next. Like an answer to a question. Or if it rains tomorrow.

But anyways, it appears that nothing ever actually touches the network itself.

There is no memory. There's nothing to remember.

Yes and no.

The model itself is not changed by using it. It really acts like a black box transformator where you put in something and get out a response specific to that something.

And it does have a memory for your session, but actually it does not have a memory. Just as you said, the previous text is just fed in again.

I recently saw a good explanation for this memory thing.

Like Reply

Durance Gaming8 days ago

Ah, I see the disconnect and I think it's my own fault. Everything you are saying is totally true, but I think we are talking about two different parts of the same overall thing. You said "the model". I know exactly what you mean with your weather example and it's kind of a big deal when you think about it. I always wondered how scientists did "studies" and "research" with AI with all it's current limitations but at the time, I was still stuck on thinking AI = Chat Bot or Image Generator or what-have-you. The input vectors can represent anything and whether or not the system can produce anything useful with the data--that's all down to the training. But this isn't what I'm trying to point at.

Not the model. The model is a combination of multiple processes. There is a surrounding framework involved.

I'm talking about the system at the core of it all. Not just ChatGPT or whatever, but any feed-forward neural network at all.

The very concept of neural network technology appears to be an invariable expression of the idea of "navigation". As a matter of fact, there's nothing stopping anyone from building a 4x4 input neural network specifically for navigating the probabilities of a 4D environment. Once you cross that 4th dimension, you are plotting points in higher-dimensional space. That, as far as I can tell, is the function of a feed-forward of a neural network. I hate to reduce such a neat thing to such a description but... It's like a fancy graphing calculator. You could do the math by hand yourself, but you wouldn't want to lol.

I hope that makes sense.

"And it does have a memory for your session, but actually it does not have a memory. Just as you said, the previous text is just fed in again."

This part, I gotta push back on--respectfully--but hear me out: That's not memory. That's repetition. Which why it only works in the same session. Like I said, it isn't a conversation it's co-creating a composition but you go once and ChatGPT goes ~34,000 times per word of it's "replies". It isn't talking to you or me, it's talking to the conversation as a whole. New session, different co-composed conversation.

I noticed this before I studied neural networks, when I got real into setting up my own models on my PC.

The more "memory" my models "remembered", the worse they performed. Tracking the source, sure enough, there was no real "memory" but instead just brute-force reapplication of data over and over and over, chewing up token limits and taking up context space.

In my opinion, "memory" is one of the worst things you could possibly give a modern AI in any business environment. Any access to irrelevant context, in any way, opens the door to compounding instability. The current way of faking "memory" is probably what leads to a lot of cybersecurity woes.

Like Reply

Durance Gaming8 days ago

Forgive me for the double reply, I read your message first then replied and am now watching the video you posted.

So far, immediately, there's a problem and maybe it's half-mine. This explanation is not what a "token" is. At least not in the system architecture sense. But now I am connecting the dots... Ok, so they call the limit a "token" limit because they charge you per token (navigation point) reached at the end of each full pass. Ok... Still watching...

Yep. A token isn't always a full word. Right. Still viewing...

Yeeeaaahhh dude! They were DECIDED and realizing that blew my mind. From my perspective, it's like writing a map of a place you've never been and then navigating probability until the landscape matches the map you drew... It's wild!

Boom--right there! Did you catch that at 4:04 to 4:10? "It doesn't care, they're just input tokens" That right there, the "it" he's talking about is the neural network. It doesn't care because it's a high-dimensional probability drive. The output is repurposed through the surrounding framework into "intelligence" and "behavior". This is what I am pointing to, the actual system itself before any arbitrary tokens are involved. The tokens are part of the framework, not the system itself... Still watching...

So, I don't want anyone to take this the wrong way... I'm not saying it's a conspiracy or anything... But the very next thing the guy does if you listen to his words and also pay attention to his body language: "But that's not even the interesting part" and he waves it away with a physical gesture. Before I taught myself linear algebra, I came from a 30+ year background of psychology and philosophy (b.s. in humanities) and that dude got uncomfortable when he realized where he was about to have to go to explain "why" the tokens don't matter. I don't think it's any kind of conspiracy, I think it might be a sense of embarrassment like the Tech world doesn't want to admit how far away from AGI modern AI architecture actually is.

I did watch the rest of the video but the rest of it revolves around token pricing... Okay... I see what's going on now. There is no real measurable way to price the usage of something like a modern AI because it's literally just automated calculations. So I can see the logic behind charging by the token.

Something isn't adding up, man... You don't need all these data centers and all this stuff to achieve something like ChatGPT. Even if everyone in the world was using it, yeah you'd need to scoop up all the RAM and processing power you could but you don't need all this storage space unless you're saving every single output of the neural network thought process and every single conversation.

I dunno... Anyways, that was a good video. I get it now, the token pricing versus actual tokens. Makes sense.

Like Reply

redonihunter8 days ago(+1)

In context the video was just a good explanation for the memory concept. Or rather the lack of memory. A session grows rapidly in token usage, because it has to be fed in again and again. The model is not fine tuned to you by usage, it "forgets".

It also explains, why excessive use of such a system needs so much computing hardware and electricity. And doing a little chatting with chatgpt does not do this. But trying to replace a software developer with ai will do that.

You seem to fuss about the word token or the definition. It's just the bits or atoms of language. A chatbot has to operate on those pieces, so measuring the usage of the bot is easiest done by counting how many pieces a session used.

Like Reply

Durance Gaming5 days ago(+1)

"You seem to fuss about the word token or the definition."

I do, forgive me--but there is a reason for that. It's because I had to comprehend them in order to build my own AI (not a chatbot, something completely different). A token is actually nothing on it's own. A single variable holding an array of strings--that's a token dictionary. The final layer gets softMaxed and produces an index list of probabilities that matches the same amount of indices in your array. Then there's that game of "Battleship" I spoke of. If index 304 sits at 0.989 then the "word" in index 304 of the word dictionary array is chosen by code logic--not the neural network itself.

There is no token use in the beginning. That's where the disconnect is. Tokens are for after the system is done with it's calculations, not before. There is an easy way to both prove and demonstrate this:

Recently, a man was able to get an AI to send him money from that AI's company. (he got caught) He did this by communicating in Morse Code to the chat agent. There are no Morse Code tokens. However, the AI was still able to understand the input. Why it worked as an adversarial prompt, that is also obvious when one stops considering the framework around the system and looks directly at how feed-forward neural networks do what they do.

So that's why I get hung up on it. The way the tech industry uses the word "tokens" to track monetization, it muddies the waters on what a token actually is and what function it performs in the overall architecture.

Check it out--you don't even need tokens at all. This is a short demo video for a completely different kind of AI that I am developing for video game NPCs (a multi-modal dual-input feed-forward/simplicial neural network architecture):

Like Reply

redonihunter5 days ago

This here needs to be done for an AI to "understand" you. https://en.wikipedia.org/wiki/Lexical_analysis

What the actual tokens are, is different for each model. But we are talking about large language models here. They deal with language and anything that can be seen as language. They are not unlike a compiler. But instead of converting source code into machine code, it converts natural language into whatever chunks the model operates on. Probabilisic meaning carrier units. Call them what you want. The process is called tokenisation and it was not invented for ai, so there already was a name to use, and that's tokens. That's how I understand the situation.

Also, each model can and probably will use different tokens and tokenisation.

Each token would influence the outcome, so it follows that counting used tokens would correlate with computational effort and therefore the costs. How accurate is this? I do not know. But since it correlates with the costs, it is reasonable to use this as a measuring unit for charging and access limitations.

Interesting bit about that Morse code thing. Seems those AI developers have forgotton to always keep user input in quotes all the way down. That was bascially an eval attack. That's what you get, when you let AI do things, instead of use them for information and generation. That was not gen ai use, that was ai agent usage.

There were or are some interesting experimental games that explore that concept a bit. Tricking the ai. I remember this one, that is on Itch. You can watch yt videos how people try to survive, which would be by tricking the ai (it's an escape horror game).

you don't even need tokens at all

I do not really grasp what you are doing there. But I have a hunch you embedded some type of optimisation problem into some kind of neural network. Which means you need no prior training data and not really a complex input to the system. If you have no natural language input, you do not need a tokenisation process. You also do not have a natural language output. And if you use a feedback loop to "learn" or optimise, you can use whatever internal native state descriptions you use for your optimisation or navigating the hyperspace of the problem territory your system tries to solve by probability.

AI was a really bad choice of words for the things that are now called AI. AI was and is used for anything a computer calculates. If it is a tad more comples than a calculator, you would say, the ai did this, the ai did that. For procedural processes. Especially in games and interactive systems.

But the ai systems everyone is talking about are gen ai systems. They are made for generating things. Like texts and images. The chatbot types are llm, large language models.

Like Reply

Durance Gaming5 days ago

Hang on, I don't mean to keep bothering you--I'm just trying to wrap my mind around this to make sure I'm not actually trying to say something else and am just too "new" to use the correct terminology.

So, let's strip away the model. Strip away the interface, the LLM, all of it. Strip it all down until all you have is the feed-forward neural network calculations. That... that right there is what I am talking about. Beyond the UI, beyond the code logic, beyond the framework that even makes the neural network useful at all... That's the part that screams at me for my attention.

And that's the part I am talking about. So maybe I'm not really talking about "AI" but actually neural network technology itself?

But whatever it is I noticed, it worked to make MUNCH.

22 vector "outer input"

8 vector "inner input"

The only embedding is in index 0 and index 7 of the 8 vector input. The higher context layer reflects data back into the system >> 20 higher context triangles to indices 1-20. But external world objects act like "errors" or "disembodied functions" by simply injecting additional data directly into the 22 vector. The disruptions in calculation produce different levels of activation.

The two separate vectors are combined into a 30 dimensional vector. MUNCH uses feed-forward and simplicial neural network calculations, so the triangles lighting up are not just for show. The triangle surfaces graph data from the physics motion of the physical node vibrations (simplicial graphing of frequencies). The feed-forward weighted matrices happen along the physical lines of the architecture. Hence the "embodied" dependency. It has to actually be physically arranged in a physics sandbox like Unity or Unreal.

The initial vector embedding is used as a sort of "ignition" because the system is a closed loop. Hence the "autonomy". The architecture cannot help but receive data simply by being online.

The movement is a result of the output. Technically speaking, this is probably closer to "Artificial Perception" rather than "Artificial Intelligence" but the philosophy of function is the same. So the output can be re-expressed directly into movement in the same way modern AI can re-express output into words, images, what-have-you. Just minus the softMax because we aren't worried about probability.

So nothing tells the agent to "move" except that it "perceives" something it's calculations express as conflict between the Convergence and Divergence nodes. That video, the weights were still defaulted to 0.15 and never changed because I did not turn on the dual-learning (outcome and association). So it can learn, but it does not need to yet. I think it will become necessary once I start coaxing out more complex behaviors and exposing it to different outside environments.

But to be honest, I don't fully know if this will work yet... I mean, it works obviously, but right now I am working with more combat-oriented tests and I am starting to suspect I may need to change a few things regarding how the inputs are handled. What's more, I can run this thing in a "bullet hell" simulation at 250+ fps and ~3ms, which is encouraging, but I think I might be outpacing my own comprehension at this point. So I've gotta slow down and move more deliberately now.

I hope at least some of this makes sense. I'm not trying to be like "I KNOW", it's more I'm trying to say "hey, look at this thing I noticed" by using my comprehension to do the same thing in a different way, so to speak.

So surely, I'm on to something here... Maybe I'm not articulating it correctly?

But yeah, MUNCH is fully functional solely due to it's architecture. Potential for harnessing behavior is purely an emergent result of the mathematical process. I haven't tried to get MUNCH to understand me yet, I've been focusing on understanding MUNCH so I can figure out the best way to train different behavioral profiles (saved weights).

Like Reply

redonihunter5 days ago

Hang on, I don't mean to keep bothering you

I am not bothered by this, but I think we are going offtopic here.

let's strip away the model. Strip away the interface, the LLM, all of it

The ai that is being talked about is generative ai. It is a misnomer, but that's how it is called. If your thingy does not produce output by considering an input in relation to a model that has been trained with the type of data you want as an output, then you do not talk about a gen ai system.

There are nieche gen ai applications that are trained on special things and thus generate special things. Like maths. Or even code, though that is not a nieche. And since you can consider a lot of things as "language", those chatbots can do a surprising amount of things.

I suggest you talk to a chatbot about this. They are good at definitions and disecting statements. If you give specs and dig and bother it, it will tell you what makes a system a gen ai system and what type of system you made. For example, the "ai" that is used for translation is not a gen ai. It's not a llm, but a nmt. But it does use a neural network. For such questions I prefer duck.ai , and I see myself using this for quick answers a lot, after failing to find it quickly with a regular search.

Like Reply

View more in thread

itch.io

Viewing post in Does AI Hatred Hurts Artists? My Experience...