Right!? I’ll be the first to admit I don’t understand context, windows, or whatever the limit is, but I can’t understand how these large language models have probably exabytes worth of data hoovered up from all over but they can’t remember what I told it either a minute ago or in another chat. I’ve taken to trying to get it to write markdown files to help its own memory, but I don’t know if those are counting against me for context or whatever either.
Not to pile on, but my real grape is the nanny nature of some of these, especially nannythropic. I can’t tell of our times they told me outright that “I won’t do XYZ. This is a hard limit you can’t get around it.” And I’m sitting here like dude you just created that test record to verify the piping you can’t go back through and delete it because it’s a record delete? Le sigh…
They key thing to know is that they don’t actually remember anything. They are completely stateless. Every question, every reply, every interaction needs to have the entirety of the conversation history fed back in to it. That’s actually what the “context” is. It would be like talking to a person, and every time you spoke you had to repeat everything you’ve ever said to them (and what they said to you!) before you could say something new. It’s insanely inefficient and why they require data centers the size of a small city. “Agents” are even worse, because they talk to themselves and each other in loops, but still have to repeat that context for every single interaction.
Right!? I’ll be the first to admit I don’t understand context, windows, or whatever the limit is, but I can’t understand how these large language models have probably exabytes worth of data hoovered up from all over but they can’t remember what I told it either a minute ago or in another chat. I’ve taken to trying to get it to write markdown files to help its own memory, but I don’t know if those are counting against me for context or whatever either.
Not to pile on, but my real grape is the nanny nature of some of these, especially nannythropic. I can’t tell of our times they told me outright that “I won’t do XYZ. This is a hard limit you can’t get around it.” And I’m sitting here like dude you just created that test record to verify the piping you can’t go back through and delete it because it’s a record delete? Le sigh…
They key thing to know is that they don’t actually remember anything. They are completely stateless. Every question, every reply, every interaction needs to have the entirety of the conversation history fed back in to it. That’s actually what the “context” is. It would be like talking to a person, and every time you spoke you had to repeat everything you’ve ever said to them (and what they said to you!) before you could say something new. It’s insanely inefficient and why they require data centers the size of a small city. “Agents” are even worse, because they talk to themselves and each other in loops, but still have to repeat that context for every single interaction.
Please do pile on. I agree with everything you said there. The whole point of these posts is so people can pile it on.