Never forget the bribery of the criminals serving on the Supreme Court.

https://www.theguardian.com/us-news/2023/aug/11/clarence-thomas-undeclared-vacations-gifts-rightwing-donors

It is a shit show from top to bottom.

https://jacobin.com/2022/06/chief-justice-john-roberts-supreme-court-extremism-campaign-finance-corruption

Gimp

I won’t touch the proprietary junk. Big tech “free” usually means street corner data whore. I have a dozen FOSS models running offline on my computer though. I also have text to image, text to speech, am working on speech to text, and probably my ironman suit after that.

These things can’t be trusted though. It is just a next word statistical prediction system combined with a categorization system. There are ways to make an LLM trustworthy, but it involves offline databases and prompting for direct citations, these are different from Chat prompt structures.

The best years of YT before 2017, there was an advanced maker and DIY tech culture revolving around people sharing projects and content just to share it. That died. This all pro CC thing has an up side to some extent, but it also lobotomized YT. Peertube might eventually get to the same kind of utility level, but it needs a lot of time and momentum to get there.

I’m sure they will eventually try to force ID’s because it would be profitable for criminal data theft ads stalkers. This is all about corrupt money and exploitation. Billionaires are worthless parasites that have no right to exist in a Democratic system. Fuck the US fascist oligarchy party.

Fire fox, Fire fox;

Fuck you Google;

We’re throwing rocks.

Alpha bet, Alpha bet;

Farming data is,

stalking/theft.

Oobabooga is the main GUI used to interact with models.

https://github.com/oobabooga/text-generation-webui

FYI, you need to find checkpoint models. In the available chat models space, naming can be ambiguous for a few reasons I’m not going to ramble about here. The main source of models is Hugging Face. Start with this model (or get the censored version):

https://huggingface.co/TheBloke/llama2_7b_chat_uncensored-GGML

First, let’s break down the title.

• This is a model based in Meta’s Llama2.
• This is not “FOSS” in the GPL/MIT type of context. This model has a license that is quite broad in scope with the key point stipulating it can not be used commercially for apps that have more than 700 million users.
• Next, it was quantized by a popular user going by “The Bloke.” I have no idea who this is IRL but I imagine this is a pseudonym or corporate alias given how much content is uploaded by this account on HF.
• This model is based on a 7 Billion parameter dataset, and is fine tuned for chat applications.
• This is uncensored meaning it will respond to most inputs as best it can. It can get NSFW, or talk about almost anything. In practice there are still some minor biases that are likely just over arching morality inherent to the datasets used, or it might be coded somewhere obscure.
• Last part of the title is that this is a GGML model. This means it can run on CPU or GPU or a split between the two.

As for options on the landing page or “model card”

• you need to get one of the older style models that have “q(numb)” as the quantization type. Do not get the ones that say “qK” as these won’t work with the llama.cpp file you will get with Oobabooga.
• look at the guide at the bottom of the model card where it tells you how much ram you need for each quantization type. If you have a Nvidia GPU with the CUDA API, enabling GPU layers makes the model run faster, and with quite a bit less system memory from what is stated on the model card.

The 7B models are about like having a conversation with your average teenager. Asking technical questions yielded around 50% accuracy in my experience. A 13B model got around 80% accuracy. The 30B WizardLM is around 90-95%. I’m still working on trying to get a 70B running on my computer. A lot of the larger models require compiling tools from source. They won’t work directly with Oobabooga.

It’s no freaking mystery anywhere. Kids are too damn expensive because just living is too damn expensive. The real fix is massive land reform that absolutely murders the real estate bubble with a nuclear bomb. Regulate the availability of funds directly to the minimum wage. You work, you live a decent life with a good balance. Build dense housing with tight local communities and perfect transportation so we’re always in contact with people in our communities. Babies will be popping up like weeds.

An article about one of the poorest European countries is not really relevant. They don’t have the same zoning stagnation nonsense that makes housing unaffordable. The stupid incentives that exploded home loan amounts combined with 100 years without zoning reforms are the problem.

Have you seen the great gatspy with Wizard too? That’s what always comes up when mine goes too far. I’m working on compiling llama.cpp from source today. I think that’s all I need to be able to use some of the other models like Llama2-70B derivatives.

The code for llama.cpp is only an 850 line python file (not exactly sure how python=CPP yet but YOLO I guess, I just started reading the code from a phone last night). This file is where all of the prompt magic happens. I think all of the easy checkpoint model stuff that works in Oobabooga uses python-llama-cpp from pip. That hasn’t had any github repo updates in 3 months, so it doesn’t work with a lot of newer and larger models. I’m not super proficient with Python. It is one of the things I had hoped to use AI to help me learn better, but I can read and usually modify someone else’s code to some extent. It looks like a lot of the functionality (likely) built into the more complex chat systems like Tavern AI are just mixing the chat, notebook, and instruct prompt techniques into one ‘context injection’ (-if that term makes any sense).

The most information I have seen someone work with independently offline was using langchain with a 300 page book. So I know at least that much is possible. I have also come across a few examples of people using langchain with up to 3 PDF files at the same time. There is also the MPT model with up to 32k context tokens but it looks like it needs server machine ram in the hundreds of GB to function.

I’m having trouble with distrobox/conda/nvidia on Fedora Workstation. I think I may start over with Nix soon, or I am going to need to look into proxmox, virtualization or go back to an immutable base to ensure I can fall back effectively. I simply can’t track down where some dependencies are getting stashed and I only have 6 distrobox containers so far. I’m only barely knowledgeable enough in Linux to manage something like this well enough for it to function. - suggestions welcome

WizardLM 30B at 4 bits with the GGML version on Oobabooga runs almost as fast as Llama2 7B on just the GPU. I set it up with 10 threads on the CPU and ~20 layers on the GPU. That leaves plenty of room for a 4096 context with a batch size of 2048. I can even run a 2GB Stable Diffusion model at the same time with my 3080’s 16GBV.

Have you tried any of the larger models? I just ordered 64GB of ram. I also got kobold mostly working. I hope to use it to try Falcon 40. I really want to try a 70B model at 2-4 bit and see how its accuracy is.

Fedora workstation. Had been on Silverblue for years, but got a machine with Nvidia and didn’t want the extra headaches of SB

This is how I use the internet.

Cookies are not needed. They are shifting the security onto the user. Secure the information on the server just like any other business. Offloading onto the client is wrong. It leads to ambiguity and abuses. Visiting a store and a business on the internet are no different. My presence gives no right to my person, searches, or tracking in the location or outside of it. Intentions are worthless. The only thing that matters is what is possible and practiced. Every loophole is exploited and should be mitigated. The data storage and coding practices must change.

Nah, it should be the default state of affairs. Data mining is stalking and theft. It centers around very poor logic and decisions.

Things like browser cookies are criminal garbage. Storing anything on a user’s computer is stalking. Draw the parallel here; if you want to shop in any local store, I want you to first tell me everything you are wearing and carrying in a way that I can tell every possible detail about it, tell where you came from before you visited this store, where you are going next. They also want to know everything you looked at, how you react to changes in items presented to you and changes in prices. They want enough information to connect you across stores based on your mode of transportation, and have enough data to connect your habits over the last two decades.

Your digital existence should not be subject to slavery either. Ownership over ourselves is a vital aspect of freedom. Privacy is about ownership and dominion. If you dislike all the digital rights management and subscription services nonsense, these exist now as a direct result of people neglecting ownership. In the big picture, this path leads all of humanity back into another age of feudalism. The only difference between a serf and a citizen is ownership over property and tools. Everything happening right now is a battle over a new age of slavery. “You will own nothing and you will be happy about it.” Eventually this turns into 'Your grandchildren will own nothing and say nothing or they will be dead about it." What you do about your privacy now will be a very big deal from the perspective of future generations.

The first fix is to regain control of the rhetoric amongst ourselves. We need a mass movement to dominate the entire sociopolitical conversation. The real power is in the human tendency to gravitate to the same topics and towards the sensational/controversial. We need to outright reject all of the inflammatory nonsense by going meta every time. Never talk about the subjects themselves. Make every conversation about the underlying manipulation and distraction. People like the blood emerald African space Karen can not be allowed to dominate the social sphere. The entire world culture average atmosphere must pushed back. Right now, we are being lead about in chains visiting a prescribed list of dog piles to shove our noses in. The dog shit is not the problem; it’s the person holding the chains. Every morning we choose who gets to hold the chains. We like our noses shoved in dog shit so much we run back to it every day without realizing we are giving the chains to the problem person every time.

We really need to crush the parasitic billionaire problem that funds the right and leverages convenient idiots of the world by stoking their prejudice and hate. There will always be some innocent whipping boy to toss in a camp. Pitchfork politics is never about the victims it is about controlling the conversation and distraction. Like in the USA, the only reason for the stupidity is to prevent closing the loopholes that enable the corrupt oligarchy. No one can investigate and make reasonable laws when they are confronted by a constant barrage of absurd nonsense. It just needs to be so inflammatory that no one can dominate the conversation with the message I’m posting now to regain any sense of rational control. The USA has 10% of the laws and protections of any other western nation. This is why the parasites exist, what they are funding, and what is being exported world wide. Russia proved that power is all about leveraging the convenient idiots. Australia and Japan have solved this issue already. Anyone with tens of billions of dollars is a worthless subhuman criminal.

I just tried it a few hours ago. Indeed, it is quite good. I knew it when a NSFW prompt test on an uncensored model generated a stable diffusion picture of a robot skeleton and a snarky reply. Like, yay we finally have a bight spot with this one.

Not exactly. Stupid people with advanced tools make stupid outputs. Venture capital is pushing the propaganda sauce hard and a lot of stupid people are jumping on AI as a corporate trend. These are the idiots.

The tools are next level. We are on the edge of this tech becoming a really big deal. There are several research papers making breakthroughs regularly and making double digit percentile improvements on efficiency and accuracy. The reason it is a big deal is because you can have around 1/4 of the knowledge of the entire internet running on hardware as powerful as a current flagship phone. Sure it lies around 1/2 the time, but these are problems that are being solved. Like, the latest and greatest models are ancient history in a matter of 2-3 weeks. To be honest, have a casual conversation with an offline and uncensored LLM. You may know it is lying from time to time, but if you’re being objective, so are most humans you encounter under casual circumstances. The sociological function and potential value of this tech is pretty powerful medicine. Like if you need someone to talk to, or to talk out an issue in private, this is a way to make that happen.

What hardware does it take to run a 30B?

Watch this ~1hr long video when you get the chance. He’s using the stalkerware LLM, but he also describes how to use langchain to parse data like what you are wanting to do.

https://piped.video/watch?v=dXxQ0LR-3Hg&t=772