Because few people know what’s realistic for LLMs
Cryptography nerd
Because few people know what’s realistic for LLMs
You’re missing options like rail
Humans learn a lot through repetition, no reason to believe that LLMs wouldn’t benefit from reinforcement of higher quality information. Especially because seeing the same information in different contexts helps mapping the links between the different contexts and helps dispel incorrect assumptions. But like I said, the only viable method they have for this kind of emphasis at scale is incidental replication of more popular works in its samples. And when something is duplicated too much it overfits instead.
They need to fundamentally change big parts of how learning happens and how the algorithm learns to fix this conflict. In particular it will need a lot more “introspective” training stages to refine what it has learned, and pretty much nobody does anything even slightly similar on large models because they don’t know how, and it would be insanely expensive anyway.
Yes, but should big companies with business models designed to be exploitative be allowed to act hypocritically?
My problem isn’t with ML as such, or with learning over such large sets of works, etc, but these companies are designing their services specifically to push the people who’s works they rely on out of work.
The irony of overfitting is that both having numerous copies of common works is a problem AND removing the duplicates would be a problem. They need an understanding of what’s representative for language, etc, but the training algorithms can’t learn that on their own and it’s not feasible go have humans teach it that and also the training algorithm can’t effectively detect duplicates and “tune down” their influence to stop replicating them exactly. Also, trying to do that latter thing algorithmically will ALSO break things as it would break its understanding of stuff like standard legalese and boilerplate language, etc.
The current generation of generative ML doesn’t do what it says on the box, AND the companies running them deserve to get screwed over.
And yes I understand the risk of screwing up fair use, which is why my suggestion is not to hinder learning, but to require the companies to track copyright status of samples and inform ends users of licensing status when the system detects a sample is substantially replicated in the output. This will not hurt anybody training on public domain or fairly licensed works, nor hurt anybody who tracks authorship when crawling for samples, and will also not hurt anybody who has designed their ML system to be sufficiently transformative that it never replicates copyrighted samples. It just hurts exploitative companies.
Remember when media companies tried to sue switch manufacturers because their routers held copies of packets in RAM and argued they needed licensing for that?
https://www.eff.org/deeplinks/2006/06/yes-slashdotters-sira-really-bad
Training an AI can end up leaving copies of copyrightable segments of the originals, look up sample recover attacks. If it had worked as advertised then it would be transformative derivative works with fair use protection, but in reality it often doesn’t work that way
See also
Apple management has explicitly stated they do not want to support better compatibility between Android and iPhone, their response when asked what parents who buy cheap Androids for their kids should do it was to buy them iPhones. Many of the problems are very easy to fix on Apple’s side and keeping them problematic is intentional.
Math and formal logic are effectively equivalent and philosophy without conditional logic is useless. Scientifically useful philosophy is just “explorative logic” or something like it
Sounds like eminent domain talk if you think there’s enough suitable available homes already
https://comptroller.nyc.gov/reports/spotlight-new-york-citys-housing-supply-challenge/ 🤷
I don’t disagree with the rest, walkable cities are important, speculators shouldn’t be involved in housing, etc. But some places genuinely have a lack of available housing and the solution is to build away.
And where are they located, and why are they empty?
There’s your next big problem, a significant fraction of them aren’t where people want (or need) to be, or are vacation homes and don’t belong in these stats (unless you want to eminent domain them). Suburbs and ghost towns and remote regions pushes the average up.
https://todayshomeowner.com/general/guides/highest-home-vacancy-rates/
Wine/Proton on Linux occasionally beats Windows on the same hardware in gaming, because there’s inefficiencies in the original environment which isn’t getting replicated unnecessarily.
It’s not quite the same with CPU instruction translation, but the main efficiency gain from ARM is being designed to idle everything it can idle while this hasn’t been a design goal of x86 for ages. A substantial factor to efficiency is figuring out what you don’t have to do, and ARM is better suited for that.
It’s not that uncommon in specialty hardware with CPU instructions extensions for a different architecture made available specifically for translation. Some stuff can be quite efficiently translated on a normal CPU of a different architecture, some stuff needs hardware acceleration. I think Microsoft has done this on some Surface devices.
The problem here is that we don’t have real AI.
We have fancier generative machine learning, and despite the claims it does not in fact generalize that well from most inputs and a lot of recurring samples end up actually embedded in the model and can thus be replicated (there’s papers on this such as sample recovery attacks and more).
They heavily embedd genre tropes and replicate existing bias and patterns much too strongly to truly claim nothing is being copied, the copying is more of a remix situation than accidental recreation.
Elements of the originals is there, and many features can often be attributed to the original authors (especially since the models often learn to mimic the style of individual authors, which means it embedds information about features of copyrighted works from individual authors and how to replicate them)
While it’s not a 1:1 replication in most instances, it frequently gets close enough that a human doing it would be sued.
This photographer lost in court for recreating the features of another work too closely
https://www.copyrightuser.org/educate/episode-1-case-file-1/
Build more housing. That’s what you need to prioritize.
Hesitation is weaker, if you don’t know history of the instance but still want to record and issue. Censure is a stronger complaint, for example if the issues have been ongoing or you know the admin is misbehaving rather than just overworked or something like it
They’re literally meant to only be bounced off the ground, that’s some BS propaganda
He didn’t even make much cash, lol. After the initial attention everything dried up and now he’s got a hard time getting normal jobs, lol
Quantum mechanics still have endless ratios which aren’t discrete. Especially ratios between stuff like wavelengths, particle states, and more
Complex numbers, and a bunch more things too
And once again the problem is that there’s not much ensuring those models are correct, there’s not enough capacity available to finetune even a significant fraction of it.