xAI accused of training Grok on real and AI-generated child pornography.
  • NaibofTabr@infosec.pubEnglish
    71·
    6 hours ago

    I just want to point out that there is a valid use case for such a model. The massive amount of images uploaded to the internet every hour has created an exponentially growing need for content moderation, and unfortunately that means some people have to spend time reviewing material flagged as CSAM. This leads to a lot of mental trauma: https://www.researchgate.net/publication/373947227_The_psychological_impacts_of_content_moderation_on_content_moderators_A_qualitative_study (not surprisingly).

    This is a fucking awful job, and there’s way too much for human moderators to handle. Most people who do this job burn out within a year or two. Supporting their work with automated recognition takes some of the strain off of them. In an ideal world we wouldn’t need humans to do this work at all, but that’s unlikely. This is a very rare case where I 100% support replacing human labor with machine learning as much as possible. I can’t really imagine a worse job, personally.

    There is a real benefit to training some models to automatically identify CSAM, but such a model should be specialized and only used internally for its intended purpose. It should never be publicly accessible. I suspect that what happened with Grok is an example of gross negligence/incompetence, where a model intended for CSAM identification was included in the overall Grok system.

    If that’s not the explanation, then this is way more disturbing because anything else would have been done with intent.

    • pulsewidth@lemmy.worldEnglish
      11·
      5 hours ago

      Occam’s Razor, I suspect X AI simply downloaded/scraped all porn they possibly could access on the internet to build their training data, and this included a large amount of child porn.

      Likely Elon-style factors:

      • time pressure. performed in a huge rush (he joined the LLM party late with massive FOMO),
      • zero internal pushback. Majority highly pressured immigrant/temporary staff (H1-B visas used a lot by Twitter in wake of Musk buyout)
      • zero importance placed on ethics / content review by leadership, likely even an ethics-negative internal culture. “Move fast, break things, fuck the government regs”.
      • loathe to pay licensing fees for materials or ‘waste time’ negotiating content-access contracts from legitimate porn businesses (“just scrape all the torrents and free sites you can find, the internet is full of free porn”).