Looking forward to this short workshop just getting started now:

Reflecting on Power and AI: The Case of GPT-3


OpenAI started in 2015 as a non-profit to help insure that there is viable open AI tech.

But in 2019 OpenAI needed more compute power and staff so they incorporated as part of Microsoft.

Here is a screen cap of the pricing model for their closed API (shared as part of this talk).

Ffrom the GPT-3 paper papers.nips.cc/paper/2020/file

"In collecting training data for GPT-3, we used the unfiltered distribution of languages reflected in internet text datasets (primarily Common Crawl)"

For those that are interested in why web archives matter, this is very significant.

Maybe someone has made something of this already, but it is strange to consider how models like GPT-3 will be trained on their own output, as they are used to generate text on the web, and then collected by projects like commoncrawl.


@edsu "trained on their own output"; that spirals inward. amiright? </part-sarcasm>

· · Web · 0 · 0 · 1
Sign in to participate in the conversation

The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!