How tech giants cut corners to harvest data for AI

In late 2021, Open AI faced a supply problem.

- 

The artificial intelligence lab exhausted every reservoir of reputable English-language text on the internet as it developed its latest AI system. It needed more data to train the next version of its technology lots more. So OpenAI researchers created a speech recognition tool called Whisper.

It could transcribe the audio from YouTube videos, yielding new conversational text that would make an AI system smarter. Some OpenAI employees discussed how such a move might go against YouTube's rules, three people with knowledge of the conversations said.

YouTube, which is owned by Google, prohibits use of its videos for applications that are "independent" of the video platform.

Ultimately, an OpenAI team transcribed more than one million hours of YouTube videos, the people said. The team included Greg Brockman, OpenAI's president, who personally helped collect the videos, two of the people said.