Meta artificial intelligence development mistakes, not using GPU in time leads to lagging behind rivals

According to news on April 25, an internal memo showed that in late summer 2022, Meta CEO Mark Zuckerberg (Mark Zuckerberg) convened the company’s executive team to analyze the company’s computing power for up to five hours, especially processing power. The capabilities of cutting-edge artificial intelligence .

The memo noted that despite Meta’s high-profile investments in artificial intelligence research and the company’s growing reliance on artificial intelligence to support its growth, the social media giant’s use of expensive artificial intelligence-optimized hardware and software systems for its main business has limited The speed is relatively slow, which hinders its ability to keep up with the pace of innovation as it scales. If it is to support AI work, Meta will need to “fundamentally change the way we design our physical infrastructure, our software systems, and our approach to providing a stable platform.”

The shakeup boosted Meta’s capex by about $4 billion per quarter, nearly doubling 2021, and caused it to suspend or cancel plans to build data centers at four locations, according to company disclosures.

Meta is facing severe financial difficulties, since last November, the company has been laying off unprecedented scale.

Meanwhile, the emergence of ChatGPT last November sparked a race among tech giants to release generative AI products. Generative AI requires massive amounts of computing power, adding to the urgency for Meta to expand, five sources said.

According to sources, Meta’s slowness in applying GPUs to artificial intelligence is one of the main problems. GPU chips are ideal for AI processing because they can perform a large number of tasks simultaneously, reducing the time it takes to process billions of pieces of data. GPU chips, however, are more expensive and chipmaker Nvidia controls 80 percent of the market and maintains a lead in the corresponding software, the sources said.

Until last year, Meta primarily used large numbers of commodity CPUs to run AI workloads. The CPU, the workhorse chip in the computing world, has dominated data centers for decades, but it hasn’t done well for artificial intelligence work.

This has led to rivals outpacing Meta in the field of AI. They use GPU chips and have better AI software, so they can develop new AI products and services faster.

Meta has also started using custom chips designed in-house to train the AI, according to two sources. But in 2021, this two-pronged approach proved to be slower and less efficient than building with a GPU at its core. GPU chips are also more flexible than Meta’s chips at running different types of models, the two sources said.

Later, as Zuckerberg pivoted the company to the metaverse, the lack of computing power left the company unable to deal with threats, including the rise of TikTok and Apple-led changes to ad privacy.

These issues have drawn the attention of former Meta board member Peter Thiel . In early 2022, he resigned without explaining why. At a board meeting before his departure, Thiel pointed out that Zuckerberg and his executives were too focused on growing the Metaverse and neglecting Meta’s core social media business, according to two people familiar with the matter. Leaving the company vulnerable to competitors like TikTok.

Meta had planned to launch a custom chip in 2022, but then backed away, ordering a multibillion-dollar Nvidia GPU chip instead that same year. At this point Meta has fallen behind peers such as Google, which in 2015 began deploying its own custom version of the GPU, called a TPU.

Meta next began to reorganize the artificial intelligence department, appointing two new engineers to lead it. During that time, dozens of executives have left Meta, nearly all of them replacing AI infrastructure leadership.

Next, Meta began retrofitting its data centers to accommodate the introduction of GPUs, chips that required more power and generated more heat and had to be clustered closely together with dedicated network connections between them. The effort required massive network capacity and new liquid cooling systems to manage the cluster’s heat, requiring them to be “completely redesigned.”

As the work progressed, Meta began internal plans to develop an even more ambitious new chip, similar to a GPU, capable of both training AI models and performing inference. The project is due to be completed around 2025, according to two sources.

Meta spokesman Jon Carvill declined to comment on the chip project.

While Meta is scaling GPUs, companies like Microsoft and Google are promoting commercial generative AI products, and Meta hasn’t made much real progress in this regard.

Meta’s chief financial officer acknowledged in February that the company isn’t currently devoting most of its computing power to generative work. “Essentially all of our AI capabilities go into ads, feeds, and Reels,” she said. Reels is Meta’s TikTok-like short video format popular with younger users.

Meta didn’t prioritize developing generative AI products until after ChatGPT launched in November, according to four sources. While the company’s AI research arm has been releasing technology prototypes since late 2021, it has not focused on turning them into products. However, as investor interest continued to mount, Zuckerberg announced in February a new high-level generative AI team that he said would “accelerate” the company’s work in this area. .

CTO Andrew Bosworth also said this month that generative artificial intelligence is the area where he and Zuckerberg spend the most time, predicting that Meta will launch new products this year.

Two people familiar with the new team said the team’s work was in the early stages of building a base model, a core program that could later be fine-tuned and adapted to different products.

Meta spokesman Carvill said the company has been developing generative AI products on different teams for more than a year. He confirmed that the work accelerated in the months after ChatGPT launched. (easy sentence)