The company will build a suite of products for customers to use to evaluate the dangers of their AI products.
Scale AI, an artificial intelligence company that helps other firms to train and label their own AI software, on Wednesday said it will launch a new AI safety lab intended to address emerging concerns over the responsible development of artificial intelligence.
Called the Safety, Evaluations and Analysis Lab, or SEAL, the unit will build a suite of products for Scale’s customers to use to evaluate possible risks posed in their AI deployments, as well as implement red team methods that seek to find faults and biases in software before it is launched. The lab’s work will include developing automated rating systems based on large language models, and conducting research about potential AI harms.
The lab is led by former Google researcher Summer Yue, who was previously a lead for Google’s Bard chatbot. In her role at Google, Yue worked on RFHL, which stands for “reinforcement learning from human feedback,” a technique for training AI that involves both human guidance as well as machine learning.
The initiative comes after the explosion of generative AI services like OpenAI’s ChatGPT and Google’s Bard and a backdraft of intense debate over AI safety. Last week, President Joe Biden issued an executive order on artificial intelligence, placing guardrails on companies developing the technology. One provision, for example, requires companies developing models that could pose national security risks to disclose them to the government. In the UK, meanwhile, Prime Minister Rishi Sunak held an AI safety summit in Bletchley Park where political and business leaders including US Vice President Kamala Harris and X’s Elon Musk gathered to discuss the new wave of technology.
“Every enterprise and government deploying an LLM application is facing the need to adopt and comply with the forthcoming standards and regulations that will be put in place,” Scale said in a blog post announcing SEAL. (Scale declined to make Yue available for an interview.)
Scale, co-founded in 2016 by 26-year-old CEO Alexandr Wang, got its start by helping companies to label data for autonomous driving before broadening its focus to include logistics, ecommerce and defense. Scale also runs the service Remotasks, which recruits freelancers around the world as data labelers. (Disclosure: Wang is slated to become a member of Forbes’ board when a planned sale of the company is completed.)
“To advance our knowledge and understanding of large language models (LLMs)–and therefore advance their capabilities–we must understand their weaknesses as much as their strengths,” Wang wrote in September, after Scale signed onto a voluntary commitment from the Biden administration on AI safety and security. “It is imperative for the tech industry to take proactive measures like these given the rapid pace of technological development.”