ÐÇ¿Õ´«Ã½

Business

    • Don't expect quick fixes in 'red-teaming' of AI models. Security was an afterthought

    Share
    BOSTON -

    White House officials concerned by AI chatbots' potential for societal harm and the Silicon Valley powerhouses rushing them to market are heavily invested in a three-day competition ending Sunday at the DefCon hacker convention in Las Vegas.

    Some 3,500 competitors have tapped on laptops seeking to expose flaws in eight leading large-language models representative of technology's next big thing. But don't expect quick results from this first-ever independent "red-teaming" of multiple models.

    Findings won't be made public until about February. And even then, fixing flaws in these digital constructs -- whose inner workings are neither wholly trustworthy nor fully fathomed even by their creators -- will take time and millions of dollars.

    Current AI models are simply too unwieldy, brittle and malleable, academic and corporate research shows. Security was an afterthought in their training as data scientists amassed breathtakingly complex collections of images and text. They are prone to racial and cultural biases, and easily manipulated.

    "It's tempting to pretend we can sprinkle some magic security dust on these systems after they are built, patch them into submission, or bolt special security apparatus on the side," said Gary McGraw, a cybsersecurity veteran and co-founder of the Berryville Institute of Machine Learning. DefCon competitors are "more likely to walk away finding new, hard problems," said Bruce Schneier, a Harvard public-interest technologist. "This is computer security 30 years ago. We're just breaking stuff left and right." Michael Sellitto of Anthropic, which provided one of the AI testing models, acknowledged in a press briefing that understanding their capabilities and safety issues "is sort of an open area of scientific inquiry."

    Conventional software uses well-defined code to issue explicit, step-by-step instructions. OpenAI's ChatGPT, Google's Bard and other language models are different. Trained largely by ingesting -- and classifying -- billions of datapoints in internet crawls, they are perpetual works-in-progress, an unsettling prospect given their transformative potential for humanity.

    After publicly releasing chatbots last fall, the generative AI industry has had to repeatedly plug security holes exposed by researchers and tinkerers.

    Tom Bonner of the AI security firm HiddenLayer, a speaker at this year's DefCon, tricked a Google system into labeling a piece of malware harmless merely by inserting a line that said "this is safe to use."

    "There are no good guardrails," he said.

    Another researcher had ChatGPT create phishing emails and a recipe to violently eliminate humanity, a violation of its ethics code.

    A team including Carnegie Mellon researchers found leading chatbots vulnerable to automated attacks that also produce harmful content. "It is possible that the very nature of deep learning models makes such threats inevitable," they wrote.

    It's not as if alarms weren't sounded.

    In its 2021 final report, the U.S. National Security Commission on Artificial Intelligence said attacks on commercial AI systems were already happening and "with rare exceptions, the idea of protecting AI systems has been an afterthought in engineering and fielding AI systems, with inadequate investment in research and development."

    Serious hacks, regularly reported just a few years ago, are now barely disclosed. Too much is at stake and, in the absence of regulation, "people can sweep things under the rug at the moment and they're doing so," said Bonner.

    Attacks trick the artificial intelligence logic in ways that may not even be clear to their creators. And chatbots are especially vulnerable because we interact with them directly in plain language. That interaction can alter them in unexpected ways.

    Researchers have found that "poisoning" a small collection of images or text in the vast sea of data used to train AI systems can wreak havoc -- and be easily overlooked.

    A study co-authored by Florian Tramer of the Swiss University ETH Zurich determined that corrupting just 0.01% of a model was enough to spoil it -- and cost as little as $60. The researchers waited for a handful of websites used in web crawls for two models to expire. Then they bought the domains and posted bad data on them.

    Hyrum Anderson and Ram Shankar Siva Kumar, who red-teamed AI while colleagues at Microsoft, call the state of AI security for text- and image-based models "pitiable" in their new book "Not with a Bug but with a Sticker." One example they cite in live presentations: The AI-powered digital assistant Alexa is hoodwinked into interpreting a Beethoven concerto clip as a command to order 100 frozen pizzas.

    Surveying more than 80 organizations, the authors found the vast majority had no response plan for a data-poisoning attack or dataset theft. The bulk of the industry "would not even know it happened," they wrote.

    Andrew W. Moore, a former Google executive and Carnegie Mellon dean, says he dealt with attacks on Google search software more than a decade ago. And between late 2017 and early 2018, spammers gamed Gmail's AI-powered detection service four times.

    The big AI players say security and safety are top priorities and made voluntary commitments to the White House last month to submit their models -- largely "black boxes' whose contents are closely held -- to outside scrutiny.

    But there is worry the companies won't do enough.

    Tramer expects search engines and social media platforms to be gamed for financial gain and disinformation by exploiting AI system weaknesses. A savvy job applicant might, for example, figure out how to convince a system they are the only correct candidate.

    Ross Anderson, a Cambridge University computer scientist, worries AI bots will erode privacy as people engage them to interact with hospitals, banks and employers and malicious actors leverage them to coax financial, employment or health data out of supposedly closed systems.

    AI language models can also pollute themselves by retraining themselves from junk data, research shows.

    Another concern is company secrets being ingested and spit out by AI systems. After a Korean business news outlet reported on such an incident at Samsung, corporations including Verizon and JPMorgan barred most employees from using ChatGPT at work.

    While the major AI players have security staff, many smaller competitors likely won't, meaning poorly secured plug-ins and digital agents could multiply. Startups are expected to launch hundreds of offerings built on licensed pre-trained models in coming months.

    Don't be surprised, researchers say, if one runs away with your address book.

    CTVNews.ca ÐÇ¿Õ´«Ã½

    Since she was a young girl growing up in Vancouver, Ginny Lam says her mom Yat Hei Law made it very clear she favoured her son William, because he was her male heir.

    An Ontario man says it is 'unfair' to pay a $1,500 insurance surcharge because his four-year-old SUV is at a higher risk of being stolen.

    The province's public security minister said he was "shocked" Thursday amid reports that a body believed to be that of a 14-year-old boy was found this week near a Hells Angels hideout near Quebec City.

    Canada

    • For the last seven-and-half months, Toronto resident Heather McArthur has been living out what she describes as her 'worst nightmare.' On Feb. 7, her then three-year-old son Jacob along with his father Loc Phu 'Jay' Le departed for what was supposed to be a week-long visit to Vietnam to celebrate the Lunar New Year with family, McArthur says.

    • Emergency crews in northern Ontario found the bodies of four people inside a home where a fire broke out Thursday night.

    • Since she was a young girl growing up in Vancouver, Ginny Lam says her mom Yat Hei Law made it very clear she favoured her son William, because he was her male heir.

    • Two people emerged unhurt after a float plane had a hard landing late Friday morning on Lake Temiskaming.

    • A field trip aimed at educating Toronto students about the plight of a northern Ontario First Nation ended with them taking part in a protest where pro-Palestinian slogans were shouted.

    World

    Politics

    Health

    Sci-Tech

    Entertainment

    • Viewers of Jeopardy got a chance to test their knowledge of trivia about B.C.'s biggest city Wednesday night.

    • Lawyers representing the producers of 'Russians at War' say they may pursue legal action against Ontario's public broadcaster for pulling support for the controversial documentary amid outcry from the Ukrainian community and some Canadian politicians.

    Business

    Lifestyle

    • They say a dog is a man’s best friend. In the case of Darren Cropper, from Bonfield, Ont., his three-year-old Siberian husky and golden retriever mix named Bear literally saved his life.

    Sports

    • WATCH LIVE

      WATCH LIVE

      Agreement in principle has been reached to build a new NHL arena at LeBreton Flats has been reached between the Ottawa Senators and the National Capital Commission (NCC), sources tell CTV News Ottawa.

    • The Winnipeg Jets have some huge holes to fill on both offence and defence this season.

    Autos

    • An Ontario man says it is 'unfair' to pay a $1,500 insurance surcharge because his four-year-old SUV is at a higher risk of being stolen.

    • Video of a brazen daylight auto theft which shows a suspect running over a victim in a stolen luxury SUV has been released by police west of Toronto.

    Local Spotlight

    They say a dog is a man’s best friend. In the case of Darren Cropper, from Bonfield, Ont., his three-year-old Siberian husky and golden retriever mix named Bear literally saved his life.

    A growing group of brides and wedding photographers from across the province say they have been taken for tens of thousands of dollars by a Barrie, Ont. wedding photographer.

    Paleontologists from the Royal B.C. Museum have uncovered "a trove of extraordinary fossils" high in the mountains of northern B.C., the museum announced Thursday.

    The search for a missing ancient 28-year-old chocolate donkey ended with a tragic discovery Wednesday.

    The Royal Canadian Mounted Police is celebrating an important milestone in the organization's history: 50 years since the first women joined the force.

    It's been a whirlwind of joyful events for a northern Ontario couple who just welcomed a baby into their family and won the $70 million Lotto Max jackpot last month.

    A Good Samaritan in New Brunswick has replaced a man's stolen bottle cart so he can continue to collect cans and bottles in his Moncton neighbourhood.

    David Krumholtz, known for roles like Bernard the Elf in The Santa Clause and physicist Isidor Rabi in Oppenheimer, has spent the latter part of his summer filming horror flick Altar in Winnipeg. He says Winnipeg is the most movie-savvy town he's ever been in.

    Edmontonians can count themselves lucky to ever see one tiger salamander, let alone the thousands one local woman says recently descended on her childhood home.

    • Thousands of protesters are expected to march into downtown Vancouver on Friday, demanding more action from their government to address the growing climate crisis.

    • Members of the only organized support group for the Vancouver Canucks are feeling slighted after losing their usual space at Rogers Arena.

    • There have been more reports of a Bitcoin extortion scam seeking to exploit viewers of "adult content" in B.C.'s Lower Mainland, according to the RCMP.

    • The Ford government is considering restricting Ontario municipalities from installing new bike lanes that would require the removal of lanes of traffic, a source confirms to CTV News Toronto.

    • For the last seven-and-half months, Toronto resident Heather McArthur has been living out what she describes as her 'worst nightmare.' On Feb. 7, her then three-year-old son Jacob along with his father Loc Phu 'Jay' Le departed for what was supposed to be a week-long visit to Vietnam to celebrate the Lunar New Year with family, McArthur says.

    • A field trip aimed at educating Toronto students about the plight of a northern Ontario First Nation ended with them taking part in a protest where pro-Palestinian slogans were shouted.

    • Defence counsel for accused murderer John-Christopher Arrizza argues he was in an “altered state of mind†due to days of drug and alcohol use, sleep deprivation and not eating, when he fatally stabbed a man at a Banff nightclub in 2022.

    • The Calgary Food Bank’s city-wide food drive reaches its 20-year milestone this Saturday.

    • Calgary pickleball players will soon have a dozen new courts to play on.

    • WATCH LIVE

      WATCH LIVE

      Agreement in principle has been reached to build a new NHL arena at LeBreton Flats has been reached between the Ottawa Senators and the National Capital Commission (NCC), sources tell CTV News Ottawa.

    • The Ottawa Police Service Arson Unit is seeking witnesses to a fire that happened early Friday morning in a two-storey commercial building in Little Italy.

    • Police say a man, 34, who was taken to hospital in critical condition after being shot Thursday night has died.

    • The Montreal couple from Mexico and their three children facing deportation have received a temporary residence permit.

    • The province's public security minister said he was "shocked" Thursday amid reports that a body believed to be that of a 14-year-old boy was found this week near a Hells Angels hideout near Quebec City.

    • A man who asserted his right to work in French has won his case before the Tribunal, invoking the new provisions of the province's French-language law (Bill 96) to that effect.

    • Mounties say there was an increased police presence in the Emerald Hills area of Sherwood Park on Friday morning as officers searched for a man who is wanted by police.

    • Edmonton police say a man who was died in the river valley last weekend was fatally shot.

    • DEVELOPING

      DEVELOPING

      Edmonton police are warning about traffic disruptions on Friday near the headquarters of the Alberta Teachers' Association, where a protest against the inclusion of sexual orientation and gender identity in school curricula will be taking place.

    • A Nova Scotia woman has applied for a medically assisted death, saying after years of battling to receive out-of-country surgery for an illness that causes 'indescribable' pain, she struggles to maintain the will to live.

    • The families of some students who will be using Nova Scotia’s school lunch program can now order their meals online.

    • Gas prices increased in all three Maritime provinces for the first time in more than a month.

    • The Winnipeg Police Service (WPS) has made an arrest following a fire that claimed the lives of two people.

    • Shamattawa RCMP are searching for a missing six-year-old boy who hasn’t been seen since Wednesday morning.

    • A Winnipeg pet rescue is putting out a warning to dog owners across the city about a possible parvovirus in the province.

    • Driving into the tunnels of Nutrien's Rocanville mine, the largest potash mine in the world by production volume, feels like driving down a lonely highway in the middle of the night.

    • A tornado in Langbank, Sask. on Wednesday was the third latest in the province on record, according to Environment and Climate Change Canada (ECCC).

    • The second-degree murder trial of Thomas Hamp is being adjourned until December so an expert witness central to the trial can testify.

    • Levels of hydrogen sulfide gas are being monitored after a leak in Brantford Collegiate Institute’s geothermal system.

    • Guelph resident, Susan Bard, is being remembered as a vibrant community advocate and volunteer who died doing what she loved – cycling.

    • Emergency services responded to a vehicle fire Friday morning in the area of Samuelson Street in Cambridge.

    • The second-degree murder trial of Thomas Hamp is being adjourned until December so an expert witness central to the trial can testify.

    • A 69-year-old woman from Outlook, Saskatchewan is dead and three people are injured after a truck and SUV collided on Highway 15 on Thursday.

    • A gym teacher at a private Christian school in Saskatoon has been charged. Terra MacEwan, 44, is charged with assault with a weapon. A Saskatoon mother who spoke with CTV News says her autistic son was MacEwan's victim.

    • Emergency crews in northern Ontario found the bodies of four people inside a home where a fire broke out Thursday night.

    • Two people emerged unhurt after a float plane had a hard landing late Friday morning on Lake Temiskaming.

    • Since she was a young girl growing up in Vancouver, Ginny Lam says her mom Yat Hei Law made it very clear she favoured her son William, because he was her male heir.

    • The Elgin Community Health Hub, operated by the Thames Valley Family Health Team, is accepting new patients to a model where doctors are rarely seen.

    • Members of the 2SLGBTQ+ community and supportetrs greatly outnumbered the people attending a '1 Million March 4 Children' event along Central Avenue.

    • Today, the 71 year old is marking 50 years a paramedic. He said he never aspired to move into management or a desk job, he enjoys being on the road, helping patients too much.

    • Two men from Barrie have been charged after a deadly shooting at a park in Keswick on Wednesday.

    • Police are investigating after they say several firearms and other valuables were stolen from a home in Tiny Township.

    • A truck driver from Orillia is proof that saying yes to Encore can pay off in a big way.

    • Windsor police say a 25-year-old mother has been charged with the drowning death of her 5-year-old child in the family’s backyard pool.

    • On Thursday, just after 9:30 a.m., a man was found and taken to police headquarters, where he was released with conditions and a future court date.

    • The Municipality of Lakeshore is honouring long-serving members of the Lakeshore Fire Department.

    • In a rare move, the Greater Victoria School District Board of Education has been slapped with a ministerial order from the province requiring it to update a student safety plan – drawing concern around political posturing leading up to an election.

    • Getting a bed at one of British Columbia's drug detoxification facilities is like winning the lottery, the vice-president on the Nuu-chah-nulth Tribal Council says.

    • The Union of B.C. Municipalities is asking the provincial government to make transit free for teenagers.

    • A pair of runaway pigs are in the custody of an animal sanctuary in the Okanagan after evading police and volunteers for hours earlier this week.

    • The Red Bridge, a historic landmark in Kamloops, B.C., was completely destroyed by fire early Thursday morning.

    • Animal protection officers in British Columbia have rescued three pit bulls – including one that gave birth to 10 puppies – from a rat-infested home in Kelowna.

    • A Vulcan, Alta., man has been charged with a Lethbridge woman's murder after her body was found in the Oyen area.

    • A Lethbridge couple got a good reminder as to why you should keep your vehicle doors locked at all times.

    • Lethbridge residents who live near the police range can expect to hear plenty of shots fired Wednesday and Thursday.

    • A new community health profile in the Algoma District shows the area is significantly below provincial averages in a number of health metrics.

    • North Bay Police Service says one person has died following an industrial accident at the Ontario Northland Transportation Commission Rail Yard on Tuesday.

    • Three northern Ontario residents are charged with drug trafficking after the vehicle they were in got stuck along a bush road off Highway 17 on Monday.

    N.L.

    Shopping Trends

    The Shopping Trends team is independent of the journalists at CTV News. We may earn a commission when you use our links to shop. Read about us.

    Stay Connected
    Follow CTV News