The 21-year-old was named by British media, who said he was originally from Syria and had been working at a fast food chicken restaurant. Previous attacks in London this year at Westminster Bridge, London Bridge and Finsbury Park as well as a blast at an Ariana Grande concert in Manchester killed dozens of people and injured more than 150.
Announcing the threat level was being lowered, Rudd said: "I would urge everybody to continue to be vigilant but not alarmed".
"It is inevitable that so-called Islamic State, or Daesh, will reach in and try and claim responsibility".
"A search is taking place at an address in the Newport following the arrest".
Assistant Commissioner Mark Rowley, National Lead for Counter Terrorism Policing provided an update on the Parsons Green Investigation and the threat level to the United Kingdom from global terrorism being reduced from critical to severe, he said:"The high pace and rapid progress of this investigation is continuing".
Another two suspects had earlier been arrested.
The entire neighborhood around their house was evacuated as a precaution when the search began Saturday.
Counter-terrorism officers have arrested two men after a crude homemade bomb sent flames shooting through a packed carriage at west London's Parsons Green underground station during the Friday morning rush hour, injuring 30 people although it apparently failed to detonate fully.
Counterterrorism police arrested a 25-year-old man in Newport, Wales, according to a statement by London's Metropolitan Police.
The apparent police interest in the shed comes two days after CCTV footage recorded just outside the house was obtained by ITV.
Police haven't charged the two men, aged 18 and 21, arrested in the case.
The level had been raised on Friday to the highest - critical - following the bucket bomb blast at Parsons Green Tube station that left 30 passengers injured.
London police say a second man has been arrested in connection with the London subway attack.
Authorities have reduced the terror threat level from "critical" to "severe", indicating another attack is highly likely but no longer believed to be imminent. He is being questioned at a south London police station but has not been charged or identified.