Hilary Duff says she looks forward to heading back to her Texas hometown to help rebuild after Hurricane Harvey wreaked havoc on parts of the state late last month.

My whole family is there. My dad is there, my aunt and uncle, my cousin, my cousin’s kids. My goddaughter evacuated out of her house on a canoe with her two babies,” the 29-year-old Houston native told PEOPLE at her Baby2Baby Back-to-School event on Thursday at The Grove in Los Angeles.

“Some people have their whole neighborhoods piled in their front yard. It’s really terrible and I can’t wait to go back and do relief work and get my hands dirty, but I think a lot of water needs to clear out.”

The storm touched down Aug. 25 as a Category 4 hurricane, dumping punishing rain on the Texas Coastal Bend.

In the week since Harvey made landfall, thousands of residents in Texas and Louisiana found refuge in shelters across the states. Floodwaters overtook entire neighborhoods, damaging at least 49,000 homes in Texas, according to the Texas Department of Public Safety.

Duff said it “felt so unfair” watching news stories about the hurricane from sunny California.

“I felt so angry about it. Like, you don’t deserve this right now,” she said. “This is ongoing and this is going to be years of rebuilding ,so we need to give what we can and keep them in our thoughts and prayers, and I think there’s a lot of hurricanes on the horizon that are really scary, too.”

Hurricane Irma is closing in on Florida, and two others, Jose and Katia, are intensifying.

“That could cause even more damage,” Duff said. “So, this seems like an ongoing problem right now, and we need to just do what we can and all band together and spread love and kindness and give what you can.”

The Baby2Baby event was sponsored by GAP and Nike.

Some 785,000 people were part of mandatory evacuations in Texas and Lousiana, and more than 200,000 homes are still without power. Officials are now allowing a portion of evacuees to return to their properties, and the process of rebuilding their homes, and their lives, begins.

At least 60 people have died as a result of the storm.