Robert Machin, 57, had just finished walking his two Boston terriers in the Bronx and was about to load them into his car when the truck appeared.
He recalls being whipped around, only to glimpse Ginger meeting her end in the sweeper's round brushes.
City sanitation officers described the dog's death as "rare and unfortunate".
But Mr Machin, a retired public transport worker, questioned whether the driver had been observing proper procedures.
The truck, he said, seemed to have been barrelling through the street at an unsafe speed.
"It happened so fast," he told the New York Daily News.
"It spun me around, and as it spun me around, I caught a last glimpse of her.
"I was devastated. I was completely dumbfounded and shocked. I mean, I just witnessed my dog sucked up into a street sweeper."
He said he had chased the truck for about two and a half blocks, shouting for the operator to stop.
The driver eventually came back but refused to turn off the whirring brushes that had crushed Ginger, until he arrived, the paper adds.
The Department of Sanitation offered its condolences to Mr Machin and his family but added: "It is important for all New Yorkers to remember to maintain the safety of their animals while walking city streets."
Ginger's master who, according to the New York Daily News, plans to contact the Humane Society and hire a lawyer, said his lost dog had been like family to him.
"My children are all grown up," he said.
"These two dogs, they're my life," he added, choking back tears.
Love the pun "suck it up"...hehehe!