Crews made the discovery Friday morning during construction to connect a colonial-era building to the city's sanitary sewer system, the Albany Times Union reported.
"As soon as we saw it, I thought, 'Well, that can't be an old telephone pole,'" Joseph Coffey, head of the city's water department, told the Times Union.
The exact date of the nearly 6-foot piping is unknown, but is believed to be from the 1700s, when about 55 miles of wood water piping was installed. The city began using cast-iron piping in the early 1800s.
It appeared to have been partially destroyed before crews had found it from other construction work over the years, Coffey said.
A wood pipe previously found that dated to 1797 was so well-preserved it hangs in the water department's office, the Times Union reported.
"Although unlikely, there is a very remote outside chance that somewhere beneath some of the oldest parts of our City, a functioning wooden water main remains," the city's water department website reads.