In any case, most of the pyramids were built in the sixth dynasty. Some pyramid building was revived in later dynasties, but none in the New Kingdom (Dynasties 18 -20) of the Late Bronze Age. However, what you say about the pyramids holds also for other pharaonic building projects: They were mainly done by free, paid labor. In fact, Ramesses III had problems when his workers went out on strike because they hadn't been paid.
To give Exodus its due, Ex. 1:11 says that the Hebrew slaves built the store cities of Pithom and Ramesses. These would have been storage areas made of mud brick. The motif of Hebrew slaves building the pyramids is more a product of Hollywood than of the Bible.
And therein we have the first problem with its historicity.
Per-Rameses is the old Avaris, which existed for a looong time, and ended up capital of the Hyksos pharaohs. So if Josephus was right about the Hyksos connection, actually moves the date of the exodus to the time of their expulsion, which incidentally is also when Avaris is razed by the assaulting Egyptian armies.
The site would remain uninhabited until Rameses II rebuilds the city. So, ah-ha, we have a possible dating of when those Hebrew slaves were supposed to have done it.
Pithom -- i.e., Per-Atum, "house of Atum" -- is a whole different story. The site at Tell-el-Maskhuta doesn't quite put it at the same time. In fact the oldest layer that shows a city, is from the 6'th-7'th centuries BCE. You know, a whole half a millennium after Per-Rameses or the supposed band of Levites legging it from Egypt.
The only thing that seems to have been there before that is... a fort from the Hyksos times. Hmm.
There is much hype about the Tell-el-Maskhuta site, not the least because someone found a structure made of bricks without straw. Woohoo, that shows the Exodus is right, right? Well, it would, except it's from the Roman period
There are also some anachronistic statues from the Ramesside period there, but the sane working hypothesis is that they were moved there, a practice which was actually bog-standard at the time. Cities bought and sold and hauled statues and obelisks from old abandoned cities all the time. It doesn't mean the city was built by Ramesses II any more than the obelisks in Alexandria mean it was built by Thutmoses III

(ETA: or for that matter, Byzantium, now Istanbul, has an obelisk of Thutmoses III. So there we go, proof that Byzantium was built by the Egyptians in the 15'th century BCE

)
So what we really have is an anachronism. The book of Exodus, written circa 6'th century BCE, mentions a city that existed in their time, but almost certainly didn't at any time suitable for the events in the book.
If you will, it's as if I were to write a Spanish-colonial-age-themed novel, where stuff happens in Fordlandia. Actually that city didn't even exist until 1928 when Ford built it.
So, you know, one more reason to not take it as a very historical source. Those guys knew stuff around their time, but don't seem to actually know what happened half a millennium before.