Re: Thanks for stirring up the Crackpots.
You are quite correct about the importance of density in relation to buoyancy but you are making a few mistakes and I quote you;
"the key question is not the total mass of the structure but the density of the structure compared to the density of the buoyant volume"
Unlike a SpaceShaft, (or otherwise known as an Atmospherically Buoyant Spar Platform Space Elevator,) a tower, like a building, has to stand on foundations or on a surface, hence it is not buoyant. You can have a very low density cylindrical structure filled with He (even with each of the building blocks somehow separated,) but it all will be just weight. Moreover, a very tall cylindrical section of the tower (or upright airbeam) will be proportionally less buoyant than a sphere of the same diameter. Let me explain.
Imagine an equatorial line on a balloon. This line will indicate the divide of the atmospheric weight above the balloon while below the equatorial line it will be the atmospheric buoyancy. If you now consider a buoyant tube with the same diameter as the balloon the flat surface areas at the top and the bottom of the tube will determine its buoyancy. Hence, the higher the top surface is to be, the lesser the atmospheric weight acting on the top area and this will directly influence the buoyancy at the bottom surface of the tube. And, when comparing both the balloon's equator and the tube's wall the atmospheric environment acting on both surfaces is nothing else but pressure, no weight or buoyancy. These top and bottom areas determine mainly the buoyant efficiency and "not" other factors such as density, viscosity, etc.. Furthermore, as you rightly indicated, it will also be necessary to take into account the weight of the structural section(s) to have the net value of buoyancy. So such buoyancy condition cannot exist with a tower but only with a SpaceShaft.
In the case of a SpaceShaft the design of the structure is such that it takes advantage of what we call cumulated buoyancy. On its own, this subject merits quite a bit of pen-and-paper so I will later, on a different posting, provide a directions to how to get a paper explaining the mechanism.
Regarding your indication to the necessary velocity to achieve orbital flight. This is indeed the real problem. However, this problem is not insurmountable either. My apologies because I will again use non mathematical descriptions to illustrate the solution.
Energetically speaking, any rocket system used for orbital missions consists of a vertical and a horizontal components. The vertical component consist of gravity and atmospheric drag opposing the thrust. While for the horizontal component; drag is negligible, especially at 20 km and above. In the case of any rocket system intended to access Space, the first stage is meant to take both "vertically" the second stage and the payload up to an altitude at which there is no atmospheric drag to affect the second stage flight. The fact is that, despite its colossal size, a first stage rocket never achieves supersonic speed. All it does is contribute a very small horizontal component of less than 300 km/hr to the second stage. And it is only after separation that the much smaller second stage takes over as to achieve escape velocity. And these all has to happen just between about 20 km of altitude and typically the Karman Line, at 100 km of altitude. However, most often, first stage (or boosters) separations happen at 20 km or below.
Therefore, what the real problem is for achieving the necessary (almost instantaneous) tangential speed. Unfortunately, there is "no will" to develop such engines by any private company. Not even SpaceX wants to do this (or the others) despite the losses with their really crazy idea of vertically landing a first stage rocket.