Elevated wood moisture contents during the thermal modification process have been shown to adversely affect the improvement in dimensional stability and hygroscopicity. This study tested the hypothesis that the effect of elevated wood moisture content is based on the impact of water on chemical reactions which determine the cell wall matrix stiffness. Samples of Scots pine sapwood (Pinus sylvestris L.) were thermally modified in saturated water vapor at different peak temperatures and durations starting either in oven-dry or in water-saturated state. For a given mass loss caused by the modification process, the improvement in maximum swelling and equilibrium moisture content was stronger for oven-dry samples. After removal of water-soluble degradation products, which caused a cell wall bulking effect, the maximum swelling even increased after modification in water-saturated state. Based on dynamic vapor sorption measurements, it was evidenced that the modification in oven-dry state increased the cell wall matrix stiffness which improved dimensional stability and hygroscopicity. Enhanced bond formation in the polymeric network, i.e., via condensation and cross-linking reactions during the treatment of oven-dry wood, is suggested as a cause for this increase in matrix stiffness. In contrast, the modification in water-saturated state enhanced the flexibility of the cell wall matrix, which increased the cell wall swelling and limited the improvement of hygroscopicity to the reduction in OH groups by removal of hemicelluloses. This enhanced matrix flexibility was potentially caused by predominant hydrolytic cleavage of bonds in case of water-saturated samples, evident from the chemical analysis of soluble degradation products, which increased the free volume between adjacent matrix polymers.