Turbulent convection processes are ubiquitous in nature and technology. We study the structure of thermal and viscous boundary layers in three-dimensional Rayleigh-Bénard convection, a paradigm of these natural flows, for a range of Rayleigh numbers that spans 6 orders of magnitude by means of direct numerical simulations. The configuration is a plane layer of aspect ratio 4 with periodic boundary conditions at the sides – the configuration that comes close to the original configuration studied by Lord Rayleigh and others. Velocity fluctuations dominate the near-wall regions at all Rayleigh numbers. A global mean flow, which is a prerequisite for several theoretical models of turbulent heat transfer, is practically absent. Rather, the velocity field close to the wall can be decomposed into regions that are dominated by local, differently oriented and transient shear motion with shear-free regions in between. Thermal plumes are found to be organized in a self-similar hierarchical network, which gets coarser with increasing distance from the wall. The thermal boundary layers are marginally stable; the critical wavelength bounds the mean thermal plume spacing in the hierarchical network for all Rayleigh numbers from below. Our studies thus underline that the character of the near-wall layers in plane-layer Rayleigh-Bénard convection differs from those of canonical wall-bounded shear flows.