Crystal fields occur due to a potential difference between chemically different atomic species. In van der Waals heterostructures such fields are naturally present perpendicular to the planes. It has been realized recently that twisted graphene multilayers provide powerful playgrounds to engineer electronic properties by the number of layers, the twist angle, applied electric biases, electronic interactions, and elastic relaxations, but crystal fields have not received the attention they deserve. Here, we show that the band structure of large-angle twisted double bilayer graphene is strongly modified by crystal fields. In particular, we experimentally demonstrate that twisted double bilayer graphene, encapsulated between hBN layers, exhibits an intrinsic band gap. By the application of an external field, the gaps in the individual bilayers can be closed, allowing to determine the crystal fields. We find that crystal fields point from the outer to the inner layers with strengths in the bottom/top bilayer Eb = 0.13 V/nm ≈ -Et = 0.12 V/nm. We show both by means of first-principles calculations and low energy models that crystal fields open a band gap in the ground state. Our results put forward a physical scenario in which a crystal field effect in carbon substantially impacts the low energy properties of twisted double bilayer graphene, suggesting that such contributions must be taken into account in other regimes to faithfully predict the electronic properties of twisted graphene multilayers.