A neural network-based framework enables precise reconstruction of the four-dimensional phase space of electron beams in particle accelerators. Utilizing a Transformer based Neural Network architecture and differentiable simulations, this approach captures complex features in beam distributions directly from experimental screen images which cannot be captured by measurement techniques used at SwissFEL. The reconstructed distribution provides access to the full (x, x', y, y') parameter space, enabling comprehensive analysis of beam properties including higher-order moments, correlations, and complex beam dynamics.