Inferences about others’ emotions have largely examined judgments from static presentation of cues (e.g., facial “expressions”), ignoring temporally dynamic and integrative aspects. Here, we extend insights from predictive processing accounts, focusing on the hypothesis of temporal dependence, to review the current evidence and advance the investigation of the contextual and dynamic nature of emotion inferences. Consistent with constructionist views, we theorize that people dynamically form models of others’ emotions by drawing on varied representations that contribute to divergent predictions and, ultimately, inferences. We review evidence for the role of internal representations in emotion inferences and for variability in conceptual representations of emotions then outline how predictive processing accounts can inform new empirical questions about the temporal dynamics in emotion inference.