As smart surveillance becomes popular in today’s smart cities, millions of closed circuit television (CCTV) cameras are ubiquitously deployed that collect huge amount of visual information. All these raw visual data are often transported over a public network to distant video analytic centers. This increases the risk of interception and the spill of individuals’ information into the wider cyberspace that causes privacy breaches. The edge computing paradigm allows the enforcement of privacy protection mechanisms at the point where the video frames are created. Nonetheless, existing cryptographic schemes are computationally unaffordable at the resource constrained network edge. Based on chaotic methods we propose three lightweight end-to-end (E2E) privacy-protection mechanisms: (1) a Dynamic Chaotic Image Enciphering (DyCIE) scheme that can run in real time at the edge; (2) a lightweight Regions of Interest (RoI) Masking (RoI-Mask) scheme that ensures the privacy of sensitive attributes on video frames; and (3) a novel lightweight Sinusoidal Chaotic Map (SCM) as a robust and efficient solution for enciphering frames at edge cameras. Design rationales are discussed and extensive experimental analyses substantiate the feasibility and security of the proposed schemes.