By their very nature Spin Waves (SWs) enable the realization of energy efficient circuits as they propagate and interfere within waveguides without consuming noticeable energy. However, SW computing can be even more energy efficient by taking advantage of the approximate computing paradigm as many applications are error-tolerant like multimedia and social media. In this paper we propose an ultra-low energy novel Approximate Full Adder (AFA) and a 2-bit inputs Multiplier (AMUL). The approximate FA consists of one Majority gate while the approximate MUL is built by means of 3 AND gates. We validate the correct functionality of our proposal by means of micromagnetic simulations and evaluate the approximate FA figure of merit against state-of-the-art accurate SW, 7nm CMOS, Spin Hall Effect (SHE), Domain Wall Motion (DWM), accurate and approximate 45nm CMOS, Magnetic Tunnel Junction (MTJ), and Spin-CMOS FA implementations. Our results indicate that AFA consumes 43% and 33% less energy than state-of-the-art accurate SW and 7nm CMOS FA, respectively, and saves 69% and 44% when compared with accurate and approximate 45nm CMOS, respectively, and provides a 2 orders of magnitude energy reduction when compared with accurate SHE, accurate and approximate DWM, MTJ, and Spin-CMOS, counterparts. In addition, it achieves the same error rate as approximate 45nm CMOS and Spin-CMOS FA whereas it exhibits 50% less error rate than the approximate DWM FA. Furthermore, it outperforms its contenders in terms of area by saving at least 29% chip real-estate. AMUL is evaluated and compared with state-of-the-art accurate SW and 16nm CMOS accurate and approximate state-of-the-art designs. The evaluation results indicate that it saves at least 2x and 5x energy in comparison with the state-of-the-art SW designs and 16nm CMOS accurate and approximate designs, respectively, and has an average error rate of 10%, while the approximate CMOS MUL has an average error rate of 12.5%, and requires at least 64% less chip real-estate.