This paper presents a novel spatiotemporal optimization approach for maximizing the output power of an ocean current turbine (OCT) under uncertain ocean velocities. In order to determine output power, ocean velocities and the power consumed and generated by an OCT system are modeled. The stochastic behavior of ocean velocities is a function of time and location, which is modeled as a Gaussian process. The power of the OCT system is composed of three parts, including generated power, power for maintaining the system at an operating depth, and power consumed for changing the water depth to reach the maximum power. Two different algorithms, including model predictive control (MPC) as a model-based method and reinforcement learning (RL) as a learning-based method, are proposed to design the optimization structure, and comparative studies are presented. On one hand, the MPC based controller is faster in finding the optimal water depth, while the RL is also computationally feasible considering the required time for changing operating depth. On the other hand, the cumulative energy production of the RL algorithm is higher than the MPC method, which verifies that the learning-based RL algorithm can provide a better solution to address the uncertainties in renewable energy systems. Results verify the efficiency of both presented methods in maximizing the total power of an OCT system, where the total harnessed energy after 200 hours shows an over 18% increase compared to the baseline.