A stochastic process is a mathematical model for the evolution through time of a particle that moves randomly through space. There are many different families of stochastic processes that are, now, well understood with varying degrees of success when building other mathematical models with applications in physics, biology, economics and engineering. Amongst some of the more commonly used stochastic processes are so-called Markov stochastic processes. For these processes, the future random evolution of the particle at any moment in time depends only on its current position and not on its historical path to date. This proposal aims to construct new families of Markov stochastic processes, never dealt with before, which respect a fundamental defining property, namely self-similarity. Roughly speaking, a stochastic process is self-similar when, after an appropriate re-scaling in space and time, the resulting random trajectory is an exact stochastic copy of itself. The importance of this new family of self-similar Markov stochastic processes will also be explored through their application in a number of different probabilistic settings. Specifically we shall: (a) Lay down the mathematical foundations, showing the existence of the self-similar Markov processes that we are interested in. (b) Explore some of their unusual properties with reference to the general theory of Markov processes. For example, we shall provide an understanding of the strange phenomenon that can occur with our self-similar family of processes in that their random trajectory "starts from infinity". (c) Look at stochastic differential equations which are "driven by a self-similar Markov stochastic processes". The former can be considered as a family of stochastic processes whose infinitesimal increments (or arbitrarily small-scale random movements) are determined by the infinitesimal increments of the latter. (d) Take advantage of the intimate relationship of self-similar Markov processes with other families of stochastic processes, known as Markov additive L'evy processes, to derive new results concerning the latter, which themselves can be fed into other applications. (e) Take all of the above knowledge and feed it into some concrete probabilistic applications known as optimal stopping problems. The latter have proved to be of prominence in a variety of scenarios which are pertinent to financial modelling.